On Fri, Jul 4, 2014, at 10:51 AM, Matthew Garrett wrote:

One of the benefits of the BLS approach is that it makes it more straightforward to handle multi-OS scenarios - people can drop config fragments into a defined location and still have them picked up by any conforming bootloader. That seems desirable, but syslinux doesn't appear to have a straightforward way to handle that.

Right, though it is an easy file to edit.

I agree that there are issues with the BLS as it currently stands (no support for chaining legacy OSes,

Yeah, I hadn't considered chainloading in my proposal...leaving it in the bootloader-specific config as the BLS suggests leaves us with os-prober for GRUB, but is that really that bad? I don't have a lot of experience with the non-Linux multiboot scenario though I admit.

the idea of using FAT for /boot) but it's more attractive to simply fix those bits than toss them out.

I agree this is worth pursuing.

I'm planning on submitting an F21 change covering this before the deadline (although, to be fair, the motivation here is mostly to get rid of the worst of os-prober rather than helping ostree - but if it works for you, that works for me)

Yep, I'll comment once I see it.

On Jul 7, 2014, at 2:43 PM, Matthew Garrett mjg59@srcf.ucam.org wrote:

Ok, here are my first attempts at modifying the spec to be more generally useful. Note that this means that gummiboot can no longer implement the full spec, but since gummiboot is also unable to do several other things that we need to do I don't see that as a problem in Fedora.

https://secure.freedesktop.org/cgit/www/log/MatthewGarrett/BootLoaderSpec.md...

* What's the distinction between $boot and $root in grub-core/commands/blscfg.c? GRUB_BOOT_DEVICE is $boot when GRUB_MACHINE_EFI. While GRUB_BOOT_DEVICE is $root on non-UEFI. Yet the revised spec says /org/freedesktop/bls goes on the ESP as $boot whether UEFI or non-EFI.

* Changes, 3rd bullet, "There's no reason to enforce VFAT as long as the filesystem is able to read it." I'm thinking "filesystem" was meant to be "firmware".

* What happens to BIOS Boot on BIOS systems where GPT is either preferred, or necessary to support > 2.2TB drives? Looks like such systems get BIOS Boot still for core.img, a FAT formatted ESP for /org/freedesktop/bls/entries, and a Linux boot partition for kernel+initramfs, grub, etc. And I'm not sure really what's gained by changing the legacy layout with one more partition, formatted as FAT, for just bootloader snippets.

* Spec doesn't address how a spec compliant installer should behave when installing along side an existing installation: Should bootability of the previous OS must be preserved? That seems a critical point of contention with most distros erring on the side of "oh, whatever" <hand waive> "user obviously doesn't care about the other OS that much, they're installing ours." I think as a default behavior it's pernicious.

* A consequence of BLS is multiple device resilient boot isn't possible because the configuration files go on a single ESP disk, which can't be on md raid (or btrfs). So it becomes a single point of failure.

On legacy BIOS systems, we've had support for this use case for years. The GUI installer also supports it out of the box, the system can boot degraded. Fedora's UEFI implementation breaks this by putting the grub.cfg on the ESP, rather than /boot/grub2 [1]; and BLS extends this to non-UEFI.

If it's an important use case, and I think it is for Workstation for sure and maybe even Server, on all archs, there needs to be a change. Alternatively we end up with long term BLS and non-BLS installs being supported, which I think is untenable both to the adoption of a BLS-like thing, as well as maintaining it.

* How about this: $bootload = The ESP, BIOS Boot, and MBR gap (or 0xEF partition). $bootbls = A neutral territory boot volume that definitely contains /org/freedesktop/bls/entries, optionally may contain kernel+initramfs. $boot = Distro specific boot (i.e. what's normally mounted at /boot on Fedora by default). $root = Distro root (i.e. what's normally mounted at /).

$bootload must be firmware readable, contains bootloader code and the least amount of information needed to find and read $bootbls. Any $bootload configuration file is static, shouldn't need modification, and is therefore easily propagated at install time to multiple disks; and to replacements when recovering from device failure.

$bootbls can be any filesystem and logical device supported by bootloader, installer, kernel. Rather than the spec proscribing filesystems, it leaves this question up to the use case and the parts needed to support it. So it could be FAT32, it could be md raid1 on XFS. The former would permit use of a future gummiboot [2], while the latter would probably require GRUB. Aside from the multiple device boot support, I'm troubled by FAT16/32 because of the corruptions I keep experiencing so I'm definitely not a fan of critical boot information being stored on this volume right now[3].

$boot may be superfluous because kernels are better suited for either $bootbls or $root, but the spec probably doesn't need to proscribe a distro specific /boot.

[1] RFE bug 1048999. Just today I realized this has been implemented by Ubuntu 14.04 LTS. Their ESP grub.cfg reads in its entirety: search.fs_uuid <volumeuuid> root hd0,gpt2 set prefix=($root) '/boot/grub' configfile $prefix/grub.cfg

[2] GRUB2 is both a floorwax and a dessert topping. Gummiboot is neither, but when it decides what it wants to be when it grows up it very clearly needs to support different volumes, even now it can be on the ESP while kernels are on the Extended Boot Partition defined by the discoverable partitions spec and he upstream BLS.

[3] Bug 1077917. Linux OS's seem to always mount it rw, which neither Windows nor OS X do. Fedora isn't fsck'ing it at mount time either, so I don't think anyone knows they've got a pending problem until boot breaks. And about 30% of my corruptions aren't fixable by fsck.vfat -a which means an fs_passno of 2 often still wouldn't fix things. I've lost count how many unrecoverable FAT32 ESP volumes I've collected while viciously testing over the last year, maybe 8-10? On the same configurations, one or two Btrfs corruptions only one of which was totally fatal, none for ext4 or XFS. So… something isn't right here. If FAT is this unreliable we probably shouldn't be persistently mounting it rw either, let alone it being used for dynamic changes to bootloader information.

Chris Murphy

Chris Murphy