Fedora 34 Change: Unify the GRUB configuration files location across all supported architectures (System-Wide Change proposal)
https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
== Summary ==
This change makes the GRUB configuration files layout to be consistent across all the supported architectures. Currently EFI is a special case since the GRUB configuration file and environment variables block are stored in the EFI System Partition (ESP) instead of the boot partition (or `/boot` directory if no boot partition is used).
== Owner ==
* Name: [[User:Javierm|Javier Martinez Canillas]] * Email: javierm@redhat.com
* Name: [[User:Gicmo|Christian Kellner]] * Email: ckellner@redhat.com
== Detailed Description ==
The GRUB configuration files layout on EFI platforms is not consistent with the other non-EFI platforms (e.g: x86 with legacy BIOS, ppc64le with Open Firmware). On platforms using EFI the GRUB configuration file (`grub.cfg`) and environment variables block (`grubenv`) are stored in the EFI System Partition (ESP) while for non-EFI platforms these are stored in the boot partition (or `/boot` directory if not boot partition is used).
The reason for this is that the path where the GRUB bootloader searches for its configuration file varies depending on the firmware interface used. In most cases, GRUB searches for it in the path set in the `$prefix` variable. The device is not specified in that case, GRUB just searches for a configuration in this path for every detected device. But on EFI, a special `$fw_path` variable is used instead. This variable specifies both the device and path from where the GRUB bootloader was loaded and these are used to search for the configuration file. This is done for security reasons, to make sure that the correct GRUB configuration file is used and not just the first one found in one of the detected devices.
Since the GRUB binary is located in the ESP, it expects to find the configuration file in that location as well. But this creates the mentioned inconsistency, because the GRUB configuration file has to be stored in `/boot/efi/EFI/fedora/grub.cfg` while for non-EFI platforms it has to be stored in `/boot/grub2/grub.cfg`.
This leads to a GRUB configuration layout that is confusing for users and error prone, since either the tools that are used to manage these files have to be aware of the differences or symbolic links used to hide the fact that the pats differ depending on the platform. Also, it creates artificial constraints on the OS installation due the differences in the configuration layout used. A system installed when using EFI won't be bootable if the firmware configuration is changed to boot using legacy BIOS instead, for example enabling the EFI Compatibility Support Module or moving the disk to a BIOS machine.
The proposal is to always store the `grub.cfg` and `grubenv` files in the `/boot/grub2/` directory, making `/boot/efi/EFI/fedora/grub.cfg` to only be a small configuration file that sets a different `$prefix` variable and loads the configuration file stored in `/boot/grub2/grub.cfg`.
The `$prefix` variable will be set to the device partition where `/boot/grub2/grub.cfg` is stored, using the partition filesystem's Universally Unique IDentifier (UUID). That way the correct GRUB configuration file will be loaded, making it as secure as the current approach where the configuration file in the ESP is used.
A drawback of the new approach is that the GRUB configuration will now depend on the boot partition (or `/boot` directory if no boot partition is used). But since the kernel and initramfs images are stored there too, the bootloader configuration already has this dependency anyways.
Note that the proposed configuration files layout is already used by the Fedora CoreOS Assembler (COSA) and OSBuild image builders. So this will make the systems installed with Anaconda to be consistent with the images generated by these.
== Benefit to Fedora ==
This change will not only simplify and make less confusing the GRUB configuration but also allow the following improvements:
* To have a consistent configuration across all the architectures using GRUB. * Allow the same installation to be booted with either EFI or legacy BIOS. * Use the same documentation and commands for all architectures instead of having EFI as a special case. * Make GRUB configuration tools more robust by not relying on symbolic links to be created and not having to handle platform specific cases. * Align with images generated by COSA and OSBuild on how the GRUB configuration files are used. * Align with other Linux distributions on how the GRUB configuration files are used.
== Scope ==
* Proposal owners: ** Modify Anaconda to generate the `grub.cfg` and `grubenv` files in `/boot/grub2/` instead of `/boot/efi/EFI/fedora/` for EFI platforms. ** Make Anaconda to generate the minimal GRUB config file in the ESP that sets the `$prefix` variable and loads the configuration file located in `/boot/grub2/grub.cfg`. ** Modify the grub2 package scriptlets to not generate the `/boot/grub2/grubenv` and `/etc/grub2-efi.cfg` symbolic links since these will not be needed anymore. ** Make any changes needed in tools that use these configuration files (`grubby`, `kernel-install` scripts, etc). * Other developers: ** The Anaconda developers will need to review and merge the patches to change where the GRUB configuration files are created. ** Test and watch for regressions.
* Release engineering:
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A
* Policies and guidelines: The policies and guidelines do not need to be updated.
* Trademark approval: No changes needed.
== Upgrade/compatibility impact ==
The changes will only be for new installations, existing systems will not be impacted and will continue using the grub.cfg and grubenv files that are located in the ESP.
== How To Test ==
* Install Fedora 34 and verify that the GRUB configuration is correct. * Check that the `/boot/efi/EFI/fedora/grub{.cfg,env}` files are not present. * Check that the `/boot/grub2/grub{.cfg,env}` files are present. * Enter the GRUB prompt by pressing 'c' from the GRUB boot menu and execute the following command: <pre>echo $prefix</pre> and verify that the prefix is set to the partition that contains the `/boot/grub2` directory. * Re-generate the GRUB configuration file with: <pre>grub2-mkconfig -o /boot/grub2/grub.cfg</pre> and verify that the GRUB configuration is still correct.
== User Experience ==
No visible changes are expected other than the configuration files being present in a different path. This should improve the overall user experience.
== Dependencies ==
None
== Contingency Plan ==
* Contingency mechanism: Revert the Anaconda and grub2 package scriptlets changes. * Contingency deadline: Beta Freeze * Blocks release? No * Blocks product? None
== Documentation == * Fedora CoreOS assembler script that creates the GRUB config file for EFI: https://github.com/coreos/coreos-assembler/blob/master/src/create_disk.sh * OSBuild org.osbuild.grub2 stage that creates the GRUB config file for EFI: https://github.com/osbuild/osbuild/blob/main/stages/org.osbuild.grub2
On Wed, 2020-12-30 at 14:53 -0500, Ben Cotton wrote:
== Benefit to Fedora ==
This change will not only simplify and make less confusing the GRUB configuration but also allow the following improvements:
- To have a consistent configuration across all the architectures using GRUB.
- Allow the same installation to be booted with either EFI or legacy BIOS.
- Use the same documentation and commands for all architectures
instead of having EFI as a special case.
- Make GRUB configuration tools more robust by not relying on symbolic
links to be created and not having to handle platform specific cases.
- Align with images generated by COSA and OSBuild on how the GRUB
configuration files are used.
- Align with other Linux distributions on how the GRUB configuration
files are used.
But:
== Upgrade/compatibility impact ==
The changes will only be for new installations, existing systems will not be impacted and will continue using the grub.cfg and grubenv files that are located in the ESP.
To me several of the benefits seem to not really be true, so long as this is the plan for upgrades.
* We wouldn't have a "consistent configuration" across everybody, really, because anyone who upgraded from pre-F34 would still have the old config; every bootloader debugging session ever would start by figuring out which case this was.
* We can't really use the same "documentation and commands" for the same reason. We either have to document both possibilities forever, or accept that our docs will be incorrect for anyone who upgraded from pre-F34.
* We can't really make the tools "more robust" in the way cited because they'll still have to handle both cases as long as both cases exist. If anything this makes them more fragile: the more divergent paths a tool has to support, the more likely it is something will break.
Hello Adam,
Thanks a lot for your feedback.
On Wed, Dec 30, 2020 at 9:22 PM Adam Williamson adamwill@fedoraproject.org wrote:
[snip]
== Upgrade/compatibility impact ==
The changes will only be for new installations, existing systems will not be impacted and will continue using the grub.cfg and grubenv files that are located in the ESP.
To me several of the benefits seem to not really be true, so long as this is the plan for upgrades.
- We wouldn't have a "consistent configuration" across everybody,
really, because anyone who upgraded from pre-F34 would still have the old config; every bootloader debugging session ever would start by figuring out which case this was.
- We can't really use the same "documentation and commands" for the
same reason. We either have to document both possibilities forever, or accept that our docs will be incorrect for anyone who upgraded from pre-F34.
- We can't really make the tools "more robust" in the way cited because
they'll still have to handle both cases as long as both cases exist. If anything this makes them more fragile: the more divergent paths a tool has to support, the more likely it is something will break. --
These are all fair points. My worry is that trying to switch to the new configuration on upgrades could lead to issues for people that have custom GRUB configs. That was the case when we did the switch to using BLS snippets and I don't really want to repeat that experience for users.
That's why I went with the conservative approach of only do this for new installs, to prevent breaking users configuration (or even worse, their booting). Maybe a middle ground could be to provide a tool for users to do the switch and make it opt-in?
Another option is to stick with the status quo but then we will never be able to attempt improving this.
Best regards, Javier
On Wed, Dec 30, 2020 at 10:08:31PM +0100, Javier Martinez Canillas wrote:
That's why I went with the conservative approach of only do this for new installs, to prevent breaking users configuration (or even worse, their booting). Maybe a middle ground could be to provide a tool for users to do the switch and make it opt-in?
This seems like a good approach to me. Documentation would have to remain doubled for a couple of releases, and then we could move to "if you've upgraded your system and not converted the config, here's how to do that now [link]" after that.
However, if there's a chance the tool will break (and that chance is the only reason to not ... just do it), how hard will it be for people to recover?
On Wed, Dec 30, 2020 at 2:09 PM Javier Martinez Canillas javier@dowhile0.org wrote:
On Wed, Dec 30, 2020 at 9:22 PM Adam Williamson adamwill@fedoraproject.org wrote:
- We wouldn't have a "consistent configuration" across everybody,
really, because anyone who upgraded from pre-F34 would still have the old config; every bootloader debugging session ever would start by figuring out which case this was.
These are all fair points. My worry is that trying to switch to the new configuration on upgrades could lead to issues for people that have custom GRUB configs. That was the case when we did the switch to using BLS snippets and I don't really want to repeat that experience for users.
That problem was the result of quite old core.img in the MBR gap (or BIOS Boot partition). As that change simultaneously depended on shipping a new GRUB module without a way to update the core.img with up-to-date GRUB modules, there was a known weak spot that we even knew of in advance.
Upgrades of customized configurations that deviate significantly from defaults aren't supported. It's best effort. We can't be blocking on people's customizations.
I think we can come pretty close to atomically renaming
/EFI/fedora/grub.cfg /EFI/fedora/grub.cfg.old /EFI/fedora/grub.cfg.new to /EFI/fedora/grub.cfg
And at least ensure the user can boot the old one, but even this I think is pretty unlikely. It's really a teeny tiny window of failure opportunity. And based on my reading of rename() if the files are already all present, and all we're doing is renaming them, there shouldn't be a case where grub.cfg is either missing entirely or zero bytes.
But dm-log-writes can help confirm or deny this. What I don't know is if this can be done with bash. The convert script probably needs to be done in C. Or at least the rename and sync parts.
On Wed, Dec 30, 2020 at 5:01 PM Chris Murphy lists@colorremedies.com wrote:
On Wed, Dec 30, 2020 at 2:09 PM Javier Martinez Canillas javier@dowhile0.org wrote:
On Wed, Dec 30, 2020 at 9:22 PM Adam Williamson adamwill@fedoraproject.org wrote:
- We wouldn't have a "consistent configuration" across everybody,
really, because anyone who upgraded from pre-F34 would still have the old config; every bootloader debugging session ever would start by figuring out which case this was.
These are all fair points. My worry is that trying to switch to the new configuration on upgrades could lead to issues for people that have custom GRUB configs. That was the case when we did the switch to using BLS snippets and I don't really want to repeat that experience for users.
That problem was the result of quite old core.img in the MBR gap (or BIOS Boot partition). As that change simultaneously depended on shipping a new GRUB module without a way to update the core.img with up-to-date GRUB modules, there was a known weak spot that we even knew of in advance.
Upgrades of customized configurations that deviate significantly from defaults aren't supported. It's best effort. We can't be blocking on people's customizations.
I think we can come pretty close to atomically renaming
/EFI/fedora/grub.cfg /EFI/fedora/grub.cfg.old /EFI/fedora/grub.cfg.new to /EFI/fedora/grub.cfg
And at least ensure the user can boot the old one, but even this I think is pretty unlikely. It's really a teeny tiny window of failure opportunity. And based on my reading of rename() if the files are already all present, and all we're doing is renaming them, there shouldn't be a case where grub.cfg is either missing entirely or zero bytes.
But dm-log-writes can help confirm or deny this. What I don't know is if this can be done with bash. The convert script probably needs to be done in C. Or at least the rename and sync parts.
Oh I forgot the part about the convert script testing for custom grub.cfg. Maybe it's possible to parse it first, and if it's custom, bail. If it's non-custom then convert it.
And a variation where we have an opt-in for this convert script for Fedora 34 cycle. And then ship it and do the conversion in Fedora 35.
I think either never fixing this, or never updating systems to the "new way" are both untenable. We saw with the BLS switch many users depend on doing in place upgrades. Many were pushing 4 or more years.
On Wed, Dec 30, 2020 at 05:08:03PM -0700, Chris Murphy wrote:
Upgrades of customized configurations that deviate significantly from defaults aren't supported. It's best effort. We can't be blocking on people's customizations.
I think we can come pretty close to atomically renaming
/EFI/fedora/grub.cfg /EFI/fedora/grub.cfg.old /EFI/fedora/grub.cfg.new to /EFI/fedora/grub.cfg
And at least ensure the user can boot the old one, but even this I think is pretty unlikely. It's really a teeny tiny window of failure opportunity. And based on my reading of rename() if the files are already all present, and all we're doing is renaming them, there shouldn't be a case where grub.cfg is either missing entirely or zero bytes.
But dm-log-writes can help confirm or deny this. What I don't know is if this can be done with bash. The convert script probably needs to be done in C. Or at least the rename and sync parts.
Oh I forgot the part about the convert script testing for custom grub.cfg. Maybe it's possible to parse it first, and if it's custom, bail. If it's non-custom then convert it.
And a variation where we have an opt-in for this convert script for Fedora 34 cycle. And then ship it and do the conversion in Fedora 35.
I think either never fixing this, or never updating systems to the "new way" are both untenable. We saw with the BLS switch many users depend on doing in place upgrades. Many were pushing 4 or more years.
I think conversion script would be needed for people doing upgrades. But first of all - documentation! It should be clear what GRUB expectations are, where the config file should be and so on. BLS conversion was hard because it was undocumented. One of the crucial scripts (grubby? install-kernel?) has been changing behaviour upon existence of some file or directory in /boot (was is loader/?). It was undocumented and so counter-intuitive that cannot recall details after merely two years.
Right now the best information about how Fedora boots and what happens on kernel installation can be founds on AdamW's blog. This is not perfect :(
Hello Tomasz,
On Thu, Dec 31, 2020 at 10:55 AM Tomasz Torcz tomek@pipebreaker.pl wrote:
[snip]
I think either never fixing this, or never updating systems to the "new way" are both untenable. We saw with the BLS switch many users depend on doing in place upgrades. Many were pushing 4 or more years.
I think conversion script would be needed for people doing upgrades. But first of all - documentation! It should be clear what GRUB
Agree.
expectations are, where the config file should be and so on. BLS conversion was hard because it was undocumented. One of the crucial scripts (grubby? install-kernel?) has been changing behaviour upon existence of some file or directory in /boot (was is loader/?). It was undocumented and so counter-intuitive that cannot recall details after merely two years.
Right now the best information about how Fedora boots and what happens on kernel installation can be founds on AdamW's blog. This is not perfect :(
Yes, we need to close the documentation gap.
The process is still quite complex. I think the long term goal should be to align with the https://systemd.io/BOOT_LOADER_INTERFACE/ and https://systemd.io/BOOT_LOADER_SPECIFICATION/ (and extend those specs to cover all the missing bits for the bootloaders used by Fedora) in order to make the interface between the bootloader and OS well defined. That will make easy to mix and match bootloaders and OS, but there is still a lot of work to do in order to achieve that.
-- Tomasz Torcz 72->| 80->| tomek@pipebreaker.pl 72->| 80->|
Best regards, Javier
Hello Chris,
Thanks a lot for the comments.
On Thu, Dec 31, 2020 at 1:02 AM Chris Murphy lists@colorremedies.com wrote:
[snip]
That problem was the result of quite old core.img in the MBR gap (or BIOS Boot partition). As that change simultaneously depended on shipping a new GRUB module without a way to update the core.img with up-to-date GRUB modules, there was a known weak spot that we even knew of in advance.
Correct.
Upgrades of customized configurations that deviate significantly from defaults aren't supported. It's best effort. We can't be blocking on people's customizations.
I agree with you but users have different expectations I think. If they deviated from the default and that didn't cause issues for them after a system wide upgrade, they expect that to be the case on the next update.
I think we can come pretty close to atomically renaming
/EFI/fedora/grub.cfg /EFI/fedora/grub.cfg.old /EFI/fedora/grub.cfg.new to /EFI/fedora/grub.cfg
And at least ensure the user can boot the old one, but even this I think is pretty unlikely. It's really a teeny tiny window of failure opportunity. And based on my reading of rename() if the files are already all present, and all we're doing is renaming them, there shouldn't be a case where grub.cfg is either missing entirely or zero bytes.
But dm-log-writes can help confirm or deny this. What I don't know is if this can be done with bash. The convert script probably needs to be done in C. Or at least the rename and sync parts.
Yes, for me is less about this tiny window but more about users having modifications in their GRUB config file that will be overwritten when generating a new one. For example in the BLS conversation some users update their kernel cmdline in the grub.cfg and that didn't match the GRUB_CMDLINE_LINUX in /etc/default/grub. Maybe what we can do is to not generate a new /boot/grub2/grub.cfg but instead copy the one in the ESP to cover these corner cases ?
I'll update the proposal based on the feedback.
Best regards, Javier
On 31.12.2020 13:36, Javier Martinez Canillas wrote:
I'll update the proposal based on the feedback.
And what about users, who use Fedora with other GNU/Linux distributions? These distributions can also switch to the same GRUB2 configuration. They will all start fighting for the same file. That's why a separate /boot/efi/EFI/$distro_name/grub.cfg configuration file was introduced.
On Thu, Jan 14, 2021, 2:24 AM Vitaly Zaitsev via devel < devel@lists.fedoraproject.org> wrote:
On 31.12.2020 13:36, Javier Martinez Canillas wrote:
I'll update the proposal based on the feedback.
And what about users, who use Fedora with other GNU/Linux distributions? These distributions can also switch to the same GRUB2 configuration. They will all start fighting for the same file. That's why a separate /boot/efi/EFI/$distro_name/grub.cfg configuration file was introduced.
That's Fedora specific. Upstream puts it in /boot/grub/ regardless of firmware.
-- Chris Murphy
On Thu, Jan 14, 2021 at 10:48 AM Chris Murphy lists@colorremedies.com wrote:
On Thu, Jan 14, 2021, 2:24 AM Vitaly Zaitsev via devel devel@lists.fedoraproject.org wrote:
On 31.12.2020 13:36, Javier Martinez Canillas wrote:
I'll update the proposal based on the feedback.
And what about users, who use Fedora with other GNU/Linux distributions? These distributions can also switch to the same GRUB2 configuration. They will all start fighting for the same file. That's why a separate /boot/efi/EFI/$distro_name/grub.cfg configuration file was introduced.
That's Fedora specific. Upstream puts it in /boot/grub/ regardless of firmware.
Also, this only solves the problem when installing different distros. The problem still exists if multiple Fedora are installed since all will share the same EFI vendor directory.
But yes, multi boot should probably also be taken into account in the design. That's why it will be important to agree with GRUB upstream on this and have buy-in from the other distros.
Best regards, Javier
On Wed, 2020-12-30 at 12:21 -0800, Adam Williamson wrote:
On Wed, 2020-12-30 at 14:53 -0500, Ben Cotton wrote:
== Benefit to Fedora ==
This change will not only simplify and make less confusing the GRUB configuration but also allow the following improvements:
- To have a consistent configuration across all the architectures
using GRUB.
- Allow the same installation to be booted with either EFI or
legacy BIOS.
- Use the same documentation and commands for all architectures
instead of having EFI as a special case.
- Make GRUB configuration tools more robust by not relying on
symbolic links to be created and not having to handle platform specific cases.
- Align with images generated by COSA and OSBuild on how the GRUB
configuration files are used.
- Align with other Linux distributions on how the GRUB
configuration files are used.
There's discussion in emails and IRC this morning that have not made it back to this Change, these are some of them but apologies if I miss some:
- right now putting /boot on btrfs (or xfs) works on EFI systems, since the GRUB config is on the EFI partition that Grub can write to. With this change, it's no longer an option unless /boot/grub2 is made a separate partition
- for comparison, SUSE stores GRUB environment variables in the Btrfs header, but apart from having to maintain a patch, this would not help the goal of unifying the config, nor help those who want a single / xfs partition
- a separate partition for storing GRUB config, no matter what architecture, is probably the ideal solution
But:
== Upgrade/compatibility impact ==
The changes will only be for new installations, existing systems will not be impacted and will continue using the grub.cfg and grubenv files that are located in the ESP.
To me several of the benefits seem to not really be true, so long as this is the plan for upgrades.
- We wouldn't have a "consistent configuration" across everybody,
really, because anyone who upgraded from pre-F34 would still have the old config; every bootloader debugging session ever would start by figuring out which case this was.
- We can't really use the same "documentation and commands" for the
same reason. We either have to document both possibilities forever, or accept that our docs will be incorrect for anyone who upgraded from pre-F34.
- We can't really make the tools "more robust" in the way cited
because they'll still have to handle both cases as long as both cases exist. If anything this makes them more fragile: the more divergent paths a tool has to support, the more likely it is something will break.
Igor was asking about migrating existing setups, and I think Javier plans to document that with the caveat that it might be flaky. But yeah, potentially having to support three configurations (BIOS only, EFI old style, EFI pointing to the new location), for me, means ideally we can agree on a mechanism that works everywhere, so that at least there's not another configuration change down the road.
To bring the separate email thread back to this, now that the Change proposal is out -- I was initially going to bring up a related Change Proposal to make /boot on btrfs work consistently -- it is also predicated on grubenv being writable, which is currently true on EFI but not on BIOS systems.
Depending on the final version of this proposal, I can retarget mine (that Javier kindly offered to help with, to help with any needed GRUB changes) for F35 to be about actually switching /boot to be on btrfs on Fedora solutions that are Btrfs-based -- on the same partition as / if LUKS is not used, and on a separate partition if it is, until we have native Btrfs encryption.
Best regards,
Am 30.12.20 um 22:14 schrieb Michel Alexandre Salim:
- a separate partition for storing GRUB config, no matter what
architecture, is probably the ideal solution
Not always. In VMs you would reduce the amount of partitions to ease up things. The main problem with Vms is, that you have LTS based linux distros on the host systems which can't mount the vm guest, if the guest uses newer filesystems. If you then use BTRFS on the boot partition or store grub configs in partionheaders, you can't access those from the guest making it impossible to change the bootconfig, if it fails for some stupid reasons like older pygrub ( xen ) boot loaders, which can't handle the kernel images compression/format. Happend last with Xenserver and Kernel 5.9.
For this, i.E. me, choose to store the boot config on / so we have just one partition with ext4, which can easily mounted on the host, as ext4 can be handled by the older LTS systems on the host.
As Fedora is a good server os, if the ui parts have been removed ;), it should be taken into any consideration, that it may not be a bare metal installation you make plans for. It still needs to run in good old stable setups ;)
Best regards and a happy New Year 2021 to all, Marius
On Wed, Dec 30, 2020 at 5:48 PM Marius Schwarz fedoradev@cloud-foo.de wrote:
Am 30.12.20 um 22:14 schrieb Michel Alexandre Salim:
- a separate partition for storing GRUB config, no matter what
architecture, is probably the ideal solution
Not always. In VMs you would reduce the amount of partitions to ease up things. The main problem with Vms is, that you have LTS based linux distros on the host systems which can't mount the vm guest, if the guest uses newer filesystems. If you then use BTRFS on the boot partition or store grub configs in partionheaders, you can't access those from the guest making it impossible to change the bootconfig, if it fails for some stupid reasons like older pygrub ( xen ) boot loaders, which can't handle the kernel images compression/format. Happend last with Xenserver and Kernel 5.9.
For this, i.E. me, choose to store the boot config on / so we have just one partition with ext4, which can easily mounted on the host, as ext4 can be handled by the older LTS systems on the host.
As Fedora is a good server os, if the ui parts have been removed ;), it should be taken into any consideration, that it may not be a bare metal installation you make plans for. It still needs to run in good old stable setups ;)
The issues Michel is referring to do not apply to Fedora Server using Btrfs, because outside of Workstation, currently no variant of Fedora has cross-layer integration with the bootloader.
That is, Fedora KDE does not currently have integration at the desktop level to trigger different GRUB states like GNOME will in Workstation. Cockpit in Fedora Server Edition also lacks this ability.
Most of this is due to missing documentation to actually *implement* the capability in other variants. But the side effect is that the concern we have is pretty much exclusive to Fedora Workstation.
Hi,
On 12/30/20 11:52 PM, Neal Gompa wrote:
On Wed, Dec 30, 2020 at 5:48 PM Marius Schwarz fedoradev@cloud-foo.de wrote:
Am 30.12.20 um 22:14 schrieb Michel Alexandre Salim:
- a separate partition for storing GRUB config, no matter what
architecture, is probably the ideal solution
Not always. In VMs you would reduce the amount of partitions to ease up things. The main problem with Vms is, that you have LTS based linux distros on the host systems which can't mount the vm guest, if the guest uses newer filesystems. If you then use BTRFS on the boot partition or store grub configs in partionheaders, you can't access those from the guest making it impossible to change the bootconfig, if it fails for some stupid reasons like older pygrub ( xen ) boot loaders, which can't handle the kernel images compression/format. Happend last with Xenserver and Kernel 5.9.
For this, i.E. me, choose to store the boot config on / so we have just one partition with ext4, which can easily mounted on the host, as ext4 can be handled by the older LTS systems on the host.
As Fedora is a good server os, if the ui parts have been removed ;), it should be taken into any consideration, that it may not be a bare metal installation you make plans for. It still needs to run in good old stable setups ;)
The issues Michel is referring to do not apply to Fedora Server using Btrfs, because outside of Workstation, currently no variant of Fedora has cross-layer integration with the bootloader.
I do not think that that is entirely true, well it depends on if anaconda also sets the menu_auto_hide=1 variable for other variants. The grub hidden boot menu stuff: https://fedoraproject.org/wiki/Changes/HiddenGrubMenu https://hansdegoede.livejournal.com/19081.html
Should mostly work, assuming they at least use a systemd user-session.
The boot menu being hidden or not depends on the boot_success grubenv variable, which gets set by a systemd-timer in the systemd-user session of non-system (aka normal) users if the user-session has lasted at least 2 minutes.
And since Fedora 33, the integration to force the boot-menu to be shown is using the standard systemd DBUS APIs for this, so that e.g. :
systemctl reboot -boot-loader-menu=60
To cause the menu to show next boot with a timeout of 60 seconds works now, except for a selinux bug which will hopefully be resolved soon: https://bugzilla.redhat.com/show_bug.cgi?id=1856399
So other desktop-environments / spins / variants could also integrate more closely with the hidden-boot-menu stuff, which is a pre-requisite for getting a fully flicker free boot.
I would be happy to answer any questions people may have about this, but I'm afraid I do not have the time to actively help with this (outside of answering questions).
As for writing docs, the FAQ on my blog answers all end-user questions that I know about. But yes we could use some more docs to help other variants integrate better with this. If someone wants to start a wiki page based on the Change + FAQ pages and then extend that as closer integration is added to other variants, go for it. Feel free to take all the text I wrote on this and put it under any FOSS license of your choice.
That is, Fedora KDE does not currently have integration at the desktop level to trigger different GRUB states like GNOME will in Workstation.
If the KDE folks want to work on this, I'm happy to provide assistance, see above.
Cockpit in Fedora Server Edition also lacks this ability.
I think that on servers just always showing the boot menu is fine and some even find this desirable.
Most of this is due to missing documentation to actually *implement* the capability in other variants. But the side effect is that the concern we have is pretty much exclusive to Fedora Workstation.
See above, I admit this is not that well documented. But I have always answered questions on this from various people, including other distros who have implemented this from scratch themselves. So the info is available in a way. And if someone can turn my emails into docs for this then that would be even better.
Regards,
Hans
Hi,
Thanks for the thorough reply, Hans! One question inline
On Tue, 2021-01-05 at 12:31 +0100, Hans de Goede wrote:
Hi,
On 12/30/20 11:52 PM, Neal Gompa wrote:
On Wed, Dec 30, 2020 at 5:48 PM Marius Schwarz < fedoradev@cloud-foo.de> wrote:
Am 30.12.20 um 22:14 schrieb Michel Alexandre Salim:
- a separate partition for storing GRUB config, no matter what
architecture, is probably the ideal solution
Not always. In VMs you would reduce the amount of partitions to ease up things. The main problem with Vms is, that you have LTS based linux distros on the host systems which can't mount the vm guest, if the guest uses newer filesystems. If you then use BTRFS on the boot partition or store grub configs in partionheaders, you can't access those from the guest making it impossible to change the bootconfig, if it fails for some stupid reasons like older pygrub ( xen ) boot loaders, which can't handle the kernel images compression/format. Happend last with Xenserver and Kernel 5.9.
For this, i.E. me, choose to store the boot config on / so we have just one partition with ext4, which can easily mounted on the host, as ext4 can be handled by the older LTS systems on the host.
As Fedora is a good server os, if the ui parts have been removed ;), it should be taken into any consideration, that it may not be a bare metal installation you make plans for. It still needs to run in good old stable setups ;)
The issues Michel is referring to do not apply to Fedora Server using Btrfs, because outside of Workstation, currently no variant of Fedora has cross-layer integration with the bootloader.
I do not think that that is entirely true, well it depends on if anaconda also sets the menu_auto_hide=1 variable for other variants. The grub hidden boot menu stuff: https://fedoraproject.org/wiki/Changes/HiddenGrubMenu https://hansdegoede.livejournal.com/19081.html
Should mostly work, assuming they at least use a systemd user- session.
The boot menu being hidden or not depends on the boot_success grubenv variable, which gets set by a systemd-timer in the systemd-user session of non-system (aka normal) users if the user-session has lasted at least 2 minutes.
`boot_success` needs to be cleared per boot though, right? So that if the user session timer doesn't set it back (so we assume the boot failed), the next boot shows the GRUB menu.
Where does this happen? I'm under the impression this happened in the GRUB execution environment, which is problematic if that environment is on a filesystem GRUB can't write to.
Apologies for seeing this late, it would be nice to get this information going into tomorrow's FESCo meeting since Javier's proposal has a -1 that is partly due to concern over breaking the hidden menu functionality -- https://pagure.io/fesco/issue/2543
And since Fedora 33, the integration to force the boot-menu to be shown is using the standard systemd DBUS APIs for this, so that e.g. :
systemctl reboot -boot-loader-menu=60
To cause the menu to show next boot with a timeout of 60 seconds works now, except for a selinux bug which will hopefully be resolved soon: https://bugzilla.redhat.com/show_bug.cgi?id=1856399
So other desktop-environments / spins / variants could also integrate more closely with the hidden-boot-menu stuff, which is a pre- requisite for getting a fully flicker free boot.
Flagging Neal here for KDE
I would be happy to answer any questions people may have about this, but I'm afraid I do not have the time to actively help with this (outside of answering questions).
No worries!
As for writing docs, the FAQ on my blog answers all end-user questions that I know about. But yes we could use some more docs to help other variants integrate better with this. If someone wants to start a wiki page based on the Change + FAQ pages and then extend that as closer integration is added to other variants, go for it. Feel free to take all the text I wrote on this and put it under any FOSS license of your choice.
Yeah, I can start a wiki and copy over the content of the FAQ, it might make sense to have something we can refer to that can be easily updated.
Best regards,
Hi,
On 1/12/21 5:55 PM, Michel Alexandre Salim wrote:
Hi,
Thanks for the thorough reply, Hans! One question inline
On Tue, 2021-01-05 at 12:31 +0100, Hans de Goede wrote:
Hi,
On 12/30/20 11:52 PM, Neal Gompa wrote:
On Wed, Dec 30, 2020 at 5:48 PM Marius Schwarz < fedoradev@cloud-foo.de> wrote:
Am 30.12.20 um 22:14 schrieb Michel Alexandre Salim:
- a separate partition for storing GRUB config, no matter what
architecture, is probably the ideal solution
Not always. In VMs you would reduce the amount of partitions to ease up things. The main problem with Vms is, that you have LTS based linux distros on the host systems which can't mount the vm guest, if the guest uses newer filesystems. If you then use BTRFS on the boot partition or store grub configs in partionheaders, you can't access those from the guest making it impossible to change the bootconfig, if it fails for some stupid reasons like older pygrub ( xen ) boot loaders, which can't handle the kernel images compression/format. Happend last with Xenserver and Kernel 5.9.
For this, i.E. me, choose to store the boot config on / so we have just one partition with ext4, which can easily mounted on the host, as ext4 can be handled by the older LTS systems on the host.
As Fedora is a good server os, if the ui parts have been removed ;), it should be taken into any consideration, that it may not be a bare metal installation you make plans for. It still needs to run in good old stable setups ;)
The issues Michel is referring to do not apply to Fedora Server using Btrfs, because outside of Workstation, currently no variant of Fedora has cross-layer integration with the bootloader.
I do not think that that is entirely true, well it depends on if anaconda also sets the menu_auto_hide=1 variable for other variants. The grub hidden boot menu stuff: https://fedoraproject.org/wiki/Changes/HiddenGrubMenu https://hansdegoede.livejournal.com/19081.html
Should mostly work, assuming they at least use a systemd user- session.
The boot menu being hidden or not depends on the boot_success grubenv variable, which gets set by a systemd-timer in the systemd-user session of non-system (aka normal) users if the user-session has lasted at least 2 minutes.
`boot_success` needs to be cleared per boot though, right? So that if the user session timer doesn't set it back (so we assume the boot failed), the next boot shows the GRUB menu.
Where does this happen? I'm under the impression this happened in the GRUB execution environment, which is problematic if that environment is on a filesystem GRUB can't write to.
Correct this happens in the grub execution environment.
Apologies for seeing this late, it would be nice to get this information going into tomorrow's FESCo meeting since Javier's proposal has a -1 that is partly due to concern over breaking the hidden menu functionality -- https://pagure.io/fesco/issue/2543
So I've read Neal's comment there and what he describes really is a special corner case, but yes it is possible for people to have created the special setup he describes and yes in that case moving grubenv to /boot will break the hidden-menu functionality.
I'm not sure if this warrants a -1 to the proposal though, I believe documenting this + some workaround for it should be sufficient.
But as mentioned in the fesco issue this has more to do with the fact that storing the grubenv in a filesystem-file is a bad idea in general.
As discussed in detail here: https://pagure.io/fedora-workstation/issue/206 we really should be moving away from that. As discussed there Suse already has grub-patches to instead store the grubenv in an part of the btrfs filesystem header which is reserved for bootloader use.
As @javierm says in the linked fedora-workstation issue, that is also the long term plan for Fedora, but we really want to discuss and develop a solution for this with/at grub-upstream so that we don't end up with conflicting solutions between distributions which stomp all over each-others data.
Regards,
Hans
Hi,
On Tue, 2021-01-12 at 19:56 +0100, Hans de Goede wrote:
Hi,
On 1/12/21 5:55 PM, Michel Alexandre Salim wrote:
Hi,
Thanks for the thorough reply, Hans! One question inline
On Tue, 2021-01-05 at 12:31 +0100, Hans de Goede wrote:
Hi,
On 12/30/20 11:52 PM, Neal Gompa wrote:
The issues Michel is referring to do not apply to Fedora Server using Btrfs, because outside of Workstation, currently no variant of Fedora has cross-layer integration with the bootloader.
I do not think that that is entirely true, well it depends on if anaconda also sets the menu_auto_hide=1 variable for other variants. The grub hidden boot menu stuff: https://fedoraproject.org/wiki/Changes/HiddenGrubMenu https://hansdegoede.livejournal.com/19081.html
Should mostly work, assuming they at least use a systemd user- session.
The boot menu being hidden or not depends on the boot_success grubenv variable, which gets set by a systemd-timer in the systemd-user session of non-system (aka normal) users if the user-session has lasted at least 2 minutes.
`boot_success` needs to be cleared per boot though, right? So that if the user session timer doesn't set it back (so we assume the boot failed), the next boot shows the GRUB menu.
Where does this happen? I'm under the impression this happened in the GRUB execution environment, which is problematic if that environment is on a filesystem GRUB can't write to.
Correct this happens in the grub execution environment.
Apologies for seeing this late, it would be nice to get this information going into tomorrow's FESCo meeting since Javier's proposal has a -1 that is partly due to concern over breaking the hidden menu functionality -- https://pagure.io/fesco/issue/2543
So I've read Neal's comment there and what he describes really is a special corner case, but yes it is possible for people to have created the special setup he describes and yes in that case moving grubenv to /boot will break the hidden-menu functionality.
I'm not sure if this warrants a -1 to the proposal though, I believe documenting this + some workaround for it should be sufficient.
But as mentioned in the fesco issue this has more to do with the fact that storing the grubenv in a filesystem-file is a bad idea in general.
It's not necessarily bad unless the filesystem is journaled, right, since GRUB's filesystem drivers basically don't support this? (so basically it's only FAT-like filesystems and ext2 that's perfectly safe).
As discussed in detail here: https://pagure.io/fedora-workstation/issue/206 we really should be moving away from that. As discussed there Suse already has grub-patches to instead store the grubenv in an part of the btrfs filesystem header which is reserved for bootloader use.
That's one of the option discussed, yes. One issue with doing it this way is we'll have to reimplement it for XFS and other future filesystems.
As @javierm says in the linked fedora-workstation issue, that is also the long term plan for Fedora, but we really want to discuss and develop a solution for this with/at grub-upstream so that we don't end up with conflicting solutions between distributions which stomp all over each-others data.
Agreed. But if we decide to use a separate partition with a properly supported filesystem, we might as well pick it now rather than make changes twice, right?
Chris suggested using a BIOS Boot partition, but another possible option is to use XBOOTLDR from https://systemd.io/BOOT_LOADER_SPECIFICATION/ - which the BLS actually prefers over the ESP if found.
(I made the mistake of assuming the change is up for discussion tomorrow, my bad. Looks like we have time to continue discussion here - or on the ticket)
Best regards,
On Wed, Jan 13, 2021 at 4:48 AM Michel Alexandre Salim michel@michel-slm.name wrote:
[snip]
As discussed in detail here: https://pagure.io/fedora-workstation/issue/206 we really should be moving away from that. As discussed there Suse already has grub-patches to instead store the grubenv in an part of the btrfs filesystem header which is reserved for bootloader use.
That's one of the option discussed, yes. One issue with doing it this way is we'll have to reimplement it for XFS and other future filesystems.
Yes, I don't think is worth it to cherry-pick Suse's patch only for btrfs and instead should aim for a general solution.
As @javierm says in the linked fedora-workstation issue, that is also the long term plan for Fedora, but we really want to discuss and develop a solution for this with/at grub-upstream so that we don't end up with conflicting solutions between distributions which stomp all over each-others data.
Agreed. But if we decide to use a separate partition with a properly supported filesystem, we might as well pick it now rather than make changes twice, right?
The thing is that implementing this proposal should be straightforward and something that could be done for F34 but solving the grubenv issue will require more time, since we first will need to agree with GRUB upstream on the approach.
Also, switching existing installations to the configuration scheme mentioned in this proposal should be feasible but changing the partition layout would be much more risky. That's why I believe that even if we solve the grubenv issue (i.e: in F35), this should only be for new installations and storing grubenv as a file has to be supported for backward compatibility.
Chris suggested using a BIOS Boot partition, but another possible option is to use XBOOTLDR from https://systemd.io/BOOT_LOADER_SPECIFICATION/ - which the BLS actually prefers over the ESP if found.
Chris' suggestion (unless I misunderstood) is not to use a partition with a filesystem but instead a partition where the grubenv could be stored as raw data. That way GRUB should be able to read and write the grubenv block without using any of its filesystem code (that's supposed to be read-only).
Best regards, Javier
On Tue, Jan 12, 2021 at 10:06 PM Javier Martinez Canillas javier@dowhile0.org wrote:
The thing is that implementing this proposal should be straightforward and something that could be done for F34 but solving the grubenv issue will require more time, since we first will need to agree with GRUB upstream on the approach.
Agree.
Also, switching existing installations to the configuration scheme mentioned in this proposal should be feasible but changing the partition layout would be much more risky. That's why I believe that even if we solve the grubenv issue (i.e: in F35), this should only be for new installations and storing grubenv as a file has to be supported for backward compatibility.
Agree. Unless I can track down a magical leopluradon to do a 100% safe conversion, I think it's OK to just depend on attrition to solve this.
I'm sometimes an outlier, but I'm a strong proponent of GUI tools constraining the options available to users, to only that which we recommend and want to support. i.e. it would be a good time for the installer to stop presenting all bootloader related things in the GUI. If the installer team folks want to support esoteric customizations in kickstart, super, I have no complaint. But the GUI, whether auto, custom, advanced, should just stop inviting users to do hapless things. If users precreate their own partitions outside the installer, with a proper GPT GUID, and the installer recognizes and automatically uses those partitions based on GUID, that'd be cool too. I'm very reluctant to ask that more and more things be supported without a plan for either growing resources to support such things. Maintainability is already a problem in the boot realm, both in Fedora and upstream (at least in GRUB).
Chris suggested using a BIOS Boot partition, but another possible option is to use XBOOTLDR from https://systemd.io/BOOT_LOADER_SPECIFICATION/ - which the BLS actually prefers over the ESP if found.
Chris' suggestion (unless I misunderstood) is not to use a partition with a filesystem but instead a partition where the grubenv could be stored as raw data. That way GRUB should be able to read and write the grubenv block without using any of its filesystem code (that's supposed to be read-only).
Correct. But also I consider it an implementation detail, decided by those who will implement and maintain it.
This hypothetical partition can be used by syslinux, sd-boot, uboot, and others, with whatever they want to put there, in whatever format or scheme they want to implement. It's just a generic bootloader partition, exclusively owned by the bootloader, for whatever purpose it wants to use it for. Likewise, its size is a detail that users don't need to care about. The upstreams can come up with a recommended size. As this hypothetical partition is modeled after the GPT "BIOS Boot" partition it could be 1MiB. If the core.img is pushing that, and if there's some idea to implement an A/B approach in this partition for atomic updating, then 2MiB is fine, even 20MiB is fine - to me those are all the same size :D Even coming from the SD Card in a Raspberry Pi type use case.
On Tue, Jan 12, 2021 at 8:48 PM Michel Alexandre Salim michel@michel-slm.name wrote:
Hi,
On Tue, 2021-01-12 at 19:56 +0100, Hans de Goede wrote:
So I've read Neal's comment there and what he describes really is a special corner case, but yes it is possible for people to have created the special setup he describes and yes in that case moving grubenv to /boot will break the hidden-menu functionality.
I'm not sure if this warrants a -1 to the proposal though, I believe documenting this + some workaround for it should be sufficient.
But as mentioned in the fesco issue this has more to do with the fact that storing the grubenv in a filesystem-file is a bad idea in general.
It's not necessarily bad unless the filesystem is journaled, right, since GRUB's filesystem drivers basically don't support this? (so basically it's only FAT-like filesystems and ext2 that's perfectly safe).
There are separate issues.
grubenv is "OK" on ext4 and XFS. The issue is that none of the file systems developers like anything other than kernel code doing writes. One idea for this is a new MBR and GPT partition type, functionally identical to the GPT "BIOS Boot" type, that's exclusively for the bootloader to use however it wants. On BIOS it'd include core.img and grubenv. On UEFI it'd just be used for grubenv. This issue needs to be taken upstream to sort out the details, so it should be set aside as it relates to this proposal.
The issue with journaled file systems is that GRUB's file system drivers have no ability to do journal replay. Therefore there is a small risk the file system is rendered unbootable in a crash, because the bootloader only sees the no-replay state of the file system used for /boot. e.g. the bootloader can see grub.cfg, bls snippets, or even the kernel as either missing or as 0 length files, until the journal has been replayed. Small risk, big penalty. My suggestion for mitigation is to use FAT for /boot in simple cases, and Btrfs in less simple cases. It's just an idea, it's not urgent, but if things are being reconsidered for simplification anyway, this makes sense to me.
Ergo the idea for a "bootloader partition that is exclusively owned by the bootloader" is separate from "bootloaders don't do journal replay and therefore can have the wrong view of things, and fail to boot in rare cases following a crash."
Chris suggested using a BIOS Boot partition, but another possible option is to use XBOOTLDR from https://systemd.io/BOOT_LOADER_SPECIFICATION/ - which the BLS actually prefers over the ESP if found.
I tentatively agree. We should ensure coordination with coreos/bootupd folks and boot loader spec folks that we're all on the same page.
-- Chris Murphy
Once upon a time, Chris Murphy lists@colorremedies.com said:
The issue with journaled file systems is that GRUB's file system drivers have no ability to do journal replay. Therefore there is a small risk the file system is rendered unbootable in a crash, because the bootloader only sees the no-replay state of the file system used for /boot. e.g. the bootloader can see grub.cfg, bls snippets, or even the kernel as either missing or as 0 length files, until the journal has been replayed. Small risk, big penalty. My suggestion for mitigation is to use FAT for /boot in simple cases, and Btrfs in less simple cases. It's just an idea, it's not urgent, but if things are being reconsidered for simplification anyway, this makes sense to me.
I've been bitten by that issue before. I would probably avoid FAT for a couple of reasons: no ownership/permissions, and could get stepped on in dual-boot setups by Windows. I'd go with one of the Linux non-journaling filesystems, like good ol' ext2. With few writes, it should always be in a "safe" state.
Ideally, it could be left mounted read-only and only remounted RW during updates (and then back to RO to make sure everything is flushed); although I guess doing that would generally cover the journaled FSes as well.
Hi,
Il giorno mer, 13/01/2021 alle 13.46 -0700, Chris Murphy ha scritto:
On Tue, Jan 12, 2021 at 8:48 PM Michel Alexandre Salim michel@michel-slm.name wrote:
... snip
There are separate issues.
grubenv is "OK" on ext4 and XFS. The issue is that none of the file systems developers like anything other than kernel code doing writes. One idea for this is a new MBR and GPT partition type, functionally identical to the GPT "BIOS Boot" type, that's exclusively for the bootloader to use however it wants. On BIOS it'd include core.img and grubenv. On UEFI it'd just be used for grubenv. This issue needs to be taken upstream to sort out the details, so it should be set aside as it relates to this proposal.
IMHO boot loaders should not write to filesystems, if this is needed to hide the GRUB menu when boot is successful, then let's display it always as it was one time. I don't think it we should follow Windows or MacOS style here, the boot menu is useful.
The issue with journaled file systems is that GRUB's file system drivers have no ability to do journal replay. Therefore there is a small risk the file system is rendered unbootable in a crash, because the bootloader only sees the no-replay state of the file system used for /boot. e.g. the bootloader can see grub.cfg, bls snippets, or even the kernel as either missing or as 0 length files, until the journal has been replayed. Small risk, big penalty. My suggestion for mitigation is to use FAT for /boot in simple cases, and Btrfs in less simple cases. It's just an idea, it's not urgent, but if things are being reconsidered for simplification anyway, this makes sense to me.
Ergo the idea for a "bootloader partition that is exclusively owned by the bootloader" is separate from "bootloaders don't do journal replay and therefore can have the wrong view of things, and fail to boot in rare cases following a crash."
Yet another partition needed to boot the OS. What when there are no available paritions in BIOS mode, because other OSes used all the available 4 partitions?
... snip
Ciao Guido
On Thu, Jan 14, 2021 at 12:35 AM Guido Aulisi guido.aulisi@gmail.com wrote:
IMHO boot loaders should not write to filesystems, if this is needed to hide the GRUB menu when boot is successful, then let's display it always as it was one time. I don't think it we should follow Windows or MacOS style here, the boot menu is useful.
It's an old design predicated on FAT and ext2. That design is outgrown. That's all.
Yet another partition needed to boot the OS. What when there are no available paritions in BIOS mode, because other OSes used all the available 4 partitions?
GRUB can read EBRs.
For bootloaders that don't understand EBR, they'll need to stick to the simple ways they're doing things already.
On 30.12.2020 20:53, Ben Cotton wrote:
This change makes the GRUB configuration files layout to be consistent across all the supported architectures. Currently EFI is a special case since the GRUB configuration file and environment variables block are stored in the EFI System Partition (ESP) instead of the boot partition (or `/boot` directory if no boot partition is used).
Why not switch from the ancient GRUB2 to the modern systemd-boot?
Vitaly Zaitsev via devel devel@lists.fedoraproject.org 于2020年12月31日周四 下午6:12写道：
On 30.12.2020 20:53, Ben Cotton wrote:
This change makes the GRUB configuration files layout to be consistent across all the supported architectures. Currently EFI is a special case since the GRUB configuration file and environment variables block are stored in the EFI System Partition (ESP) instead of the boot partition (or `/boot` directory if no boot partition is used).
Why not switch from the ancient GRUB2 to the modern systemd-boot?
iirc systemd-boot don't support legacy BIOS system.
-- Sincerely, Vitaly Zaitsev (vitaly@easycoding.org) _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
On 31.12.2020 11:28, Qiyu Yan wrote:
iirc systemd-boot don't support legacy BIOS system.
Yes of course. Systemd-boot is not a bootloader. It is an EFIStub kernel manager with a simple menu.
I've been using systemd-boot for over 1.5 years and it works great.
It can only be enabled for Fedora UEFI configurations. The deprecated legacy configurations should stay on GRUB2.
On Thu, Dec 31, 2020 at 06:28:22PM +0800, Qiyu Yan wrote:
Vitaly Zaitsev via devel devel@lists.fedoraproject.org 于2020年12月31日周四 下午6:12写道：
On 30.12.2020 20:53, Ben Cotton wrote:
This change makes the GRUB configuration files layout to be consistent across all the supported architectures. Currently EFI is a special case since the GRUB configuration file and environment variables block are stored in the EFI System Partition (ESP) instead of the boot partition (or `/boot` directory if no boot partition is used).
Why not switch from the ancient GRUB2 to the modern systemd-boot?
iirc systemd-boot don't support legacy BIOS system.
You can install systemd-boot and grub2-pc side-by-side, sharing the bls boot configuration, and have a system bootable in both uefi and bios mode.
We don't have systemd-boot on !x86 though. aarch64 didn't work last time I tried. armhfp seems to not be supported (at least the fedora rpm hasn't binaries). s390x and ppc are not efi platforms.
So if we want cross-platform consistency grub2 is the best option we have. The config file is ugly, but thanks to bls you can largely ignore that now. No need to touch it for kernel updates etc.
take care, Gerd
On Thu, Dec 31, 2020 at 10:10 AM Vitaly Zaitsev via devel devel@lists.fedoraproject.org wrote:
On 30.12.2020 20:53, Ben Cotton wrote:
This change makes the GRUB configuration files layout to be consistent across all the supported architectures. Currently EFI is a special case since the GRUB configuration file and environment variables block are stored in the EFI System Partition (ESP) instead of the boot partition (or `/boot` directory if no boot partition is used).
Why not switch from the ancient GRUB2 to the modern systemd-boot?
Because it doesn't support traditional BIOS boot, or the boot systems of POWER, Z-series and numerous other corner cases. Just look at the thread about BIOS support from a few months ago to see how controversial even considering that was. If we then move to it just for UEFI based platforms there needs to be a lot of logic to deal with different boot paths.
On 31.12.2020 11:50, Peter Robinson wrote:
Because it doesn't support traditional BIOS boot, or the boot systems of POWER, Z-series and numerous other corner cases. Just look at the thread about BIOS support from a few months ago to see how controversial even considering that was. If we then move to it just for UEFI based platforms there needs to be a lot of logic to deal with different boot paths.
An option can be added to the Fedora installer to choose between GRUB2 and systemd-boot for the UEFI configurations. On legacy, this step can be automatically skipped.
On Thu, Dec 31, 2020 at 11:29 AM Vitaly Zaitsev via devel devel@lists.fedoraproject.org wrote:
On 31.12.2020 11:50, Peter Robinson wrote:
Because it doesn't support traditional BIOS boot, or the boot systems of POWER, Z-series and numerous other corner cases. Just look at the thread about BIOS support from a few months ago to see how controversial even considering that was. If we then move to it just for UEFI based platforms there needs to be a lot of logic to deal with different boot paths.
An option can be added to the Fedora installer to choose between GRUB2 and systemd-boot for the UEFI configurations. On legacy, this step can be automatically skipped.
Of course it could, who do you propose to do that work and support all the various options and code required? It's easy to say it "works for me" in your one machine use case, but supporting it acros 1000s of config options with users that are less capable and don't know what a boot loader is it hard, it takes a lot of work so people don't end up with unbootable machines.
On 31.12.2020 12:37, Peter Robinson wrote:
Of course it could, who do you propose to do that work and support all the various options and code required?
It can be easily installed during Fedora installation by executing the following:
1. Make sure the ESP partition has more than 512 MB of free space (systemd-boot uses an ESP partition to store kernels; the separate /boot is no longer needed). 2. Add Fedora boot flags instead of the /etc/default/grub to the /etc/kernel/cmdline. 3. Install systemd-boot to the ESP partition: bootctl --path=/boot/efi install. 4. Execute kernel-install scripts (I think this stage will be the same as under GRUB2): kernel-install add $(uname -r) /lib/modules/$(uname -r)/vmlinuz. 5. Installation completed.
On Thu, Dec 31, 2020 at 12:54 PM Vitaly Zaitsev via devel devel@lists.fedoraproject.org wrote:
On 31.12.2020 12:37, Peter Robinson wrote:
Of course it could, who do you propose to do that work and support all the various options and code required?
It can be easily installed during Fedora installation by executing the following:
- Make sure the ESP partition has more than 512 MB of free space
(systemd-boot uses an ESP partition to store kernels; the separate /boot is no longer needed). 2. Add Fedora boot flags instead of the /etc/default/grub to the /etc/kernel/cmdline. 3. Install systemd-boot to the ESP partition: bootctl --path=/boot/efi install. 4. Execute kernel-install scripts (I think this stage will be the same as under GRUB2): kernel-install add $(uname -r) /lib/modules/$(uname -r)/vmlinuz. 5. Installation completed.
As Peter said this was already discussed in the "The future of legacy BIOS support in Fedora" thread [0]. I mentioned there that Anaconda already supports an option to use extlinux, so the same could be done for sd-boot.
It shouldn't be a lot of work as you mentioned but someone will have to propose the patches to Anaconda.
[0]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Best regards, Javier
On 30.12.2020 20:53, Ben Cotton wrote:
The proposal is to always store the `grub.cfg` and `grubenv` files in the `/boot/grub2/` directory, making `/boot/efi/EFI/fedora/grub.cfg` to only be a small configuration file that sets a different `$prefix` variable and loads the configuration file stored in `/boot/grub2/grub.cfg`.
And what about users, who use Fedora with other GNU/Linux distributions? These distributions can also switch to the same GRUB2 configuration. They will all start fighting for the same file. That's why a separate /boot/efi/EFI/$distro_name/grub.cfg configuration file was introduced.
On Wed, Dec 30, 2020 at 02:53:00PM -0500, Ben Cotton wrote:
[snip]
The proposal is to always store the `grub.cfg` and `grubenv` files in the `/boot/grub2/` directory, making `/boot/efi/EFI/fedora/grub.cfg` to only be a small configuration file that sets a different `$prefix` variable and loads the configuration file stored in `/boot/grub2/grub.cfg`.
The `$prefix` variable will be set to the device partition where `/boot/grub2/grub.cfg` is stored, using the partition filesystem's Universally Unique IDentifier (UUID). That way the correct GRUB configuration file will be loaded, making it as secure as the current approach where the configuration file in the ESP is used.
I don't see how adding more files and splitting information between them helps simplify things. You still need a system specific grub.cfg in the EFI directory so you haven't removed anything.
And now users who are trying to debug things have more places to look for problems.
If you want a consistent way to find the grub.cfg on bios and efi, why not create a symlink in /boot/grub2/ like we already do for grubenv? Actually, we already have symlinks in /etc/grub2.cfg and /etc/grub2-efi.cfg
Brian
On Tue, Jan 12, 2021 at 8:24 PM Brian C. Lane bcl@redhat.com wrote:
[snip]
The `$prefix` variable will be set to the device partition where `/boot/grub2/grub.cfg` is stored, using the partition filesystem's Universally Unique IDentifier (UUID). That way the correct GRUB configuration file will be loaded, making it as secure as the current approach where the configuration file in the ESP is used.
I don't see how adding more files and splitting information between them helps simplify things. You still need a system specific grub.cfg in the EFI directory so you haven't removed anything.
Yes, but this file should be static and never change. As mentioned, it will just search the correct partition using GRUB's search command, set a new $prefix variable and finally run `configfile $prefix/grub.cfg`.
And now users who are trying to debug things have more places to look for problems.
If you want a consistent way to find the grub.cfg on bios and efi, why not create a symlink in /boot/grub2/ like we already do for grubenv? Actually, we already have symlinks in /etc/grub2.cfg and /etc/grub2-efi.cfg
Symlinks have been proven to be error prone and a constant source of issues for users. In fact one of the goals of this change is to get rid of the grubenv symlink.
Very often we have bugs filed about users removing the symlink, then creating a new grubenv with `grub2-editenv create` in /boot/grub2/grubenv and then complaining about the GRUB env variables not being updated (because /boot/grub2/grubenv is now a real file instead of a symlink to the one in the ESP).
So the goal is for users to only care about /boot/grub2/grub{.cfg,env} and never having to touch the files in the ESP. That's what most distros do (and even we do on images generated with coreos-assembler and OSBuild), to avoid making EFI a special case and to have consistency across EFI, x86_64 legacy BIOS and ppc64le.
Best regards, Javier
devel@lists.stg.fedoraproject.org
-
Adam Williamson -
Ben Cotton -
Brian C. Lane -
Chris Adams -
Chris Murphy -
Gerd Hoffmann -
Guido Aulisi -
Hans de Goede -
Javier Martinez Canillas -
Marius Schwarz -
Matthew Miller -
Michel Alexandre Salim -
Neal Gompa -
Peter Robinson -
Qiyu Yan -
Tomasz Torcz -
Vitaly Zaitsev