This is largely directed to the WG, as a request to clarify a part of the workstation product tech spec. It relates to a thread on the anaconda list regarding os-prober, and a thread on this list regarding release criteria, both of which are referenced below.
I am cross posting to the server@ list as well, while they don't have a dual-boot requirement in their spec it stands to reason the ability to dual-boot Fedora Server/CentOS/RHEL version n and n+1 could come in handy when doing migrations while still having a fall back position. Perhaps replies should drop the other cross posting since the requirements for the two products are different? But I leave up to the person replying to decide.
The WorkstationTechnical Spec says: "One aspect of storage configuration that will be needed is support for dual-boot setups (preserving preexisting Windows or OS X installations), since e.g. students may be required to run software on those platforms for their coursework."
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)? 1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
2. If the answer to 1a. is yes, and 1b. is no, does this dual-boot requirement apply to both BIOS and UEFI?
3. If resources cannot meet the dual-boot requirement by ship time, should the installer inform the user that their previous installation will be preserved but may not be bootable?
4. Why is the preservation of an existing Linux OS, including a previous Fedora, not explicit in the spec? Should it be?
The answers to the above will help determine the scope of QA testing in this area, and avoid lengthy debate during blocker meetings. Maybe it'll provide some kick in the pants for old bugs with unimplemented solutions. Or maybe it will make it clear that the UX in this area doesn't need improvement and therefore effort testing and developing can be better spent elsewhere. So in any case, clarification will be helpful.
References:
"grub2, 30_os-prober, os-prober: A Proposal" https://www.redhat.com/archives/anaconda-devel-list/2014-June/msg00020.html
Initial very rough Workstation release criteria draft https://lists.fedoraproject.org/pipermail/desktop/2014-June/009931.html
https://bugzilla.redhat.com/show_bug.cgi?id=825236 https://bugzilla.redhat.com/show_bug.cgi?id=964828 https://bugzilla.redhat.com/show_bug.cgi?id=1048999 https://bugzilla.redhat.com/show_bug.cgi?id=1010704
Thanks,
Chris Murphy
On Thu, 2014-06-26 at 18:51 -0600, Chris Murphy wrote:
This is largely directed to the WG, as a request to clarify a part of the workstation product tech spec. It relates to a thread on the anaconda list regarding os-prober, and a thread on this list regarding release criteria, both of which are referenced below.
I am cross posting to the server@ list as well, while they don't have a dual-boot requirement in their spec it stands to reason the ability to dual-boot Fedora Server/CentOS/RHEL version n and n+1 could come in handy when doing migrations while still having a fall back position. Perhaps replies should drop the other cross posting since the requirements for the two products are different? But I leave up to the person replying to decide.
The WorkstationTechnical Spec says: "One aspect of storage configuration that will be needed is support for dual-boot setups (preserving preexisting Windows or OS X installations), since e.g. students may be required to run software on those platforms for their coursework."
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)? 1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
If the answer to 1a. is yes, and 1b. is no, does this dual-boot requirement apply to both BIOS and UEFI?
If resources cannot meet the dual-boot requirement by ship time, should the installer inform the user that their previous installation will be preserved but may not be bootable?
Why is the preservation of an existing Linux OS, including a previous Fedora, not explicit in the spec? Should it be?
The answers to the above will help determine the scope of QA testing in this area, and avoid lengthy debate during blocker meetings. Maybe it'll provide some kick in the pants for old bugs with unimplemented solutions. Or maybe it will make it clear that the UX in this area doesn't need improvement and therefore effort testing and developing can be better spent elsewhere. So in any case, clarification will be helpful.
What I want to see in the area of boot loader configuration is that we adopt the boot loader spec or an elaboration of it. With a declarative system like that we can have boot configuration UI in the control center. With grub-configuration that is edited by scripts that is not feasible.
As to why dualbooting with Windows is explicitly called out and installing multiple Fedoras is not, I'd say that is because users might have a need to run applications under Windows - for applications that run under Linux, you are much more likely to be able to just run them on the Fedora Workstation instead.
My personal opinions:
On Thu, 2014-06-26 at 18:51 -0600, Chris Murphy wrote:
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)?
Yes.
1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
No. Users will not be able to recover from this scenario.
- If the answer to 1a. is yes, and 1b. is no, does this dual-boot
requirement apply to both BIOS and UEFI?
Yes.
- If resources cannot meet the dual-boot requirement by ship time,
should the installer inform the user that their previous installation will be preserved but may not be bootable?
I think that would be OK, if the warning is clear and you have to click a red button to bypass it.
Can you link to the discussion on why the above requirements are problematic?
- Why is the preservation of an existing Linux OS, including a
previous Fedora, not explicit in the spec? Should it be?
This would be nice to have. When I tried dual-booting a year or so ago, the status quo was that os-prober usually worked, but not really reliably.
If we adopt the bootloader spec, then it we should add a guarantee that we preserve the ability to boot other Linux systems that adhere to that spec (which is admittedly none). But there was significant debate over whether Fedora should adopt that spec.
On Fri, Jun 27, 2014 at 6:52 PM, Michael Catanzaro mcatanzaro@gnome.org wrote:
My personal opinions:
On Thu, 2014-06-26 at 18:51 -0600, Chris Murphy wrote:
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)?
Yes.
1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
No. Users will not be able to recover from this scenario.
- If the answer to 1a. is yes, and 1b. is no, does this dual-boot
requirement apply to both BIOS and UEFI?
Yes.
- If resources cannot meet the dual-boot requirement by ship time,
should the installer inform the user that their previous installation will be preserved but may not be bootable?
I think that would be OK, if the warning is clear and you have to click a red button to bypass it.
No that's completely broken imo. It used to work in the past suddenly breaking it and telling our users "sorry you cannot boot your other os" is just not acceptable.
So if you cannot met it for whatever reason you can either 1) revert whatever change broke it or 2) delay the release
On Jun 27, 2014, at 6:13 AM, Matthias Clasen mclasen@redhat.com wrote:
On Thu, 2014-06-26 at 18:51 -0600, Chris Murphy wrote:
The WorkstationTechnical Spec says: "One aspect of storage configuration that will be needed is support for dual-boot setups (preserving preexisting Windows or OS X installations), since e.g. students may be required to run software on those platforms for their coursework."
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)? 1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
If the answer to 1a. is yes, and 1b. is no, does this dual-boot requirement apply to both BIOS and UEFI?
If resources cannot meet the dual-boot requirement by ship time, should the installer inform the user that their previous installation will be preserved but may not be bootable?
Why is the preservation of an existing Linux OS, including a previous Fedora, not explicit in the spec? Should it be?
The answers to the above will help determine the scope of QA testing in this area, and avoid lengthy debate during blocker meetings. Maybe it'll provide some kick in the pants for old bugs with unimplemented solutions. Or maybe it will make it clear that the UX in this area doesn't need improvement and therefore effort testing and developing can be better spent elsewhere. So in any case, clarification will be helpful.
What I want to see in the area of boot loader configuration is that we adopt the boot loader spec or an elaboration of it. With a declarative system like that we can have boot configuration UI in the control center. With grub-configuration that is edited by scripts that is not feasible.
As to why dualbooting with Windows is explicitly called out and installing multiple Fedoras is not, I'd say that is because users might have a need to run applications under Windows - for applications that run under Linux, you are much more likely to be able to just run them on the Fedora Workstation instead.
"Breaking bootability of other Linux OS's would not be a release blocking bug. While breaking bootability of Windows or OS X should be considered a release blocking bug, unless the installer adequately informs the user in advance that their prior OS may not be bootable after installing Fedora."
Is that a reasonable policy?
Myriad issues exist with the current bootloaderspec not least of which is that it explicitly says Windows and OS X dual booting are out of scope for the spec; and its conf file format doesn't support referencing files on other volumes therefore chainloading isn't possible. This is probably a separate thread.
Chris Murphy
On Fri, 2014-06-27 at 16:31 -0600, Chris Murphy wrote:
"Breaking bootability of other Linux OS's would not be a release blocking bug. While breaking bootability of Windows or OS X should be considered a release blocking bug, unless the installer adequately informs the user in advance that their prior OS may not be bootable after installing Fedora."
Is that a reasonable policy?
We've never blocked on OS X like that, and I'd want to know if the folks involved in actually implementing this (anaconda team, basically) are on board with that.
On Jun 27, 2014, at 10:52 AM, Michael Catanzaro mcatanzaro@gnome.org wrote:
My personal opinions:
On Thu, 2014-06-26 at 18:51 -0600, Chris Murphy wrote:
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)?
Yes.
1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
No. Users will not be able to recover from this scenario.
- If the answer to 1a. is yes, and 1b. is no, does this dual-boot
requirement apply to both BIOS and UEFI?
Yes.
- If resources cannot meet the dual-boot requirement by ship time,
should the installer inform the user that their previous installation will be preserved but may not be bootable?
I think that would be OK, if the warning is clear and you have to click a red button to bypass it.
Can you link to the discussion on why the above requirements are problematic?
This is it. I'm uncertain how they can be requirements without a mandate from the Workstation WG or FESCo. The bugs aren't new [1], they vary from 1-3 years old, and none made it to either freeze exception or release blocker status expressly because there's no applicable release criteria. Hence request to clarify the scope of the tech spec "preserve existing" language.
- Why is the preservation of an existing Linux OS, including a
previous Fedora, not explicit in the spec? Should it be?
This would be nice to have. When I tried dual-booting a year or so ago, the status quo was that os-prober usually worked, but not really reliably.
I don't have a short explanation for this, but a lot of the problems here are bugs I summarize below. But in addition, right now grub-mkconfig creates generic boot entries from scratch for other Linux OS's that get found. Therefore two problems arise with those boot entries: the distro/user specified boot parameters aren't used, and any kernel updates in those other Linux OS's aren't reflected in the GRUB menu. GRUB has tools to do better, it's just that they aren't being used by upstream's own 30_os-prober script. If instead it used configfile to point to the distro specific grub.cfg/conf, neither problem described would happen. So someone build a ship yard, but no one's building ships. It's kinda weird.
If we adopt the bootloader spec, then it we should add a guarantee that we preserve the ability to boot other Linux systems that adhere to that spec (which is admittedly none). But there was significant debate over whether Fedora should adopt that spec.
Well we sorta have with respect to the implied configuration file format. Fedora Atomic/OSTree, and the blscfg.mod patch for GRUB (since GRUB upstream rejects bootloaderspec as written) both use the bls conf file format. The problem is the way we're implementing it departs from the spec in some fundamental ways, in part because the spec is so limiting (e.g. $BOOT is required to be FAT16/32) so it's instantly lost interoperability.
Chris Murphy
[1] Summary of Fedora bootloading bugs:
BIOS, preexisting Windows: reliably has a working boot entry created. For other combinations all bets are off:
BIOS or UEFI, preexisting Linux using LVM: boot entries not created because preexisting OS's LV's aren't activated by the installer, so os-prober/grub2-mkconfig don't find the preexisting OS. Two year old bug, finger pointing, no agreement on who should fix it. https://bugzilla.redhat.com/show_bug.cgi?id=825236
UEFI, preexisting Windows:, boot entry either not created for Windows or it doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=986731 ## Year old bug. https://bugzilla.redhat.com/show_bug.cgi?id=1010704 ## This bug was rejected as an F20 blocker on the basis that the release criteria for Windows only applies to BIOS.
UEFI, preexisting Linux no-LVM: boot entry does not work. This is a Fedora GRUB specific bug, over a year old, was rejected for freeze exception so it stands to reason it would have been rejected as a blocker as well. https://bugzilla.redhat.com/show_bug.cgi?id=964828
UEFI, preexisting OS X: boot entries don't work, have never worked for me in 3+ years. Two 18 month old bugs. https://bugzilla.redhat.com/show_bug.cgi?id=904668 ## Kernel panic might now be fixed by upstream, but upstream's solution is flaky and prone to breakage every time Apple makes a kernel change. Upstream insists on using their own bootloader rather than Apple's for unknown reasons. https://bugzilla.redhat.com/show_bug.cgi?id=893179 ## Described fix by adding GRUB xnu modules to grubx64.efi has not been implemented. Secondary problem is that there are two superfluous entries that don't work due to confusion resulting from Fedora's mactel-boot support (os-prober thinks the Fedora boot entries are an OS X installation).
On Sun, 2014-06-29 at 11:15 -0600, Chris Murphy wrote:
This is it. I'm uncertain how they can be requirements without a mandate from the Workstation WG or FESCo. The bugs aren't new [1], they vary from 1-3 years old, and none made it to either freeze exception or release blocker status expressly because there's no applicable release criteria. Hence request to clarify the scope of the tech spec "preserve existing" language.
They're indeed not requirements from the WG or from FESCO, just requirements from me, so if they hold no weight for you, that's totally fine. (I mentioned my response was my own opinion; I should also clarify that I'm not a WG member.) I'm not sure if you'll get more responses.
[1] Summary of Fedora bootloading bugs:
BIOS, preexisting Windows: reliably has a working boot entry created. For other combinations all bets are off:
Yeah, this is what I suspected.
Thanks for putting all the bugs together in one place. Each of the bugs you mention OUGHT to be a beta blocker, but this isn't a perfect world....
UEFI, preexisting Windows:, boot entry either not created for Windows or it doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=986731 ## Year old bug. https://bugzilla.redhat.com/show_bug.cgi?id=1010704 ## This bug was rejected as an F20 blocker on the basis that the release criteria for Windows only applies to BIOS.
This Windows one is most serious by far. I'd interpret this as a violation of the guideline you quoted: "One aspect of storage configuration that will be needed is support for dual-boot setups (preserving preexisting Windows or OS X installations)."
Adam, I see in that bug your comment: "Discussed at 2013-11-13 blocker review meeting - http://meetbot.fedoraproject.org/fedora-blocker-review/2013-11-13/f20-final-... . We agreed to amend the criterion to specify it covers the BIOS case only, and consequently this bug is rejected as a blocker."
I think this should be reconsidered. I can't figure out how to get to setup or a boot menu on my father's new laptop; if I were foolish enough to install Fedora on it, we'd never get back to Windows again.
On Jun 27, 2014, at 7:47 PM, Adam Williamson awilliam@redhat.com wrote:
On Fri, 2014-06-27 at 16:31 -0600, Chris Murphy wrote:
"Breaking bootability of other Linux OS's would not be a release blocking bug. While breaking bootability of Windows or OS X should be considered a release blocking bug, unless the installer adequately informs the user in advance that their prior OS may not be bootable after installing Fedora."
Is that a reasonable policy?
We've never blocked on OS X like that, and I'd want to know if the folks involved in actually implementing this (anaconda team, basically) are on board with that.
1. The absolute simplest way to solve this would be to fix this bug, which would make 2 of the 4 broken OS X entries functional. This is a Fedora specific bug, with a documented fix, and no action taken in a year. So yeah it does seem it needs to be a blocker to get traction but if there's another way to get it fixed… https://bugzilla.redhat.com/show_bug.cgi?id=893179#c9 https://bugzilla.redhat.com/show_bug.cgi?id=903937 ##closed duplicate but more concise explanation of problem and solution
2. Another approach I took was refusing the premise we even need OS X boot entries in GRUB, so why not just disable os-prober on Macs, and then the four broken OS X options won't even appear. Instead the user can use the built-in firmware boot manager to boot either Fedora or OS X. Maybe there's some legitimate concern some users won't know how to activate this boot manager using the option key at the startup chime? This RFE was likewise proposed a while ago, changed to something else, neither proposal going forward, and then being close without action. https://bugzilla.redhat.com/show_bug.cgi?id=982847
Thing is, even if we go with 2, arguably 1 is still a bug and should be fixed. Maybe the user wants OS X boot entries in GRUB, and if they want them they should work.
Chris Murphy
On Jun 29, 2014, at 1:55 PM, Michael Catanzaro mcatanzaro@gnome.org wrote:
Adam, I see in that bug your comment: "Discussed at 2013-11-13 blocker review meeting - http://meetbot.fedoraproject.org/fedora-blocker-review/2013-11-13/f20-final-... . We agreed to amend the criterion to specify it covers the BIOS case only, and consequently this bug is rejected as a blocker."
I think this should be reconsidered. I can't figure out how to get to setup or a boot menu on my father's new laptop; if I were foolish enough to install Fedora on it, we'd never get back to Windows again.
Since that bug and blocker review, I've become aware via Bootloaderspec and GRUB devel list that UEFI firmware is floating around in the wild that doesn't respond to keyboard input at boot time. This is for faster boots. So you can't get access to the boot manager pre-boot. You have to boot an OS, run a user space application to tickle the firmware into being activated on the next boot, reboot, and then you get the boot manager. So if you don't have a bootable OS with such a user space application you can't get to the firmware boot manager. Which means, installing Fedora on such a system makes Windows unbootable if the GRUB menu doesn't have a working Windows boot entry.
It sounds like your father's laptop may have such firmware.
So yes I think it's fairly self-evident that this need to work on UEFI computers now too, and should be a release blocking bug if it doesn't work. My vague understanding is the broken Windows boot entries is fixed in upstream, but I don't know if Fedora Rawhide's GRUB 2.02 has that fix and can't test it.
Chris Murphy
On Sun, Jun 29, 2014 at 03:11:08PM -0600, Chris Murphy wrote:
Another approach I took was refusing the premise we even need OS X boot entries in GRUB, so why not just disable os-prober on Macs, and then the four broken OS X options won't even appear. Instead the user can use the built-in firmware boot manager to boot either Fedora or OS X. Maybe there's some legitimate concern some users won't know how to activate this boot manager using the option key at the startup chime? This RFE was likewise proposed a while ago, changed to something else, neither proposal going forward, and then being close without action. https://bugzilla.redhat.com/show_bug.cgi?id=982847
The best plan is probably to figure out how to extend the boot loader spec appropriately, add support for it to grub and gut os-prober entirely.
On Jun 30, 2014, at 1:23 PM, Matthew Garrett mjg59@srcf.ucam.org wrote:
On Sun, Jun 29, 2014 at 03:11:08PM -0600, Chris Murphy wrote:
Another approach I took was refusing the premise we even need OS X boot entries in GRUB, so why not just disable os-prober on Macs, and then the four broken OS X options won't even appear. Instead the user can use the built-in firmware boot manager to boot either Fedora or OS X. Maybe there's some legitimate concern some users won't know how to activate this boot manager using the option key at the startup chime? This RFE was likewise proposed a while ago, changed to something else, neither proposal going forward, and then being close without action. https://bugzilla.redhat.com/show_bug.cgi?id=982847
The best plan is probably to figure out how to extend the boot loader spec appropriately, add support for it to grub and gut os-prober entirely.
Ok for long term. In the next two weeks before freeze is it possible to modify the grub2-efi package spec file GRUB_MODULES= so that the grux64.efi has xnu, xnu_uuid, xnu_uuid_test modules baked in? That would fix the main problem in bug 893179 so that the first two OS X entries would then have a chance of working.
Chris Murphy
On Mon, Jun 30, 2014 at 03:09:01PM -0600, Chris Murphy wrote:
Ok for long term. In the next two weeks before freeze is it possible to modify the grub2-efi package spec file GRUB_MODULES= so that the grux64.efi has xnu, xnu_uuid, xnu_uuid_test modules baked in? That would fix the main problem in bug 893179 so that the first two OS X entries would then have a chance of working.
Not unless somebody writes signature checking support for them, no.
On Jun 30, 2014, at 4:20 PM, Matthew Garrett mjg59@srcf.ucam.org wrote:
On Mon, Jun 30, 2014 at 03:09:01PM -0600, Chris Murphy wrote:
Ok for long term. In the next two weeks before freeze is it possible to modify the grub2-efi package spec file GRUB_MODULES= so that the grux64.efi has xnu, xnu_uuid, xnu_uuid_test modules baked in? That would fix the main problem in bug 893179 so that the first two OS X entries would then have a chance of working.
Not unless somebody writes signature checking support for them, no.
Ahh. So without that, it'd be possible to execute arbitrary code masquerading as xnu on a Secure Boot system?
Chris Murphy
On Mon, Jun 30, 2014 at 10:35:17PM -0600, Chris Murphy wrote:
On Jun 30, 2014, at 4:20 PM, Matthew Garrett mjg59@srcf.ucam.org wrote:
On Mon, Jun 30, 2014 at 03:09:01PM -0600, Chris Murphy wrote:
Ok for long term. In the next two weeks before freeze is it possible to modify the grub2-efi package spec file GRUB_MODULES= so that the grux64.efi has xnu, xnu_uuid, xnu_uuid_test modules baked in? That would fix the main problem in bug 893179 so that the first two OS X entries would then have a chance of working.
Not unless somebody writes signature checking support for them, no.
Ahh. So without that, it'd be possible to execute arbitrary code masquerading as xnu on a Secure Boot system?
Yeah. One option would be to just disable the code if secure boot is enabled - Macs don't implement it, so that would be fine for basically every real world case. But I'd still prefer to chain the Apple bootloader rather than fiddling with XNU.
Hi Chris, Just wanted to say that David Cantrell and the Anaconda team are looking at this to see what they can do to try to improve things for F21. So thanks for putting this list together, it is very useful.
Christian
----- Original Message -----
From: "Chris Murphy" lists@colorremedies.com To: "Discussions about development for the Fedora desktop" desktop@lists.fedoraproject.org, "Michael Catanzaro" mcatanzaro@gnome.org Cc: server@lists.fedoraproject.org Sent: Sunday, June 29, 2014 7:15:25 PM Subject: Re: Multibooting UX, how well it ought to work
On Jun 27, 2014, at 10:52 AM, Michael Catanzaro mcatanzaro@gnome.org wrote:
My personal opinions:
On Thu, 2014-06-26 at 18:51 -0600, Chris Murphy wrote:
1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)?
Yes.
1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?
No. Users will not be able to recover from this scenario.
- If the answer to 1a. is yes, and 1b. is no, does this dual-boot
requirement apply to both BIOS and UEFI?
Yes.
- If resources cannot meet the dual-boot requirement by ship time,
should the installer inform the user that their previous installation will be preserved but may not be bootable?
I think that would be OK, if the warning is clear and you have to click a red button to bypass it.
Can you link to the discussion on why the above requirements are problematic?
This is it. I'm uncertain how they can be requirements without a mandate from the Workstation WG or FESCo. The bugs aren't new [1], they vary from 1-3 years old, and none made it to either freeze exception or release blocker status expressly because there's no applicable release criteria. Hence request to clarify the scope of the tech spec "preserve existing" language.
- Why is the preservation of an existing Linux OS, including a
previous Fedora, not explicit in the spec? Should it be?
This would be nice to have. When I tried dual-booting a year or so ago, the status quo was that os-prober usually worked, but not really reliably.
I don't have a short explanation for this, but a lot of the problems here are bugs I summarize below. But in addition, right now grub-mkconfig creates generic boot entries from scratch for other Linux OS's that get found. Therefore two problems arise with those boot entries: the distro/user specified boot parameters aren't used, and any kernel updates in those other Linux OS's aren't reflected in the GRUB menu. GRUB has tools to do better, it's just that they aren't being used by upstream's own 30_os-prober script. If instead it used configfile to point to the distro specific grub.cfg/conf, neither problem described would happen. So someone build a ship yard, but no one's building ships. It's kinda weird.
If we adopt the bootloader spec, then it we should add a guarantee that we preserve the ability to boot other Linux systems that adhere to that spec (which is admittedly none). But there was significant debate over whether Fedora should adopt that spec.
Well we sorta have with respect to the implied configuration file format. Fedora Atomic/OSTree, and the blscfg.mod patch for GRUB (since GRUB upstream rejects bootloaderspec as written) both use the bls conf file format. The problem is the way we're implementing it departs from the spec in some fundamental ways, in part because the spec is so limiting (e.g. $BOOT is required to be FAT16/32) so it's instantly lost interoperability.
Chris Murphy
[1] Summary of Fedora bootloading bugs:
BIOS, preexisting Windows: reliably has a working boot entry created. For other combinations all bets are off:
BIOS or UEFI, preexisting Linux using LVM: boot entries not created because preexisting OS's LV's aren't activated by the installer, so os-prober/grub2-mkconfig don't find the preexisting OS. Two year old bug, finger pointing, no agreement on who should fix it. https://bugzilla.redhat.com/show_bug.cgi?id=825236
UEFI, preexisting Windows:, boot entry either not created for Windows or it doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=986731 ## Year old bug. https://bugzilla.redhat.com/show_bug.cgi?id=1010704 ## This bug was rejected as an F20 blocker on the basis that the release criteria for Windows only applies to BIOS.
UEFI, preexisting Linux no-LVM: boot entry does not work. This is a Fedora GRUB specific bug, over a year old, was rejected for freeze exception so it stands to reason it would have been rejected as a blocker as well. https://bugzilla.redhat.com/show_bug.cgi?id=964828
UEFI, preexisting OS X: boot entries don't work, have never worked for me in 3+ years. Two 18 month old bugs. https://bugzilla.redhat.com/show_bug.cgi?id=904668 ## Kernel panic might now be fixed by upstream, but upstream's solution is flaky and prone to breakage every time Apple makes a kernel change. Upstream insists on using their own bootloader rather than Apple's for unknown reasons. https://bugzilla.redhat.com/show_bug.cgi?id=893179 ## Described fix by adding GRUB xnu modules to grubx64.efi has not been implemented. Secondary problem is that there are two superfluous entries that don't work due to confusion resulting from Fedora's mactel-boot support (os-prober thinks the Fedora boot entries are an OS X installation). -- desktop mailing list desktop@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/desktop
On Jul 1, 2014, at 12:35 AM, Matthew Garrett mjg59@srcf.ucam.org wrote:
On Mon, Jun 30, 2014 at 10:35:17PM -0600, Chris Murphy wrote:
On Jun 30, 2014, at 4:20 PM, Matthew Garrett mjg59@srcf.ucam.org wrote:
On Mon, Jun 30, 2014 at 03:09:01PM -0600, Chris Murphy wrote:
Ok for long term. In the next two weeks before freeze is it possible to modify the grub2-efi package spec file GRUB_MODULES= so that the grux64.efi has xnu, xnu_uuid, xnu_uuid_test modules baked in? That would fix the main problem in bug 893179 so that the first two OS X entries would then have a chance of working.
Not unless somebody writes signature checking support for them, no.
Ahh. So without that, it'd be possible to execute arbitrary code masquerading as xnu on a Secure Boot system?
Yeah. One option would be to just disable the code if secure boot is enabled - Macs don't implement it, so that would be fine for basically every real world case. But I'd still prefer to chain the Apple bootloader rather than fiddling with XNU.
I'd say until there's a replacement for os-prober's functionality that can also recognize encrypted OS X installs, and grub2-mkconfig creates OS X boot entries using chainloader rather than xnu modules, the simplest solution is anaconda adding DISABLE_OS_PROBER="True" to /etc/default/grub on Macs.
Upstream's solution mystifies me, it's been broken for ~2 years at least, and while it ought to be working now in GRUB 2.02, it's at the whim of Apple's future kernel changes. So not only is it a maintenance hassle, but it also can't boot encrypted OS X installs. I just tested chainloading the Apple bootloader from GRUB on an encrypted OS X installation and it works.
I'm going to guess a significant minority, if not majority, of OS X users who also install Fedora, are using encrypted OS X installations. Because os-prober doesn't search Apple Boot partition types, and can't read encrypted Core Storage partitions, OS X boot entries aren't created at all for encrypted OS X installs. So we already have a relatively common scenario where there aren't OS X boot entries. So I still think suppressing os-prober on Macs is a better outcome than unencrypted OS X installs having a GRUB menu with four non-working boot menu entries, it also makes the GRUB menu consistent whether the OS X install is encrypted or not.
Chris Murphy
On Mon, Jun 30, 2014, at 12:23 PM, Matthew Garrett wrote:
On Sun, Jun 29, 2014 at 03:11:08PM -0600, Chris Murphy wrote:
Another approach I took was refusing the premise we even need OS X boot entries in GRUB, so why not just disable os-prober on Macs, and then the four broken OS X options won't even appear. Instead the user can use the built-in firmware boot manager to boot either Fedora or OS X. Maybe there's some legitimate concern some users won't know how to activate this boot manager using the option key at the startup chime? This RFE was likewise proposed a while ago, changed to something else, neither proposal going forward, and then being close without action. https://bugzilla.redhat.com/show_bug.cgi?id=982847
The best plan is probably to figure out how to extend the boot loader spec appropriately, add support for it to grub and gut os-prober entirely.
While not quite related to OS X, I followed up to this here: https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html
On Sun, 2014-06-29 at 14:55 -0500, Michael Catanzaro wrote:
UEFI, preexisting Windows:, boot entry either not created for Windows or it doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=986731 ## Year old bug. https://bugzilla.redhat.com/show_bug.cgi?id=1010704 ## This bug was rejected as an F20 blocker on the basis that the release criteria for Windows only applies to BIOS.
This Windows one is most serious by far. I'd interpret this as a violation of the guideline you quoted: "One aspect of storage configuration that will be needed is support for dual-boot setups (preserving preexisting Windows or OS X installations)."
Adam, I see in that bug your comment: "Discussed at 2013-11-13 blocker review meeting - http://meetbot.fedoraproject.org/fedora-blocker-review/2013-11-13/f20-final-... . We agreed to amend the criterion to specify it covers the BIOS case only, and consequently this bug is rejected as a blocker."
I think this should be reconsidered. I can't figure out how to get to setup or a boot menu on my father's new laptop; if I were foolish enough to install Fedora on it, we'd never get back to Windows again.
Given our experience of real-world UEFI deployments since that time, I do agree; at that time we were still hopeful that the EFI boot manager could be relied on as a viable multiboot mechanism, but it does seem to be becoming apparent that that is not the case, and we should be making sure grub can handle booting pre-existing UEFI Windows installs. We've got rather a lot of Fedora.next stuff on ATM, but I guess what I'd suggest at this point is to re-nominate that bug as a Beta or Final blocker for F21, with a summary of this discussion.
I poked at this a bit more today. What I've got so far:
* http://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/ - a modified version of the BLS, along with a description and justification of the changes. * An entirely untested patch to grub2 that implements all of the spec other than the devicetree bit. * No changes to os-prober * No changes to anaconda * No changes to grubby
But those shouldn't really be too difficult to fix (a-ha ha ha). Diff follows.
commit cce95c9dbc123e9b65e683913c5a29befa826a89 Author: Matthew Garrett matthew.garrett@nebula.com Date: Mon Jul 7 18:50:44 2014 -0400
Extend BLS support
diff --git a/grub-core/commands/blscfg.c b/grub-core/commands/blscfg.c index 4274aca..6b762fb 100644 --- a/grub-core/commands/blscfg.c +++ b/grub-core/commands/blscfg.c @@ -36,14 +36,13 @@ GRUB_MOD_LICENSE ("GPLv3+"); #ifdef GRUB_MACHINE_EFI #define GRUB_LINUX_CMD "linuxefi" #define GRUB_INITRD_CMD "initrdefi" -#define GRUB_BLS_CONFIG_PATH "/EFI/fedora/loader/entries/" #define GRUB_BOOT_DEVICE "($boot)" #else #define GRUB_LINUX_CMD "linux" #define GRUB_INITRD_CMD "initrd" -#define GRUB_BLS_CONFIG_PATH "/loader/entries/" #define GRUB_BOOT_DEVICE "($root)" #endif +#define GRUB_BLS_CONFIG_PATH "/org/freedesktop/bls/entries/"
static int parse_entry ( const char *filename, @@ -54,7 +53,7 @@ static int parse_entry ( char *p; grub_file_t f = NULL; grub_off_t sz; - char *title = NULL, *options = NULL, *clinux = NULL, *initrd = NULL, *src = NULL; + char *title = NULL, *options = NULL, *clinux = NULL, *initrd = NULL, *src = NULL, *root = NULL, *chainload = NULL; const char *args[2] = { NULL, NULL };
if (filename[0] == '.') @@ -113,25 +112,103 @@ static int parse_entry ( if (!initrd) goto finish; } + else if (grub_strncmp (buf, "filesystem ", 10) == 0) + { + char *path = buf + 10; + + if (grub_strncmp (p, "0x", 2)) /* BIOS-style disk name */ + { + char c, *d = path; + int drive, partition, partoff = 0; + + while ((c = *d++) != '\0') + { + if (c == ',') + partoff = d - path; + } + if (partoff) + { + drive = (int) grub_strtoul (p, 0, 16); + partition = (int) grub_strtoul (path + partoff, 0, 16); + } + else + { + drive = (int) grub_strtoul (path, 0, 16); + partition = 0; + } + + drive -= 0x80; + if (partition) + { + root = grub_xasprintf ("set root=(hd(%d,%d))\n", + drive, + partition); + } + else + { + root = grub_xasprintf ("set root=hd(%d)\n", + drive); + } + if (!root) + goto finish; + } + else /* Assume UUID */ + { + root = grub_xasprintf ("insmod search_fs_uuid\n" + "search --no-floppy -u %s\n", + path); + if (!root) + goto finish; + } + } + else if (grub_strncmp (buf, "chainload", 14) == 0) + { + chainload = grub_xasprintf ("chainloader %s+1", GRUB_BOOT_DEVICE); + if (!chainload) + goto finish; + } + else if (grub_strncmp (buf, "efi ", 4) == 0) + { + chainload = grub_xasprintf ("chainloader %s%s", GRUB_BOOT_DEVICE, + buf + 4); + if (!chainload) + goto finish; + }
grub_free(buf); }
- if (!linux) + if (!linux && !chainload) { - grub_printf ("Skipping file %s with no 'linux' key.", p); + grub_printf ("Skipping file %s with no 'linux' or 'chainload' key.", + p); goto finish; }
args[0] = title ? title : filename;
- src = grub_xasprintf ("load_video\n" - "set gfx_payload=keep\n" - "insmod gzio\n" - GRUB_LINUX_CMD " %s%s%s%s\n" - "%s%s%s%s", - GRUB_BOOT_DEVICE, clinux, options ? " " : "", options ? options : "", - initrd ? GRUB_INITRD_CMD " " : "", initrd ? GRUB_BOOT_DEVICE : "", initrd ? initrd : "", initrd ? "\n" : ""); + if (chainload) { + src = grub_xasprintf ("insmod part_msdos\n" + "insmod part_gpt\n" + "insmod chain\n" + "%s" + "%s", + root ? root : "", + chainload); + } else { + src = grub_xasprintf ("load_video\n" + "set gfx_payload=keep\n" + "insmod gzio\n" + "insmod part_msdos\n" + "insmod part_gpt\n" + "insmod ext2\n" + "insmod xfs\n" + "%s" + GRUB_LINUX_CMD " %s%s%s%s\n" + "%s%s%s%s", + GRUB_BOOT_DEVICE, root ? root : "", clinux, options ? " " : "", options ? options : "", + initrd ? GRUB_INITRD_CMD " " : "", initrd ? GRUB_BOOT_DEVICE : "", initrd ? initrd : "", initrd ? "\n" : ""); + }
grub_normal_add_menu_entry (1, args, NULL, NULL, "bls", NULL, NULL, src, 0);
@@ -142,6 +219,8 @@ finish: grub_free (clinux); grub_free (initrd); grub_free (src); + grub_free (chainload); + grub_free (root);
if (f) grub_file_close (f);
On Sun, 2014-06-29 at 11:15 -0600, Chris Murphy wrote:
[1] Summary of Fedora bootloading bugs:
BIOS, preexisting Windows: reliably has a working boot entry created. For other combinations all bets are off:
BIOS or UEFI, preexisting Linux using LVM: boot entries not created because preexisting OS's LV's aren't activated by the installer, so os-prober/grub2-mkconfig don't find the preexisting OS. Two year old bug, finger pointing, no agreement on who should fix it. https://bugzilla.redhat.com/show_bug.cgi?id=825236
UEFI, preexisting Windows:, boot entry either not created for Windows or it doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=986731 ## Year old bug. https://bugzilla.redhat.com/show_bug.cgi?id=1010704 ## This bug was rejected as an F20 blocker on the basis that the release criteria for Windows only applies to BIOS.
UEFI, preexisting Linux no-LVM: boot entry does not work. This is a Fedora GRUB specific bug, over a year old, was rejected for freeze exception so it stands to reason it would have been rejected as a blocker as well. https://bugzilla.redhat.com/show_bug.cgi?id=964828
UEFI, preexisting OS X: boot entries don't work, have never worked for me in 3+ years. Two 18 month old bugs. https://bugzilla.redhat.com/show_bug.cgi?id=904668 ## Kernel panic might now be fixed by upstream, but upstream's solution is flaky and prone to breakage every time Apple makes a kernel change. Upstream insists on using their own bootloader rather than Apple's for unknown reasons. https://bugzilla.redhat.com/show_bug.cgi?id=893179 ## Described fix by adding GRUB xnu modules to grubx64.efi has not been implemented. Secondary problem is that there are two superfluous entries that don't work due to confusion resulting from Fedora's mactel-boot support (os-prober thinks the Fedora boot entries are an OS X installation).
Hi,
Am I alone in thinking we should block final on all of these? We're delayed so much already that it seems like it might be best to take care of these once and for all. Especially if we are going to pretend that Fedora is a good alternative to Mac; we should not pretend that if we cannot boot on Macs.
A nonfunctional installer is a really big problem. At the worst, the installer should be able to detect these situations in advance and warn the user.
On Aug 16, 2014, at 6:36 AM, Michael Catanzaro mcatanzaro@gnome.org wrote:
On Sun, 2014-06-29 at 11:15 -0600, Chris Murphy wrote:
[1] Summary of Fedora bootloading bugs:
BIOS, preexisting Windows: reliably has a working boot entry created. For other combinations all bets are off:
BIOS or UEFI, preexisting Linux using LVM: boot entries not created because preexisting OS's LV's aren't activated by the installer, so os-prober/grub2-mkconfig don't find the preexisting OS. Two year old bug, finger pointing, no agreement on who should fix it. https://bugzilla.redhat.com/show_bug.cgi?id=825236
UEFI, preexisting Windows:, boot entry either not created for Windows or it doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=986731 ## Year old bug. https://bugzilla.redhat.com/show_bug.cgi?id=1010704 ## This bug was rejected as an F20 blocker on the basis that the release criteria for Windows only applies to BIOS.
UEFI, preexisting Linux no-LVM: boot entry does not work. This is a Fedora GRUB specific bug, over a year old, was rejected for freeze exception so it stands to reason it would have been rejected as a blocker as well. https://bugzilla.redhat.com/show_bug.cgi?id=964828
UEFI, preexisting OS X: boot entries don't work, have never worked for me in 3+ years. Two 18 month old bugs. https://bugzilla.redhat.com/show_bug.cgi?id=904668 ## Kernel panic might now be fixed by upstream, but upstream's solution is flaky and prone to breakage every time Apple makes a kernel change. Upstream insists on using their own bootloader rather than Apple's for unknown reasons. https://bugzilla.redhat.com/show_bug.cgi?id=893179 ## Described fix by adding GRUB xnu modules to grubx64.efi has not been implemented. Secondary problem is that there are two superfluous entries that don't work due to confusion resulting from Fedora's mactel-boot support (os-prober thinks the Fedora boot entries are an OS X installation).
Hi,
Am I alone in thinking we should block final on all of these? We're delayed so much already that it seems like it might be best to take care of these once and for all. Especially if we are going to pretend that Fedora is a good alternative to Mac; we should not pretend that if we cannot boot on Macs.
Fedora post-install behavior on Macs: - GRUB menu appears by default - Fedora boot entries work - OS X boot entries don't work ## The two bugs above relate to this.
I don't know the % of Mac users who know about the option-key @ boot chime trick to get the firmware's boot manager to appear, but that's the work around to boot OS X. If the user doesn't know this, it's a problem, but the fix is literally "push this button". Whereas the other bugs above require esoteric knowledge to fix. And actually, there are bigger Mac problems than this: wireless, bluetooth, trackpad, and overheating/MCE problems are much worse, maybe deal killers. (At least I can't use Fedora, or any linux, full time right now. It's just way too frustrating, and may even be cooking themselves. [1])
A nonfunctional installer is a really big problem. At the worst, the installer should be able to detect these situations in advance and warn the user.
I think the other bugs in the list need priority, resources being limited and all. We should commonbugs the Mac bugs, which have the same push this button solution.
Chris Murphy
On Sat, 2014-08-16 at 11:50 -0600, Chris Murphy wrote:
I think the other bugs in the list need priority, resources being limited and all. We should commonbugs the Mac bugs, which have the same push this button solution.
OK, it sounds like Macs indeed have bigger issues. I will propose "BIOS or UEFI, preexisting Linux using LVM" and "UEFI, preexisting Linux no-LVM" as blockers, then, but not "UEFI, preexisting OS X." Thanks Chris.
desktop@lists.stg.fedoraproject.org
-
Adam Williamson -
Adam Williamson -
Chris Murphy -
Christian Schaller -
Colin Walters -
drago01 -
Matthew Garrett -
Matthias Clasen -
Michael Catanzaro