The well known BIOSboot partition issue - Fedora does not install flawlessly on HDs larger 2 TB in Raid 1
I am doing some test installations to contribute some of our internal documentation for Fedora Server documentation. Thereby a problem with the BiOSboot partition occurred again.
To set up the system with RAID 1, manual partitioning must be performed during installation. I have created in the Anaconda GUI: - BIOSboot Partition - /boot as default partition and changed to raid 1 - /ROOT as LVM changed to raid 1 as well (and some additional LVs in that VG)
The installation aborted with the error message:
"The following error occurred while installing the boot loader. The system will not be bootable. Do you want to ignore this and continue with the installation? Boot loader install failed"
Anaconda creates BIOSboot only on the first hard disk, but then tries to install grub also on the 2nd hard disk and fails because of the missing BIOSboot partition there.
This was already discussed a long time ago and considered to create a BIOSboot partition automatically when creating a GPT even without user action. However, concerns were raised about this and the proposal was obviously not implemented.
I have not found a way in the current Anaconda manual partition GUI to install BIOSboot on the 2nd disk as well. My simple workaround was to boot a live system, set up GPT and BIOSboot, shutdown, and then boot the Fedora Server installation image.
The installation then went without a fuss. But is this the intended way? I suspect it isn’t. I probably missed something.
Any hint appreciated.
Peter
On Thu, Dec 31, 2020 at 3:12 PM Peter Boy pboy@uni-bremen.de wrote:
I am doing some test installations to contribute some of our internal documentation for Fedora Server documentation. Thereby a problem with the BiOSboot partition occurred again.
To set up the system with RAID 1, manual partitioning must be performed during installation. I have created in the Anaconda GUI:
- BIOSboot Partition
- /boot as default partition and changed to raid 1
- /ROOT as LVM changed to raid 1 as well (and some additional LVs in that VG)
The installation aborted with the error message:
"The following error occurred while installing the boot loader. The system will not be bootable. Do you want to ignore this and continue with the installation? Boot loader install failed"
Anaconda creates BIOSboot only on the first hard disk, but then tries to install grub also on the 2nd hard disk and fails because of the missing BIOSboot partition there.
This was already discussed a long time ago and considered to create a BIOSboot partition automatically when creating a GPT even without user action. However, concerns were raised about this and the proposal was obviously not implemented.
I have not found a way in the current Anaconda manual partition GUI to install BIOSboot on the 2nd disk as well. My simple workaround was to boot a live system, set up GPT and BIOSboot, shutdown, and then boot the Fedora Server installation image.
The installation then went without a fuss. But is this the intended way? I suspect it isn’t. I probably missed something.
Any hint appreciated.
What is the system? Does it support being switches to booting from UEFI and using GPT partitioning and then you won't need the BIOSboot partition.
Am 31.12.2020 um 19:24 schrieb Peter Robinson pbrobinson@gmail.com:
I have not found a way in the current Anaconda manual partition GUI to install BIOSboot on the 2nd disk as well. My simple workaround was to boot a live system, set up GPT and BIOSboot, shutdown, and then boot the Fedora Server installation image.
The installation then went without a fuss. But is this the intended way? I suspect it isn’t. I probably missed something.
Any hint appreciated.
What is the system? Does it support being switches to booting from UEFI and using GPT partitioning and then you won't need the BIOSboot partition.
It’s a rented dedicated Server (by Hetzner, a big European business) so I can’t adjust anything in the firmware. And i guess there are others who might have the same issue. So I would like to write a reliable and proper documentation / how to.
On Thu, Dec 31, 2020 at 11:24 AM Peter Robinson pbrobinson@gmail.com wrote:
What is the system? Does it support being switches to booting from UEFI and using GPT partitioning and then you won't need the BIOSboot partition.
It's even worse on UEFI. There it should create an ESP on each drive. But this is not allowed because there can be only one ESP for /boot/efi. The very idea of /boot/efi is deeply unfortunate. Nevertheless, the installer won't create an ESP on each drive. And even if you manually create an ESP on each drive, the installer only mounts one to /boot/efi and thus only that one gets the bootloader. And only one is added as a boot entry in NVRAM.
The area of multiple boot devices is unsolved. It might be solvable by bootupd. Decouple /boot/efi and /boot from RPM entirely. Have a single canonical reference for what their states should be, and use bootupd to enforce sync on one or more volumes that apply to the installation.
Am 31.12.2020 um 23:33 schrieb Chris Murphy lists@colorremedies.com:
On Thu, Dec 31, 2020 at 11:24 AM Peter Robinson pbrobinson@gmail.com wrote:
What is the system? Does it support being switches to booting from UEFI and using GPT partitioning and then you won't need the BIOSboot partition.
It's even worse on UEFI. There it should create an ESP on each drive. But this is not allowed because there can be only one ESP for /boot/efi.
I have to admit, I haven't used UEFI for Fedora yet. Does this really mean that you can't use a software raid with UEFI resp. that if the "main disk" fails, the system might continue to run but can no longer be booted? What a step backwards that would be.
On Fri, Jan 8, 2021 at 2:31 AM Peter Boy pboy@uni-bremen.de wrote:
Am 31.12.2020 um 23:33 schrieb Chris Murphy lists@colorremedies.com:
On Thu, Dec 31, 2020 at 11:24 AM Peter Robinson pbrobinson@gmail.com wrote:
What is the system? Does it support being switches to booting from UEFI and using GPT partitioning and then you won't need the BIOSboot partition.
It's even worse on UEFI. There it should create an ESP on each drive. But this is not allowed because there can be only one ESP for /boot/efi.
I have to admit, I haven't used UEFI for Fedora yet. Does this really mean that you can't use a software raid with UEFI resp. that if the "main disk" fails, the system might continue to run but can no longer be booted? What a step backwards that would be.
I haven't tried recently. I'm pretty sure it was only available in CentOS and thus RHEL. It works but it's invalid, and practically speaking it's asking for trouble because the firmware can write to the ESP. And other possible confusion. The long term approach is probably coreos/bootupd to sync multiple ESPs from a common source in /usr/ .
Since moving to BLS, the grub.cfg on the ESP is static. So changes are only going to /boot which can be raid1. The near term approach is to manually sync /boot/efi to the secondary ESPs. And then add an entry in NVRAM so it's an explicit fallback.
On Thu, Dec 31, 2020 at 8:12 AM Peter Boy pboy@uni-bremen.de wrote:
I am doing some test installations to contribute some of our internal documentation for Fedora Server documentation. Thereby a problem with the BiOSboot partition occurred again.
To set up the system with RAID 1, manual partitioning must be performed during installation. I have created in the Anaconda GUI:
- BIOSboot Partition
- /boot as default partition and changed to raid 1
- /ROOT as LVM changed to raid 1 as well (and some additional LVs in that VG)
The installation aborted with the error message:
"The following error occurred while installing the boot loader. The system will not be bootable. Do you want to ignore this and continue with the installation? Boot loader install failed"
Anaconda creates BIOSboot only on the first hard disk, but then tries to install grub also on the 2nd hard disk and fails because of the missing BIOSboot partition there.
This was already discussed a long time ago and considered to create a BIOSboot partition automatically when creating a GPT even without user action. However, concerns were raised about this and the proposal was obviously not implemented.
I have not found a way in the current Anaconda manual partition GUI to install BIOSboot on the 2nd disk as well. My simple workaround was to boot a live system, set up GPT and BIOSboot, shutdown, and then boot the Fedora Server installation image.
The installation then went without a fuss. But is this the intended way? I suspect it isn’t. I probably missed something.
I tried both 2-disk Automatic, and 2-disk Custom-Automatic (Custom/manual partitioning, but click the blue text to create partitions automatically). They both succeed but only one disk has BIOS Boot
As I fiddle with Custom UI, manually creating BIOS boot, the Device(s)>Modify dialog claims this partition is on both devices. But the installation summary only shows it on one. And the installation succeeds without failing.
storage.log shows the command: grub2-install --no-floppy /dev/vda
While /dev/vdb exists, it is not specified here. Now, there is a separate dialog for bootloaders accessed in the Installation Destination panel at the bottom, there's some blue text "full disk summary and bootloader" which is just another way to get into trouble, but is layered upon anything the user chooses in Custom that relates to bootloader stuff.
I'd characterize this primarily as a bug. But the most central problem is that esoteric bootloader related things should not exist in a GUI installer. The installer should just do the right thing and present no options at all whatsoever to the user. I'd go so far as to say this should apply to $BOOT as well. All such things could be controlled by kickstart, I leave that question up to the Anaconda and bootloader teams. But their single design and recommendation for each use case they want to support should not be user overridable in the GUI. Hands down. It's currently a giant sand trap.
-- Chris Murphy
Sorry, I had to repair my test systems which took some time.
Am 31.12.2020 um 23:26 schrieb Chris Murphy lists@colorremedies.com:
On Thu, Dec 31, 2020 at 8:12 AM Peter Boy pboy@uni-bremen.de wrote:
I am doing some test installations to contribute some of our internal documentation for Fedora Server documentation. Thereby a problem with the BiOSboot partition occurred again.
To set up the system with RAID 1, manual partitioning must be performed during installation. I have created in the Anaconda GUI:
- BIOSboot Partition
- /boot as default partition and changed to raid 1
- /ROOT as LVM changed to raid 1 as well (and some additional LVs in that VG)
The installation aborted with the error message:
"The following error occurred while installing the boot loader. The system will not be bootable. Do you want to ignore this and continue with the installation? Boot loader install failed"
Anaconda creates BIOSboot only on the first hard disk, but then tries to install grub also on the 2nd hard disk and fails because of the missing BIOSboot partition there.
This was already discussed a long time ago and considered to create a BIOSboot partition automatically when creating a GPT even without user action. However, concerns were raised about this and the proposal was obviously not implemented.
I have not found a way in the current Anaconda manual partition GUI to install BIOSboot on the 2nd disk as well. My simple workaround was to boot a live system, set up GPT and BIOSboot, shutdown, and then boot the Fedora Server installation image.
The installation then went without a fuss. But is this the intended way? I suspect it isn’t. I probably missed something.
...
As I fiddle with Custom UI, manually creating BIOS boot, the Device(s)>Modify dialog claims this partition is on both devices. But the installation summary only shows it on one.
Ir’s even worse. If you open the dialog box the first time, both disks are selected (if you select two disks at the initial Installationdestination form. As soon as you click on one of the disks it changes to non-selected and it is no longer possible to select both at the same time.
And the installation succeeds without failing.
storage.log shows the command: grub2-install --no-floppy /dev/vda
I tried several time. As soon as you install a /boot partition on RAID the installation issues a warning, the system would not be bootable if you proceed.
If you ignore the warning, the system boots up. In storage.log you find:
INFO:program:Running in chroot '/mnt/sysroot'... grub2-install --no-floppy /dev/sda INFO:program:Installing for i386-pc platform. INFO:program:Installation finished. No error reported. ... INFO:program:Running in chroot '/mnt/sysroot'... grub2-install --no-floppy /dev/sdb INFO:program:Installing for i386-pc platform. INFO:program:grub2-install: warning: ../grub-core/partmap/gpt.c:190:this GPT partition label contains no BIOS Boot Partition; embedding won't be possible. INFO:program:grub2-install: error: embedding is not possible, but this is required for RAID and LVM install. DEBUG:program:Return code: 1 ...
Anaconda analyses the problem correctly, but unfortunately does not provide a fix
I'd characterize this primarily as a bug. But the most central problem is that esoteric bootloader related things should not exist in a GUI installer. The installer should just do the right thing and present no options at all whatsoever to the user.
A Fedora Server installation will probably run disks in RAID mode quite often. In this respect, this bug is decidedly "uncomely“. HD larger than 2 TB are becoming more and more affordable and are likely to become more common even in small installations.
I tried the „Advanced Custom (Bildet-GUI“. Here, it is possible to install a BIOSboot on multiple disks and if you do so the installation of RAID 1 system succeeds without failing. But it is much more complex and time-consuming (and error-prone).
Question is, how are we going to write an installation guide as a Fedora project. Do we want a "simple" way (ie. Booting a live system before installation and preparing the disks on the command line) or a "Fedora Tools" way (cumbersome Blivet GUI)?
On Fri, Jan 8, 2021 at 2:23 AM Peter Boy pboy@uni-bremen.de wrote:
Question is, how are we going to write an installation guide as a Fedora project. Do we want a "simple" way (ie. Booting a live system before installation and preparing the disks on the command line) or a "Fedora Tools" way (cumbersome Blivet GUI)?
I can't figure it out at all https://bugzilla.redhat.com/show_bug.cgi?id=1914607
As for CLI, I tested this method and it works: 1. MBR skip this step. GPT create BIOS boot on each drive. 2. do an installation 3. post-install run grub2-install on both drives
-- Chris Murphy
server@lists.fedoraproject.org
-
Chris Murphy -
Peter Boy -
Peter Robinson