I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
What would be the best way to approach this?:
1) Do a system upgrade and then convert to BTRFS by backing everything up and restoring it (I'd need guidance on how to do this).
2) Do a complete system install and then restore from backups.
I'm guessing that (2) is the simplest answer, but I'd appreciate any comments, especially from people who have actually done either of these.
poc
Patrick O'Callaghan writes:
I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
What would be the best way to approach this?:
- Do a system upgrade and then convert to BTRFS by backing everything
up and restoring it (I'd need guidance on how to do this).
- Do a complete system install and then restore from backups.
I'm guessing that (2) is the simplest answer, but I'd appreciate any comments, especially from people who have actually done either of these.
If you're set on this kind of a disruption, a clean install, followed by restoring your home directory from backups, will be more reliable. Attempting to do /full/ reinstall of the /entire/ system from backups is a much more complicated affair. Your backup/restore method must correctly not just reinstall files and set their ownership and permissions, but also all other extended attributes, specifically selinux contexts. I'm not aware of anything in Fedora that uses other extended attributes for something, but there may be.
This is less of an issue when restoring just your home directory from backups.
On Mon, 2020-11-02 at 08:19 -0500, Sam Varshavchik wrote:
Patrick O'Callaghan writes:
I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
What would be the best way to approach this?:
- Do a system upgrade and then convert to BTRFS by backing everything
up and restoring it (I'd need guidance on how to do this).
- Do a complete system install and then restore from backups.
I'm guessing that (2) is the simplest answer, but I'd appreciate any comments, especially from people who have actually done either of these.
If you're set on this kind of a disruption, a clean install, followed by restoring your home directory from backups, will be more reliable. Attempting to do /full/ reinstall of the /entire/ system from backups is a much more complicated affair. Your backup/restore method must correctly not just reinstall files and set their ownership and permissions, but also all other extended attributes, specifically selinux contexts. I'm not aware of anything in Fedora that uses other extended attributes for something, but there may be.
This is less of an issue when restoring just your home directory from backups.
That makes sense, thanks. I will also need to restore some things in /etc of course, but that can be done manually.
poc
On 11/2/20 4:45 AM, Patrick O'Callaghan wrote:
I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
On Mon, Nov 2, 2020 at 5:45 AM Patrick O'Callaghan pocallaghan@gmail.com wrote:
I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
ext4 can be converted to Btrfs but I can't strongly recommend it because you're not going to get the same layout as a default installation. The conversion won't remove LVM, and it won't add the subvolume layout we're using where "home" and "root" subvolumes are assigned to /home and / mountpoints respectively.
What would be the best way to approach this?:
- Do a system upgrade and then convert to BTRFS by backing everything
up and restoring it (I'd need guidance on how to do this).
- Do a complete system install and then restore from backups.
I'm guessing that (2) is the simplest answer, but I'd appreciate any comments, especially from people who have actually done either of these.
Top choice: Backup /home. Optionally /etc. And hand it over to the installer for complete wipe and clean install. From scratch setup. And after going through initial setup, restore /home (specifically restore the contents of ~/ for each user). Probably the most straightforward.
Second choice: Esoteric but a rather neat trick, is btrfs conversion, snapshot root and home. And use Btrfs send/receive to populate a new Btrfs file system with those snapshots. The conversion to Btrfs is merely a means to being able to use send/receive to replicate them. You get to keep your customizations without a clean install, but you do get the subvolume layout of a clean install. It is a bit partition-ninja. And there are post steps like all the bootloader stuff. It really depends on how comfortable you are with a rather low level process of migrating the data, almost inevitably messing it up, and working through the screwups. I've done quite a few of these and manage to screw it up somehow, and have to backtrack but I also don't panic easily, not least of which is a bunch of backups. So no matter how badly I mess it up I know I'm not losing things I care about.
On 03/11/2020 07:13, Chris Murphy wrote:
On Mon, Nov 2, 2020 at 5:45 AM Patrick O'Callaghan pocallaghan@gmail.com wrote:
I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
ext4 can be converted to Btrfs but I can't strongly recommend it because you're not going to get the same layout as a default installation. The conversion won't remove LVM, and it won't add the subvolume layout we're using where "home" and "root" subvolumes are assigned to /home and / mountpoints respectively.
What would be the best way to approach this?:
[snip]
- Do a complete system install and then restore from backups.
I'm guessing that (2) is the simplest answer, but I'd appreciate any comments, especially from people who have actually done either of these.
Top choice: Backup /home. Optionally /etc. And hand it over to the installer for complete wipe and clean install. From scratch setup. And after going through initial setup, restore /home (specifically restore the contents of ~/ for each user). Probably the most straightforward.
Is this the best advice for complicated setups too? My main computer has been upgraded again and again since about Fedora 18, with parts being changed out as and when. When new disks are added the old ones remain until they die; where I have multiple disks of the same size I RAID them for performance. My current disk setup looks like this:
root@snoopy 09:22:35 ~ # lsblk --merge NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 111.8G 0 disk ├─sda1 8:1 0 500M 0 part /boot ┌┈▶ └─sda2 8:2 0 111.3G 0 part ┆ sdb 8:16 0 111.8G 0 disk ┆ ├─sdb1 8:17 0 64G 0 part [SWAP] └┬▶ └─sdb2 8:18 0 47.8G 0 part └┈┈fedora_snoopy-root 253:0 0 159.1G 0 lvm / sdc 8:32 1 14.6T 0 disk └─sdc1 8:33 1 14.6T 0 part /backup sdd 8:48 0 931.5G 0 disk ┌┈▶ └─sdd1 8:49 0 931.5G 0 part ┆ sde 8:64 0 931.5G 0 disk ├┈▶ └─sde1 8:65 0 931.5G 0 part ┆ sdf 8:80 1 3.7T 0 disk ┌┈▶┆ └─sdf1 8:81 1 3.7T 0 part ┆ ┆ sdg 8:96 1 3.7T 0 disk └┬▶┆ └─sdg1 8:97 1 3.7T 0 part ┌┈▶ └┈┆┈┈┈md1 9:1 0 7.3T 0 raid0 ┆ ┆ sdh 8:112 0 931.5G 0 disk ┆ ├┈▶ └─sdh1 8:113 0 931.5G 0 part ┆ ┆ sdi 8:128 0 931.5G 0 disk ┆ └┬▶ └─sdi1 8:129 0 931.5G 0 part └┬▶ └┈┈md0 9:0 0 3.7T 0 raid0 └┈┈┈┈┈┈┈┈vg_home-lv_home 253:1 0 10.9T 0 lvm /home sr0 11:0 1 496.7M 0 rom
If I do a clean install of Fedora 33 onto this will it cope? I am hoping that BTRFS / Anaconda will spot all those disks and make optimal use of them, better even than my manual arrangement. Is that a fair assumption, or would I be better just upgrading F32 -> F33 as previously?
Many thanks, Chris R.
On Tue, Nov 3, 2020 at 2:25 AM Christopher Ross fdra6390@tebibyte.org wrote:
Is this the best advice for complicated setups too? My main computer has been upgraded again and again since about Fedora 18, with parts being changed out as and when. When new disks are added the old ones remain until they die; where I have multiple disks of the same size I RAID them for performance.
Offhand I don't see a reason to change strategies. Over that time frame there are likely ext4 and XFS features that you don't have access to without a reformat. Offhand, checksummed journal and metadata (in both file systems). And reflink copies are now supported on XFS since about 1 year. Btrfs is a bit different in that it takes advantage of feature flags which can be set on existing file systems rather than a reformat [1].
My current disk setup looks like this:
root@snoopy 09:22:35 ~ # lsblk --merge NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 111.8G 0 disk ├─sda1 8:1 0 500M 0 part /boot ┌┈▶ └─sda2 8:2 0 111.3G 0 part ┆ sdb 8:16 0 111.8G 0 disk ┆ ├─sdb1 8:17 0 64G 0 part [SWAP] └┬▶ └─sdb2 8:18 0 47.8G 0 part └┈┈fedora_snoopy-root 253:0 0 159.1G 0 lvm / sdc 8:32 1 14.6T 0 disk └─sdc1 8:33 1 14.6T 0 part /backup sdd 8:48 0 931.5G 0 disk ┌┈▶ └─sdd1 8:49 0 931.5G 0 part ┆ sde 8:64 0 931.5G 0 disk ├┈▶ └─sde1 8:65 0 931.5G 0 part ┆ sdf 8:80 1 3.7T 0 disk ┌┈▶┆ └─sdf1 8:81 1 3.7T 0 part ┆ ┆ sdg 8:96 1 3.7T 0 disk └┬▶┆ └─sdg1 8:97 1 3.7T 0 part ┌┈▶ └┈┆┈┈┈md1 9:1 0 7.3T 0 raid0 ┆ ┆ sdh 8:112 0 931.5G 0 disk ┆ ├┈▶ └─sdh1 8:113 0 931.5G 0 part ┆ ┆ sdi 8:128 0 931.5G 0 disk ┆ └┬▶ └─sdi1 8:129 0 931.5G 0 part └┬▶ └┈┈md0 9:0 0 3.7T 0 raid0 └┈┈┈┈┈┈┈┈vg_home-lv_home 253:1 0 10.9T 0 lvm /home sr0 11:0 1 496.7M 0 rom
If I do a clean install of Fedora 33 onto this will it cope? I am hoping that BTRFS / Anaconda will spot all those disks and make optimal use of them, better even than my manual arrangement. Is that a fair assumption, or would I be better just upgrading F32 -> F33 as previously?
That's pretty complex. I'd say just keep it until you've got a good reason to change it. It's valid to put Btrfs on these device mapper layers. It might be easiest to do this in the Advanced-Custom partitioning UI so you can be very explicit about what you want. Automatic partitioning is intended for the more simple case of one or maybe two drives.
In the somewhat common case of two drives: NVMe and one large HDD, if you hand over both drives to the installer, Fedora 32 and older makes the largest partition on both drives PV's and adds them to one VG. And on Fedora 33 now using Btrfs it creates one big btrfs out of those partitions, in a functionally similar "concatenate" arrangement [2] This is probably suboptimal by most estimations. I think most people would probably want all the boot, system related things, and applications on the NVMe drive, and /home on the HDD. Automatic partitioning doesn't have a way to anticipate these things.
It's also valid to mix and match if you want. You can repurpose your existing root LV with a new clean install that uses Btrfs just for system root; and reuse your ext4 on LVM "home" LV assigned to /home mountpoint without reformatting. If you want a Btrfs /home at some point, you've got the option to either convert it; or do the backup, mkfs, restore dance.
Btrfs does not offer different raid levels for subvolumes. The redundancy profile is set separate on metadata block groups and data block groups, for that entire file system. So if you want raid0 for system stuff and raid1 for user files, those are still separate file systems.
[1] Three things are mkfs only options: (--csum) checksum algorithm is crc32c by default; (--nodesize) the metadata block size is 16KiB by default; (--mixed) mixed block groups instead of separate metadata and data block groups which is the default, tends to be used for small devices under 5GiB, like USB sticks. Everything else can be changed after mkfs time.
[2] The small difference is that the Btrfs mkfs default for multiple devices uses raid1 for the metadata block group. Metadata in the Btrfs context means the file system itself, such as the various trees that include things like file inodes and the references to where the data itself is located. To say "metadata profile is raid1" means two copies of the file system, meaning one device can file and the file system itself is OK. Since there's only one copy of data, anything on a failed drive is lost, so this is not likely a bootable system. But files on surviving drives are OK.
On Tue, 2020-11-03 at 00:13 -0700, Chris Murphy wrote:
On Mon, Nov 2, 2020 at 5:45 AM Patrick O'Callaghan pocallaghan@gmail.com wrote:
I'd normally upgrade, but my /dev/sda uses LVM to handle root, /home etc. and from what I read this cannot be converted directly to BTRFS, which I'm interested in using.
ext4 can be converted to Btrfs but I can't strongly recommend it because you're not going to get the same layout as a default installation. The conversion won't remove LVM, and it won't add the subvolume layout we're using where "home" and "root" subvolumes are assigned to /home and / mountpoints respectively.
OK.
What would be the best way to approach this?:
- Do a system upgrade and then convert to BTRFS by backing everything
up and restoring it (I'd need guidance on how to do this).
- Do a complete system install and then restore from backups.
I'm guessing that (2) is the simplest answer, but I'd appreciate any comments, especially from people who have actually done either of these.
Top choice: Backup /home. Optionally /etc. And hand it over to the installer for complete wipe and clean install. From scratch setup. And after going through initial setup, restore /home (specifically restore the contents of ~/ for each user). Probably the most straightforward.
Yes, that looks like what I'll go for.
Second choice: Esoteric but a rather neat trick, is btrfs conversion, snapshot root and home. And use Btrfs send/receive to populate a new Btrfs file system with those snapshots. The conversion to Btrfs is merely a means to being able to use send/receive to replicate them. You get to keep your customizations without a clean install, but you do get the subvolume layout of a clean install. It is a bit partition-ninja. And there are post steps like all the bootloader stuff. It really depends on how comfortable you are with a rather low level process of migrating the data, almost inevitably messing it up, and working through the screwups. I've done quite a few of these and manage to screw it up somehow, and have to backtrack but I also don't panic easily, not least of which is a bunch of backups. So no matter how badly I mess it up I know I'm not losing things I care about.
Here be dragons :-) No thanks, I had enough trouble getting my head around LVM (one reason I'm keen to move to BTRFS).
poc