From: Harald Hoyer harald@redhat.com
This patch is to install everything to /lib/modules/$KERNEL_VERSION and only have %ghost files for /boot files.
Installation in %posttrans with kernel-install should work already.
F20: systemd >= 208-31 F21: systemd >= 216-21 F >= 22: systemd >= 215-12
The main target is to have all rpms install only to /usr, so in the end we would have a self contained installation in /usr.
As of now, we still have to cope with /var and /etc, but you have to start somewhere. --- kernel.spec | 39 +++++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/kernel.spec b/kernel.spec index 74e27fb..a7ae323 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1538,19 +1538,24 @@ BuildKernel() { %{make} -s ARCH=$Arch V=1 %{?_smp_mflags} $MakeTarget %{?sparse_mflags} %{?kernel_mflags} %{make} -s ARCH=$Arch V=1 %{?_smp_mflags} modules %{?sparse_mflags} || exit 1
+ mkdir -p $RPM_BUILD_ROOT/%{image_install_path} + mkdir -p $RPM_BUILD_ROOT/lib/modules/$KernelVer +%if %{with_debuginfo} + mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/boot + mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/%{image_install_path} +%endif + %ifarch %{arm} aarch64 %{make} -s ARCH=$Arch V=1 dtbs dtbs_install INSTALL_DTBS_PATH=$RPM_BUILD_ROOT/%{image_install_path}/dtb-$KernelVer + cp $RPM_BUILD_ROOT/%{image_install_path}/dtb-$KernelVer find arch/$Arch/boot/dts -name '*.dtb' -type f | xargs rm -f %endif
# Start installing the results -%if %{with_debuginfo} - mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/boot - mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/%{image_install_path} -%endif - mkdir -p $RPM_BUILD_ROOT/%{image_install_path} install -m 644 .config $RPM_BUILD_ROOT/boot/config-$KernelVer + install -m 644 .config $RPM_BUILD_ROOT/lib/modules/$KernelVer/config install -m 644 System.map $RPM_BUILD_ROOT/boot/System.map-$KernelVer + install -m 644 System.map $RPM_BUILD_ROOT/lib/modules/$KernelVer/System.map
# We estimate the size of the initramfs because rpm needs to take this size # into consideration when performing disk space calculations. (See bz #530778) @@ -1558,6 +1563,7 @@ BuildKernel() {
if [ -f arch/$Arch/boot/zImage.stub ]; then cp arch/$Arch/boot/zImage.stub $RPM_BUILD_ROOT/%{image_install_path}/zImage.stub-$KernelVer || : + cp arch/$Arch/boot/zImage.stub $RPM_BUILD_ROOT/lib/modules/$KernelVer/zImage.stub-$KernelVer || : fi %if %{signmodules} # Sign the image if we're using EFI @@ -1571,13 +1577,14 @@ BuildKernel() { $CopyKernel $KernelImage \ $RPM_BUILD_ROOT/%{image_install_path}/$InstallName-$KernelVer chmod 755 $RPM_BUILD_ROOT/%{image_install_path}/$InstallName-$KernelVer + cp $RPM_BUILD_ROOT/%{image_install_path}/$InstallName-$KernelVer $RPM_BUILD_ROOT/lib/modules/$KernelVer/$InstallName
# hmac sign the kernel for FIPS echo "Creating hmac file: $RPM_BUILD_ROOT/%{image_install_path}/.vmlinuz-$KernelVer.hmac" ls -l $RPM_BUILD_ROOT/%{image_install_path}/$InstallName-$KernelVer sha512hmac $RPM_BUILD_ROOT/%{image_install_path}/$InstallName-$KernelVer | sed -e "s,$RPM_BUILD_ROOT,," > $RPM_BUILD_ROOT/%{image_install_path}/.vmlinuz-$KernelVer.hmac; + cp $RPM_BUILD_ROOT/%{image_install_path}/.vmlinuz-$KernelVer.hmac $RPM_BUILD_ROOT/lib/modules/$KernelVer/.vmlinuz.hmac
- mkdir -p $RPM_BUILD_ROOT/lib/modules/$KernelVer # Override $(mod-fw) because we don't want it to install any firmware # we'll get it from the linux-firmware package and we don't want conflicts %{make} -s ARCH=$Arch INSTALL_MOD_PATH=$RPM_BUILD_ROOT modules_install KERNELRELEASE=$KernelVer mod-fw= @@ -1990,7 +1997,6 @@ popd make DESTDIR=$RPM_BUILD_ROOT bootwrapper_install WRAPPER_OBJDIR=%{_libdir}/kernel-wrapper WRAPPER_DTSDIR=%{_libdir}/kernel-wrapper/dts %endif
- ### ### clean ### @@ -2061,7 +2067,7 @@ fi\ # %define kernel_variant_posttrans() \ %{expand:%%posttrans %{?1:%{1}-}core}\ -/bin/kernel-install add %{KVERREL}%{?1:+%{1}} /%{image_install_path}/vmlinuz-%{KVERREL}%{?1:+%{1}} || exit $?\ +/bin/kernel-install add %{KVERREL}%{?1:+%{1}} /lib/modules/%{KVERREL}%{?1:+%{1}}/vmlinuz || exit $?\ %{nil}
# @@ -2088,7 +2094,7 @@ fi}\ # %define kernel_variant_preun() \ %{expand:%%preun %{?1:%{1}-}core}\ -/bin/kernel-install remove %{KVERREL}%{?1:+%{1}} /%{image_install_path}/vmlinuz-%{KVERREL}%{?1:+%{1}} || exit $?\ +/bin/kernel-install remove %{KVERREL}%{?1:+%{1}} /lib/modules/%{KVERREL}%{?1:+%{1}}/vmlinuz || exit $?\ %{nil}
%kernel_variant_preun @@ -2206,13 +2212,18 @@ fi %defattr(-,root,root)\ %{!?_licensedir:%global license %%doc}\ %license linux-%{KVERREL}/COPYING\ -/%{image_install_path}/%{?-k:%{-k*}}%{!?-k:vmlinuz}-%{KVERREL}%{?2:+%{2}}\ -/%{image_install_path}/.vmlinuz-%{KVERREL}%{?2:+%{2}}.hmac \ +/lib/modules/%{KVERREL}%{?2:+%{2}}/%{?-k:%{-k*}}%{!?-k:vmlinuz}\ +%ghost /%{image_install_path}/%{?-k:%{-k*}}%{!?-k:vmlinuz}-%{KVERREL}%{?2:+%{2}}\ +/lib/modules/%{KVERREL}%{?2:+%{2}}/.vmlinuz.hmac \ +%ghost /%{image_install_path}/.vmlinuz-%{KVERREL}%{?2:+%{2}}.hmac \ %ifarch %{arm} aarch64\ -/%{image_install_path}/dtb-%{KVERREL}%{?2:+%{2}} \ +/lib/modules/%{KVERREL}%{?2:+%{2}}/dtb \ +%ghost /%{image_install_path}/dtb-%{KVERREL}%{?2:+%{2}} \ %endif\ -%attr(600,root,root) /boot/System.map-%{KVERREL}%{?2:+%{2}}\ -/boot/config-%{KVERREL}%{?2:+%{2}}\ +%attr(600,root,root) /lib/modules/%{KVERREL}%{?2:+%{2}}/System.map\ +%ghost /boot/System.map-%{KVERREL}%{?2:+%{2}}\ +/lib/modules/%{KVERREL}%{?2:+%{2}}/config\ +%ghost /boot/config-%{KVERREL}%{?2:+%{2}}\ %ghost /boot/initramfs-%{KVERREL}%{?2:+%{2}}.img\ %dir /lib/modules\ %dir /lib/modules/%{KVERREL}%{?2:+%{2}}\
Am 04.05.2015 um 19:22 schrieb harald@redhat.com:
From: Harald Hoyer harald@redhat.com
This patch is to install everything to /lib/modules/$KERNEL_VERSION and only have %ghost files for /boot files.
Installation in %posttrans with kernel-install should work already.
F20: systemd >= 208-31 F21: systemd >= 216-21 F >= 22: systemd >= 215-12
The main target is to have all rpms install only to /usr, so in the end we would have a self contained installation in /usr
what the hell - the kernel itself belongs to /boot without any but or if - period, otherwise i would have had a lot of fun for things like the transition to grub2 and need to move the first partition
in fact here on all machiens it was just get one system with it's *decicated /boot disk* to come up and prepare that change by just copy the dd-image of the first host via SSH to 30 other machines, unmount /boot, dd the image back there and the distr-upgrade was fine on all production machines
go away with your "everything needs to be below /usr"
kernel-core /boot/.vmlinuz-3.19.6-200.fc21.x86_64.hmac kernel-core /boot/System.map-3.19.6-200.fc21.x86_64 kernel-core /boot/config-3.19.6-200.fc21.x86_64 kernel-core /boot/initramfs-3.19.6-200.fc21.x86_64.img kernel-core /boot/vmlinuz-3.19.6-200.fc21.x86_64
On Mon, May 04, 2015 at 07:32:39PM +0200, Reindl Harald wrote:
Am 04.05.2015 um 19:22 schrieb harald@redhat.com:
From: Harald Hoyer harald@redhat.com
This patch is to install everything to /lib/modules/$KERNEL_VERSION and only have %ghost files for /boot files.
Installation in %posttrans with kernel-install should work already.
F20: systemd >= 208-31 F21: systemd >= 216-21 F >= 22: systemd >= 215-12
The main target is to have all rpms install only to /usr, so in the end we would have a self contained installation in /usr
what the hell - the kernel itself belongs to /boot without any but or if - period, otherwise i would have had a lot of fun for things like the transition to grub2 and need to move the first partition
in fact here on all machiens it was just get one system with it's *decicated /boot disk* to come up and prepare that change by just copy the dd-image of the first host via SSH to 30 other machines, unmount /boot, dd the image back there and the distr-upgrade was fine on all production machines
go away with your "everything needs to be below /usr"
It's not obvious from the patch or description, but the %posttrans that Harald mentioned actually copies the files from /usr to /boot. At least as far as I understand. Then on removal, it will remove them as well.
So if I understand the patch correctly, your usecase will still be valid as the actual files are still in /boot. Nobody is expecting grub (or whatever other loaded) to read files out of /usr/.
josh
On 05/04/2015 01:49 PM, Josh Boyer wrote:
So if I understand the patch correctly, your usecase will still be valid as the actual files are still in /boot. Nobody is expecting grub (or whatever other loaded) to read files out of /usr/.
Moving files in %posttrans to a different location seems like one heck of a hack. What problem are we trying to solve here with a "self-contained installation in /usr"?
~tom
== Red Hat
Meh, not in CC and not subscribed to kernel-list. Anyway...
On 04.05.2015 19:57, Tom Callaway wrote:
On 05/04/2015 01:49 PM, Josh Boyer wrote:
So if I understand the patch correctly, your usecase will still be valid as the actual files are still in /boot. Nobody is expecting grub (or whatever other loaded) to read files out of /usr/.
Moving files in %posttrans to a different location seems like one heck of a hack. What problem are we trying to solve here with a "self-contained installation in /usr"?
~tom
== Red Hat
"heck of a hack"?? We do a lot more for the kernel in postrans: - depmod - create the initrd and move it to /boot - modify the bootloader conf (grubby)
I don't see why additionally copying some files make it a "heck of a hack"?
With "self-contained installation in /usr" we want to minimize the location of distributed files, so we can have: - stateless systems with a shared /usr gold master - easy distribution of vendor supplied OS parts - factory reset
In our model: - /usr is vendor supplied - /var is OS state - /etc is admin config - /boot is machine owned and sometimes even shared with other OS (EFI partition )
Devconf2012 talk (3 years ago???!!!!): https://rvokal.fedorapeople.org/devconf2012/harald-A_streamlined_and_fully_com patible_Linux_Files.pdf
Lennart's blog post: http://0pointer.de/blog/projects/stateless.html
Hi,
In our model:
- /usr is vendor supplied
- /var is OS state
- /etc is admin config
- /boot is machine owned and sometimes even shared with other OS (EFI partition
Well, at least in fedora + rhel the efi partition is /boot/efi ... I suspect you are not happy with that? Care to outline why and what your plans are?
One advantage I can see when the efi partition becomes /boot is that the uefi firmware can see the kernels and booting becomes simpler: No need for ext2 filesystem support in the boot loader, and you can probably even use uefi shell to boot your system.
cheers, Gerd
On 06.05.2015 13:55, Gerd Hoffmann wrote:
Hi,
In our model:
- /usr is vendor supplied
- /var is OS state
- /etc is admin config
- /boot is machine owned and sometimes even shared with other OS (EFI partition
Well, at least in fedora + rhel the efi partition is /boot/efi ... I suspect you are not happy with that? Care to outline why and what your plans are?
Well, if you use gummiboot, /boot can/should be the EFI partition.
One advantage I can see when the efi partition becomes /boot is that the uefi firmware can see the kernels and booting becomes simpler: No need for ext2 filesystem support in the boot loader, and you can probably even use uefi shell to boot your system.
That is exactly what we do with the bootloader spec [1] (BLS) and gummiboot.
install-kernel supports this scheme along with the old one.
If a bootloader is used, which supports the BLS, then multiple linux installations can happily share /boot.
[1] http://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/
And it even gets better with combined kernel+initrd+cmdline uefi executables [2], which enables secure boot signing of even the initrd and the cmdline.
[2] https://harald.hoyer.xyz/2015/02/25/single-uefi-executable-for-kernelinitrdcmdline/
So, what is installed to /boot should not already be decided in the rpm, but in install-kernel.
On Wed, May 06, 2015 at 11:34:41AM +0200, Harald Hoyer wrote:
Meh, not in CC and not subscribed to kernel-list. Anyway...
On 04.05.2015 19:57, Tom Callaway wrote:
On 05/04/2015 01:49 PM, Josh Boyer wrote:
So if I understand the patch correctly, your usecase will still be valid as the actual files are still in /boot. Nobody is expecting grub (or whatever other loaded) to read files out of /usr/.
Moving files in %posttrans to a different location seems like one heck of a hack. What problem are we trying to solve here with a "self-contained installation in /usr"?
~tom
== Red Hat
"heck of a hack"?? We do a lot more for the kernel in postrans:
- depmod
- create the initrd and move it to /boot
- modify the bootloader conf (grubby)
I don't see why additionally copying some files make it a "heck of a hack"?
With "self-contained installation in /usr" we want to minimize the location of distributed files, so we can have:
- stateless systems with a shared /usr gold master
- easy distribution of vendor supplied OS parts
- factory reset
Hi,
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
This is one of those ideas, I am curious to see how it plays out. It can turn into nothing or allow us to do more interesting things from a package maintaince or sysadmin perspective.
The only problem is how does one go about implementing ideas like this, aside from creating their own distro?
If all we are doing is adding new files to /lib/modules, then it is low risk, I would think. But I would probably keep this in rawhide somehow (if at all possible).
Then again I like some of the ideas of the stateless model as it makes updating machines (servers big and small) easier. I almost think Docker but with distros instead of apps.
Just my 2cents.
Cheers, Don
On Wed, May 06, 2015 at 10:29:36AM -0400, Don Zickus wrote:
On Wed, May 06, 2015 at 11:34:41AM +0200, Harald Hoyer wrote:
Meh, not in CC and not subscribed to kernel-list. Anyway...
On 04.05.2015 19:57, Tom Callaway wrote:
On 05/04/2015 01:49 PM, Josh Boyer wrote:
So if I understand the patch correctly, your usecase will still be valid as the actual files are still in /boot. Nobody is expecting grub (or whatever other loaded) to read files out of /usr/.
Moving files in %posttrans to a different location seems like one heck of a hack. What problem are we trying to solve here with a "self-contained installation in /usr"?
~tom
== Red Hat
"heck of a hack"?? We do a lot more for the kernel in postrans:
- depmod
- create the initrd and move it to /boot
- modify the bootloader conf (grubby)
I don't see why additionally copying some files make it a "heck of a hack"?
With "self-contained installation in /usr" we want to minimize the location of distributed files, so we can have:
- stateless systems with a shared /usr gold master
- easy distribution of vendor supplied OS parts
- factory reset
Hi,
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
Actually, I was hoping some other kernel maintainers would chip in so your opinion does matter. I really don't want to change this in Fedora to only have it reverted in a future RHEL. Maybe Jarod or Rafael would be kind enough to review as well...
And yes, your summary is correct.
This is one of those ideas, I am curious to see how it plays out. It can turn into nothing or allow us to do more interesting things from a package maintaince or sysadmin perspective.
The only problem is how does one go about implementing ideas like this, aside from creating their own distro?
If all we are doing is adding new files to /lib/modules, then it is low risk, I would think. But I would probably keep this in rawhide somehow (if at all possible).
If we apply it, it would start in rawhide and work its way through the normal Fedora release process. So at this point the earliest release it would land in would be Fedora 23. The backwards compatibility Harald noted was for ease of use in testing rawhide kernels on older userspace.
Then again I like some of the ideas of the stateless model as it makes updating machines (servers big and small) easier. I almost think Docker but with distros instead of apps.
Just my 2cents.
Thanks.
josh
On Wed, May 06, 2015 at 10:41:28AM -0400, Josh Boyer wrote:
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
Actually, I was hoping some other kernel maintainers would chip in so your opinion does matter. I really don't want to change this in Fedora to only have it reverted in a future RHEL. Maybe Jarod or Rafael would be kind enough to review as well...
Off the top of my head, if it works out for Fedora, I currently can't see a reason RHEL would revert it. But that depends on what quirks falls out. :-)
And yes, your summary is correct.
Thanks!
Cheers, Don
This is one of those ideas, I am curious to see how it plays out. It can turn into nothing or allow us to do more interesting things from a package maintaince or sysadmin perspective.
The only problem is how does one go about implementing ideas like this, aside from creating their own distro?
If all we are doing is adding new files to /lib/modules, then it is low risk, I would think. But I would probably keep this in rawhide somehow (if at all possible).
If we apply it, it would start in rawhide and work its way through the normal Fedora release process. So at this point the earliest release it would land in would be Fedora 23. The backwards compatibility Harald noted was for ease of use in testing rawhide kernels on older userspace.
Then again I like some of the ideas of the stateless model as it makes updating machines (servers big and small) easier. I almost think Docker but with distros instead of apps.
Just my 2cents.
Thanks.
josh
On Wed, May 06, 2015 at 11:48:00AM -0400, Don Zickus wrote:
On Wed, May 06, 2015 at 10:41:28AM -0400, Josh Boyer wrote:
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
Actually, I was hoping some other kernel maintainers would chip in so your opinion does matter. I really don't want to change this in Fedora to only have it reverted in a future RHEL. Maybe Jarod or Rafael would be kind enough to review as well...
Off the top of my head, if it works out for Fedora, I currently can't see a reason RHEL would revert it. But that depends on what quirks falls out. :-)
First pass through, I see a few oddities, some of which aren't the fault of this patch, but if manipulating these areas, might as well fix them up...
1) %image_install_path is never defined to anything but boot, for all supported arches. I think this is ia64 legacy, when it was /boot/efi, but we should just have a single define for it now.
2) we do this, both before and after the patch:
mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/boot mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/%{image_install_path}
...which per #1, is obviously redundantly redundant.
3) after the patch, there are multiple install calls to put stuff into $RPM_BUILD_ROOT/boot/, but then they're %ghost'ed out. This seems a little bit of a waste, just do a touch for the /boot variants of those files and %ghost the same. Similarly, with the .hmac file, don't copy it around, put it where you want the real one, touch the %ghost.
4) it appears there's already logic inside kernel-install to copy the necessary files over to /boot at install time, and with the %ghost, they'll properly report as being part of the kernel package, but how long has that support actually been in kernel-install? Do you possibly want to add an explicit Requires: systemd >= x-y, as noted in the patch header? (I would).
Those issues aside, this doesn't really look all that scary at all.
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
On Wed, May 6, 2015 at 2:53 PM, Jarod Wilson jarod@redhat.com wrote:
On Wed, May 06, 2015 at 11:48:00AM -0400, Don Zickus wrote:
On Wed, May 06, 2015 at 10:41:28AM -0400, Josh Boyer wrote:
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
Actually, I was hoping some other kernel maintainers would chip in so your opinion does matter. I really don't want to change this in Fedora to only have it reverted in a future RHEL. Maybe Jarod or Rafael would be kind enough to review as well...
Off the top of my head, if it works out for Fedora, I currently can't see a reason RHEL would revert it. But that depends on what quirks falls out. :-)
First pass through, I see a few oddities, some of which aren't the fault of this patch, but if manipulating these areas, might as well fix them up...
- %image_install_path is never defined to anything but boot, for all
supported arches. I think this is ia64 legacy, when it was /boot/efi, but we should just have a single define for it now.
we do this, both before and after the patch:
mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/boot mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/%{image_install_path}
...which per #1, is obviously redundantly redundant.
- after the patch, there are multiple install calls to put stuff into
$RPM_BUILD_ROOT/boot/, but then they're %ghost'ed out. This seems a little bit of a waste, just do a touch for the /boot variants of those files and %ghost the same. Similarly, with the .hmac file, don't copy it around, put it where you want the real one, touch the %ghost.
Actually, I don't think we want to do that. We want to account for the space that will be eaten by the real files that get copied to /boot so that RPM can do it's disk space requirements estimates. We don't want people to get 2/3 of the way installing a new kernel to have it fail because they don't have sufficient space in /boot.
(We do something similar for the initramfs file already in a different place, but that is literally generated from content outside the kernel RPM so we can't just install a real file).
- it appears there's already logic inside kernel-install to copy the
necessary files over to /boot at install time, and with the %ghost, they'll properly report as being part of the kernel package, but how long has that support actually been in kernel-install? Do you possibly want to add an explicit Requires: systemd >= x-y, as noted in the patch header? (I would).
Harald noted that all current Fedora releases are covered in the initial posting. We could add an explicit systemd Requires, but it'll just default to the oldest version in F20.
Those issues aside, this doesn't really look all that scary at all.
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
Copied as far as I know. Yes it's slightly inefficient, but worrying about that case (which isn't default at all) seems kind of pointless.
josh
On Wed, May 06, 2015 at 03:00:08PM -0400, Josh Boyer wrote:
On Wed, May 6, 2015 at 2:53 PM, Jarod Wilson jarod@redhat.com wrote:
On Wed, May 06, 2015 at 11:48:00AM -0400, Don Zickus wrote:
On Wed, May 06, 2015 at 10:41:28AM -0400, Josh Boyer wrote:
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
Actually, I was hoping some other kernel maintainers would chip in so your opinion does matter. I really don't want to change this in Fedora to only have it reverted in a future RHEL. Maybe Jarod or Rafael would be kind enough to review as well...
Off the top of my head, if it works out for Fedora, I currently can't see a reason RHEL would revert it. But that depends on what quirks falls out. :-)
First pass through, I see a few oddities, some of which aren't the fault of this patch, but if manipulating these areas, might as well fix them up...
- %image_install_path is never defined to anything but boot, for all
supported arches. I think this is ia64 legacy, when it was /boot/efi, but we should just have a single define for it now.
we do this, both before and after the patch:
mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/boot mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/%{image_install_path}
...which per #1, is obviously redundantly redundant.
- after the patch, there are multiple install calls to put stuff into
$RPM_BUILD_ROOT/boot/, but then they're %ghost'ed out. This seems a little bit of a waste, just do a touch for the /boot variants of those files and %ghost the same. Similarly, with the .hmac file, don't copy it around, put it where you want the real one, touch the %ghost.
Actually, I don't think we want to do that. We want to account for the space that will be eaten by the real files that get copied to /boot so that RPM can do it's disk space requirements estimates. We don't want people to get 2/3 of the way installing a new kernel to have it fail because they don't have sufficient space in /boot.
(We do something similar for the initramfs file already in a different place, but that is literally generated from content outside the kernel RPM so we can't just install a real file).
Ah, okay, that makes some sense. A bit wasteful on the packaging size, but sound justification.
- it appears there's already logic inside kernel-install to copy the
necessary files over to /boot at install time, and with the %ghost, they'll properly report as being part of the kernel package, but how long has that support actually been in kernel-install? Do you possibly want to add an explicit Requires: systemd >= x-y, as noted in the patch header? (I would).
Harald noted that all current Fedora releases are covered in the initial posting. We could add an explicit systemd Requires, but it'll just default to the oldest version in F20.
I think its probably safest to add it, lest someone try upgrading to this kernel from F19 or something. I know its not supported, but we can at least hand people the loaded gun with the safety on. :p
Those issues aside, this doesn't really look all that scary at all.
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
Copied as far as I know. Yes it's slightly inefficient, but worrying about that case (which isn't default at all) seems kind of pointless.
Yeah, really more idle curiosity, it'd be a systemd package issue regardless, and wouldn't really directly impact this patch being accepted or not.
On 05/06/2015 02:00 PM, Josh Boyer wrote:
On Wed, May 6, 2015 at 2:53 PM, Jarod Wilson jarod@redhat.com wrote:
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
Copied as far as I know. Yes it's slightly inefficient, but worrying about that case (which isn't default at all) seems kind of pointless.
Hmm. If don't know off the top of my head if Fedora cloud images have a separate /boot or not, but disk space is a big concern in such environments.
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images have a separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
On Thu, May 07, 2015 at 10:08:56PM -0400, Matthew Miller wrote:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images have a separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
What about amending the patch to use
cp --reflink=auto ...
?
Zbyszek
On May 8, 2015 11:38 AM, "Zbigniew Jędrzejewski-Szmek" zbyszek@in.waw.pl wrote:
On Thu, May 07, 2015 at 10:08:56PM -0400, Matthew Miller wrote:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images
have a
separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
What about amending the patch to use
cp --reflink=auto ...
?
The kernel-install tool is what would need to be patched. It is what is doing the copying at install time. Doing it in the spec is pointless as we're mucking around in the build root not the actual system.
josh
On Fri, May 08, 2015 at 05:11:15PM -0400, Josh Boyer wrote:
On May 8, 2015 11:38 AM, "Zbigniew Jędrzejewski-Szmek" zbyszek@in.waw.pl wrote:
On Thu, May 07, 2015 at 10:08:56PM -0400, Matthew Miller wrote:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images
have a
separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
What about amending the patch to use
cp --reflink=auto ...
?
The kernel-install tool is what would need to be patched. It is what is doing the copying at install time. Doing it in the spec is pointless as we're mucking around in the build root not the actual system.
Strictly speaking, using --reflink=auto would be useful in the spec too, to avoid copying the files. In a build root it is very likely to succeed too.
But you're right of course, it's kernel-install that counts.
Patches for kernel-install and dracut attached.
Zbyszek
On Fri, May 08, 2015 at 05:11:15PM -0400, Josh Boyer wrote:
On May 8, 2015 11:38 AM, "Zbigniew Jędrzejewski-Szmek" zbyszek@in.waw.pl wrote:
On Thu, May 07, 2015 at 10:08:56PM -0400, Matthew Miller wrote:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images
have a
separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
What about amending the patch to use
cp --reflink=auto ...
?
The kernel-install tool is what would need to be patched. It is what is doing the copying at install time. Doing it in the spec is pointless as we're mucking around in the build root not the actual system.
Strictly speaking, using --reflink=auto would be useful in the spec too, to avoid copying the files. In a build root it is very likely to succeed too.
But you're right of course, it's kernel-install that counts.
Patches for kernel-install and dracut attached.
[Since this patch is for systemd, resending it to systemd-devel. I should have done in the first place. If intend to push it in a few days if nobody objects.]
Zbyszek
On Sat, 16.05.15 22:16, Zbigniew Jędrzejewski-Szmek (zbyszek@in.waw.pl) wrote:
On Fri, May 08, 2015 at 05:11:15PM -0400, Josh Boyer wrote:
On May 8, 2015 11:38 AM, "Zbigniew Jędrzejewski-Szmek" zbyszek@in.waw.pl wrote:
On Thu, May 07, 2015 at 10:08:56PM -0400, Matthew Miller wrote:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images
have a
separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
What about amending the patch to use
cp --reflink=auto ...
?
The kernel-install tool is what would need to be patched. It is what is doing the copying at install time. Doing it in the spec is pointless as we're mucking around in the build root not the actual system.
Strictly speaking, using --reflink=auto would be useful in the spec too, to avoid copying the files. In a build root it is very likely to succeed too.
But you're right of course, it's kernel-install that counts.
Patches for kernel-install and dracut attached.
[Since this patch is for systemd, resending it to systemd-devel. I should have done in the first place. If intend to push it in a few days if nobody objects.]
--reflink=auto is the default in cp since a while. Instead of cluttering systemd's or dracut's trees with it, I'd rather recommend upgrading coreutils as necessary.
I don't think this should go in,
Lennart
On Mon, May 18, 2015 at 02:15:09PM +0200, Lennart Poettering wrote:
On Sat, 16.05.15 22:16, Zbigniew Jędrzejewski-Szmek (zbyszek@in.waw.pl) wrote:
On Fri, May 08, 2015 at 05:11:15PM -0400, Josh Boyer wrote:
On May 8, 2015 11:38 AM, "Zbigniew Jędrzejewski-Szmek" zbyszek@in.waw.pl wrote:
On Thu, May 07, 2015 at 10:08:56PM -0400, Matthew Miller wrote:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
Hmm. If don't know off the top of my head if Fedora cloud images
have a
separate /boot or not, but disk space is a big concern in such environments.
They don't have a separate boot, fwiw.
What about amending the patch to use
cp --reflink=auto ...
?
The kernel-install tool is what would need to be patched. It is what is doing the copying at install time. Doing it in the spec is pointless as we're mucking around in the build root not the actual system.
Strictly speaking, using --reflink=auto would be useful in the spec too, to avoid copying the files. In a build root it is very likely to succeed too.
But you're right of course, it's kernel-install that counts.
Patches for kernel-install and dracut attached.
[Since this patch is for systemd, resending it to systemd-devel. I should have done in the first place. If intend to push it in a few days if nobody objects.]
--reflink=auto is the default in cp since a while. Instead of cluttering systemd's or dracut's trees with it, I'd rather recommend upgrading coreutils as necessary.
I don't think that's true. With today's git, without --reflink=auto, cp does a normal read() + write(). I think the default for mv is to use reflink though.
Zbyszek
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
On 05/06/2015 02:00 PM, Josh Boyer wrote:
On Wed, May 6, 2015 at 2:53 PM, Jarod Wilson jarod@redhat.com wrote:
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
Copied as far as I know. Yes it's slightly inefficient, but worrying about that case (which isn't default at all) seems kind of pointless.
Hmm. If don't know off the top of my head if Fedora cloud images have a separate /boot or not, but disk space is a big concern in such environments.
You could also argue simplifying package maintenance and sysadmin stuff is a big concern for throw away cloud images. ;-) With a change like this you could almost throw away /boot and regenerate it on the fly. :-)
As I stated in another thread, it is hard to know what interesting things you could do with this approach if you don't play around with it.
Maybe if you detect /boot is on the same partition as /usr you go completely nuts and symlink the kernel images to the /usr/lib/modules ones (or maybe just /lib if /lib/modules hasn't moved to /usr/lib/modules yet).
Cheers, Don
Am 08.05.2015 um 16:51 schrieb Don Zickus:
On Thu, May 07, 2015 at 04:29:11PM -0500, Ian Pilcher wrote:
On 05/06/2015 02:00 PM, Josh Boyer wrote:
On Wed, May 6, 2015 at 2:53 PM, Jarod Wilson jarod@redhat.com wrote:
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
Copied as far as I know. Yes it's slightly inefficient, but worrying about that case (which isn't default at all) seems kind of pointless.
Hmm. If don't know off the top of my head if Fedora cloud images have a separate /boot or not, but disk space is a big concern in such environments.
You could also argue simplifying package maintenance and sysadmin stuff is a big concern for throw away cloud images. ;-) With a change like this you could almost throw away /boot and regenerate it on the fly. :-)
invalid argumentation because if you follow that statement you could say "throw away all computers at all"
but to keep on topic: copy the kernel files to the system partition perverts the recent improvement of kernel-core / kernel-modules
As I stated in another thread, it is hard to know what interesting things you could do with this approach if you don't play around with it.
interesting things?
boot don't need to be interesting it needs to be relieable and the sepearte /boot makes it so
otherwise some dist-upgrades over the years would have been impossible and yes i maintain some dozen production machines installed in 2008 with Fedora 9 now running Fedora 21 without re-install
Maybe if you detect /boot is on the same partition as /usr you go completely nuts and symlink the kernel images to the /usr/lib/modules ones (or maybe just /lib if /lib/modules hasn't moved to /usr/lib/modules yet)
UsrMove was long ago
On Wednesday, May 06, 2015 03:00:08 PM Josh Boyer wrote:
On Wed, May 6, 2015 at 2:53 PM, Jarod Wilson jarod@redhat.com wrote:
On Wed, May 06, 2015 at 11:48:00AM -0400, Don Zickus wrote:
On Wed, May 06, 2015 at 10:41:28AM -0400, Josh Boyer wrote:
Not that my opinion matters much, but I think this is an interesting mind shift. The end result is the same as today, just extra files in /lib/modules/`uname -r`, right?
Actually, I was hoping some other kernel maintainers would chip in so your opinion does matter. I really don't want to change this in Fedora to only have it reverted in a future RHEL. Maybe Jarod or Rafael would be kind enough to review as well...
Off the top of my head, if it works out for Fedora, I currently can't see a reason RHEL would revert it. But that depends on what quirks falls out. :-)>
First pass through, I see a few oddities, some of which aren't the fault of this patch, but if manipulating these areas, might as well fix them up...
- %image_install_path is never defined to anything but boot, for all
supported arches. I think this is ia64 legacy, when it was /boot/efi, but we should just have a single define for it now.
- we do this, both before and after the patch: mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/boot mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/%{image_install_path}
...which per #1, is obviously redundantly redundant.
- after the patch, there are multiple install calls to put stuff into
$RPM_BUILD_ROOT/boot/, but then they're %ghost'ed out. This seems a little bit of a waste, just do a touch for the /boot variants of those files and %ghost the same. Similarly, with the .hmac file, don't copy it around, put it where you want the real one, touch the %ghost.
Actually, I don't think we want to do that. We want to account for the space that will be eaten by the real files that get copied to /boot so that RPM can do it's disk space requirements estimates. We don't want people to get 2/3 of the way installing a new kernel to have it fail because they don't have sufficient space in /boot.
(We do something similar for the initramfs file already in a different place, but that is literally generated from content outside the kernel RPM so we can't just install a real file).
- it appears there's already logic inside kernel-install to copy the
necessary files over to /boot at install time, and with the %ghost, they'll properly report as being part of the kernel package, but how long has that support actually been in kernel-install? Do you possibly want to add an explicit Requires: systemd >= x-y, as noted in the patch header? (I would).
Harald noted that all current Fedora releases are covered in the initial posting. We could add an explicit systemd Requires, but it'll just default to the oldest version in F20.
Those issues aside, this doesn't really look all that scary at all.
One other thought: what happens when /boot is on the same file system as /usr and/or /lib? Does the file get unnecessarily copied, or is it hardlinked or _____?
Copied as far as I know. Yes it's slightly inefficient, but worrying about that case (which isn't default at all) seems kind of pointless.
It actually makes a lot of sense to hardlink, the default filesystem setup for cloud images is just / and in the cloud they worry a lot about wasted space.
Dennis
On Mon, May 4, 2015 at 1:22 PM, harald@redhat.com wrote:
From: Harald Hoyer harald@redhat.com
This patch is to install everything to /lib/modules/$KERNEL_VERSION and only have %ghost files for /boot files.
Installation in %posttrans with kernel-install should work already.
F20: systemd >= 208-31 F21: systemd >= 216-21 F >= 22: systemd >= 215-12
The main target is to have all rpms install only to /usr, so in the end we would have a self contained installation in /usr.
As of now, we still have to cope with /var and /etc, but you have to start somewhere.
I've been testing this locally today across all supported releases and with a variety of different machines and scenarios. It's tested fairly well, with the end result being no real change noticable after the install/update. Unless there are major objections, I plan on committing this Monday and I'll follow up with a few commits for Jarod's fixes.
(I would commit it today, but I'm on PTO tomorrow and there likely won't be another build until Monday anyway.)
josh
kernel@lists.fedoraproject.org