Hi All,
As has been discussed a few times on-list, the Cloud people have asked us to look at reducing the overall installed size for the kernel. I've poked at this for a bit now and come up with the following patch to split the kernel into kernel-core and kernel-drivers subpackages. This is part of the "Modular Kernel Packaging" F21 Change request [1].
I'd really appreciate it if people could look the patch over and provide further feedback and/or Acks. I've done scratch builds on all arches successfully and listed them below. I've also been maintaining this as a COPR for a while, and initial testing seems to be going well.
s390x: http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1371179 aarch64: http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=2266137 ppc/ppc64: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=1759593 primary arches: http://koji.fedoraproject.org/koji/taskinfo?taskID=6695887
COPR: http://copr-fe.cloud.fedoraproject.org/coprs/jwboyer/kernel-core/
josh
[1] https://fedoraproject.org/wiki/Changes/Modular_Kernel_Packaging_for_Cloud
This creates kernel-core and kernel-drivers subpackages. The kernel package remains as a meta-package the requires both of the subpackages. This allows most installs to continue on as-is with upgrades working.
The contents of the kernel-core and kernel-drivers subpackages are determined at build time through the filter-modules.sh script. This script "removes" pre-defined subsystems and modules and generates a filelist for the %files section of each subpackage. The contents of each are per-arch, with arch override files taken into account. This allows us to make the split useful for varying arches. --- filter-aarch64.sh | 5 ++ filter-armv7hl.sh | 5 ++ filter-i686.sh | 5 ++ filter-modules.sh | 132 +++++++++++++++++++++++++++++++++ filter-ppc.sh | 5 ++ filter-ppc64.sh | 5 ++ filter-ppc64le.sh | 5 ++ filter-s390x.sh | 3 + filter-x86_64.sh | 3 + kernel.spec | 213 ++++++++++++++++++++++++++++++++++++++++-------------- 10 files changed, 326 insertions(+), 55 deletions(-) create mode 100644 filter-aarch64.sh create mode 100644 filter-armv7hl.sh create mode 100644 filter-i686.sh create mode 100755 filter-modules.sh create mode 100644 filter-ppc.sh create mode 100644 filter-ppc64.sh create mode 100644 filter-ppc64le.sh create mode 100644 filter-s390x.sh create mode 100644 filter-x86_64.sh
diff --git a/filter-aarch64.sh b/filter-aarch64.sh new file mode 100644 index 0000000..ef3dc9f --- /dev/null +++ b/filter-aarch64.sh @@ -0,0 +1,5 @@ +#! /bin/bash + +driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject " diff --git a/filter-armv7hl.sh b/filter-armv7hl.sh new file mode 100644 index 0000000..476b630 --- /dev/null +++ b/filter-armv7hl.sh @@ -0,0 +1,5 @@ +#! /bin/bash + +driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn media memstick message mmc nfc ntb pcmcia platform ssb staging uio uwb" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject" diff --git a/filter-i686.sh b/filter-i686.sh new file mode 100644 index 0000000..0e98ded --- /dev/null +++ b/filter-i686.sh @@ -0,0 +1,5 @@ +#! /bin/bash + +driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mfd mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject hid-sensor-hub hid-sensor-magn-3d hid-sensor-incl-3d hid-sensor-als.ko hid-sensor-gyro-3d hid-sensor-iio-common hid-sensor-accel-3d hid-sensor-trigger hid-sensor-als" diff --git a/filter-modules.sh b/filter-modules.sh new file mode 100755 index 0000000..fd857c7 --- /dev/null +++ b/filter-modules.sh @@ -0,0 +1,132 @@ +#! /bin/bash +# +# Called as filter-modules.sh list-of-modules Arch + +# Set the default dirs/modules to filter out +driverdirs="atm auxdisplay bcma bluetooth fmc iio infiniband isdn leds media memstick message mfd mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb" + +netdrvs="appletalk dsa hamradio ieee802154 irda ppp slip usb wireless" + +ethdrvs="3com adaptec alteon amd atheros broadcom cadence calxeda chelsio cisco dec dlink emulex icplus marvell mellanox neterion nvidia oki-semi packetengines qlogic rdc renesas sfc silan sis smsc stmicro sun tehuti ti wiznet xircom" + +scsidrvs="aacraid aic7xxx aic94xx be2iscsi bfa bnx2i bnx2fc csiostor cxgbi esas2r fcoe fnic isci libsas lpfc megaraid mpt2sas mpt3sas mvsas pm8001 qla2xxx qla4xxx sym53c8xx_2 ufs" + +ttydrvs="ipwireless" + +usbdrvs="atm wusbcore" + +fsdrvs="affs befs coda cramfs dlm ecryptfs hfs hfsplus isofs jfs minix ncpfs nilfs2 ocfs2 reiserfs romfs squashfs sysv ubifs udf ufs" + +netprots="appletalk atm ax25 batman-adv bluetooth dccp dsa ieee802154 irda l2tp mac80211 mac802154 netrom nfc rds rfkill rose sctp wireless" + +drmdrvs="ast gma500 mgag200 via nouveau" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject hid-sensor-hub" + +# Grab the arch-specific filter list overrides +source ./filter-$2.sh + +filter_dir() { + filelist=$1 + dir=$2 + + grep -v -e "${dir}/" ${filelist} > ${filelist}.tmp + + if [ $? -ne 0 ] + then + echo "Couldn't remove ${dir}. Skipping." + else + grep -e "${dir}/" ${filelist} >> k-d.list + mv ${filelist}.tmp $filelist + fi + + return 0 +} + +filter_ko() { + filelist=$1 + mod=$2 + + grep -v -e "${mod}.ko" ${filelist} > ${filelist}.tmp + + if [ $? -ne 0 ] + then + echo "Couldn't remove ${mod}.ko Skipping." + else + grep -e "${mod}.ko" ${filelist} >> k-d.list + mv ${filelist}.tmp $filelist + fi + + return 0 +} + +# Filter the drivers/ subsystems +for subsys in ${driverdirs} +do + filter_dir $1 drivers/${subsys} +done + +# Filter the networking drivers +for netdrv in ${netdrvs} +do + filter_dir $1 drivers/net/${netdrv} +done + +# Filter the ethernet drivers +for eth in ${ethdrvs} +do + filter_dir $1 drivers/net/ethernet/${eth} +done + +# SCSI +for scsi in ${scsidrvs} +do + filter_dir $1 drivers/scsi/${scsi} +done + +# TTY +for tty in ${ttydrvs} +do + filter_dir $1 drivers/tty/${tty} +done + +# USB +for usb in ${usbdrvs} +do + filter_dir $1 drivers/usb/${usb} +done + +# Filesystems +for fs in ${fsdrvs} +do + filter_dir $1 fs/${fs} +done + +# Network protocols +for prot in ${netprots} +do + filter_dir $1 kernel/net/${prot} +done + +# DRM +for drm in ${drmdrvs} +do + filter_dir $1 drivers/gpu/drm/${drm} +done + +# Just kill sound. +filter_dir $1 kernel/sound + +# Now go through and filter any single .ko files that might have deps on the +# things we filtered above +for mod in ${singlemods} +do + filter_ko $1 ${mod} +done + +# Go through our generated drivers list and remove the .ko files. We'll +# restore them later. +for mod in `cat k-d.list` +do + rm -rf $mod +done diff --git a/filter-ppc.sh b/filter-ppc.sh new file mode 100644 index 0000000..83bdb29 --- /dev/null +++ b/filter-ppc.sh @@ -0,0 +1,5 @@ +#! /bin/bash + +driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject" diff --git a/filter-ppc64.sh b/filter-ppc64.sh new file mode 100644 index 0000000..83bdb29 --- /dev/null +++ b/filter-ppc64.sh @@ -0,0 +1,5 @@ +#! /bin/bash + +driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject" diff --git a/filter-ppc64le.sh b/filter-ppc64le.sh new file mode 100644 index 0000000..83bdb29 --- /dev/null +++ b/filter-ppc64le.sh @@ -0,0 +1,5 @@ +#! /bin/bash + +driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb" + +singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject" diff --git a/filter-s390x.sh b/filter-s390x.sh new file mode 100644 index 0000000..19252a3 --- /dev/null +++ b/filter-s390x.sh @@ -0,0 +1,3 @@ +#! /bin/bash + +# Defaults work so no need to override diff --git a/filter-x86_64.sh b/filter-x86_64.sh new file mode 100644 index 0000000..19252a3 --- /dev/null +++ b/filter-x86_64.sh @@ -0,0 +1,3 @@ +#! /bin/bash + +# Defaults work so no need to override diff --git a/kernel.spec b/kernel.spec index 5fc7e37..f5c8f53 100644 --- a/kernel.spec +++ b/kernel.spec @@ -34,7 +34,7 @@ Summary: The Linux kernel # For non-released -rc kernels, this will be appended after the rcX and # gitX tags, so a 3 here would become part of release "0.rcX.gitX.3" # -%global baserelease 1 +%global baserelease 9 %global fedora_build %{baserelease}
# base_sublevel is the kernel version we're starting with and patching @@ -385,29 +385,6 @@ Summary: The Linux kernel %define kernel_prereq fileutils, systemd >= 203-2 %define initrd_prereq dracut >= 027
-# -# This macro does requires, provides, conflicts, obsoletes for a kernel package. -# %%kernel_reqprovconf <subpackage> -# It uses any kernel_<subpackage>_conflicts and kernel_<subpackage>_obsoletes -# macros defined above. -# -%define kernel_reqprovconf \ -Provides: kernel = %{rpmversion}-%{pkg_release}\ -Provides: kernel-%{_target_cpu} = %{rpmversion}-%{pkg_release}%{?1:+%{1}}\ -Provides: kernel-drm-nouveau = 16\ -Provides: kernel-uname-r = %{KVERREL}%{?1:+%{1}}\ -Requires(pre): %{kernel_prereq}\ -Requires(pre): %{initrd_prereq}\ -Requires(pre): linux-firmware >= 20130724-29.git31f6b30\ -Requires(preun): systemd >= 200\ -%{expand:%%{?kernel%{?1:_%{1}}_conflicts:Conflicts: %%{kernel%{?1:_%{1}}_conflicts}}}\ -%{expand:%%{?kernel%{?1:_%{1}}_obsoletes:Obsoletes: %%{kernel%{?1:_%{1}}_obsoletes}}}\ -%{expand:%%{?kernel%{?1:_%{1}}_provides:Provides: %%{kernel%{?1:_%{1}}_provides}}}\ -# We can't let RPM do the dependencies automatic because it'll then pick up\ -# a correct but undesirable perl dependency from the module headers which\ -# isn't required for the kernel proper to function\ -AutoReqProv: no\ -%{nil}
Name: kernel%{?variant} Group: System Environment/Kernel @@ -419,8 +396,9 @@ Release: %{pkg_release} # SET %%nobuildarches (ABOVE) INSTEAD ExclusiveArch: noarch %{all_x86} x86_64 ppc ppc64 ppc64p7 s390 s390x %{arm} aarch64 ppc64le ExclusiveOS: Linux +Requires: kernel-%{?variant:%{variant}-}core-uname-r = %{KVERREL}%{?variant} +Requires: kernel-%{?variant:%{variant}-}drivers-uname-r = %{KVERREL}%{?variant}
-%kernel_reqprovconf
# # List the packages used during the kernel build @@ -464,6 +442,15 @@ Source15: merge.pl Source16: mod-extra.list Source17: mod-extra.sh Source18: mod-sign.sh +Source90: filter-x86_64.sh +Source91: filter-armv7hl.sh +Source92: filter-i686.sh +Source93: filter-aarch64.sh +Source94: filter-ppc.sh +Source95: filter-ppc64.sh +Source96: filter-ppc64le.sh +Source97: filter-s390x.sh +Source99: filter-modules.sh %define modsign_cmd %{SOURCE18}
Source19: Makefile.release @@ -652,10 +639,31 @@ Patch25052: net-xen-netback-disable-rogue-vif-in-kthread-context.patch BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root
%description -The kernel package contains the Linux kernel (vmlinuz), the core of any -Linux operating system. The kernel handles the basic functions -of the operating system: memory allocation, process allocation, device -input and output, etc. +The kernel meta package + +# +# This macro does requires, provides, conflicts, obsoletes for a kernel package. +# %%kernel_reqprovconf <subpackage> +# It uses any kernel_<subpackage>_conflicts and kernel_<subpackage>_obsoletes +# macros defined above. +# +%define kernel_reqprovconf \ +Provides: kernel = %{rpmversion}-%{pkg_release}\ +Provides: kernel-%{_target_cpu} = %{rpmversion}-%{pkg_release}%{?1:+%{1}}\ +Provides: kernel-drm-nouveau = 16\ +Provides: kernel-uname-r = %{KVERREL}%{?1:+%{1}}\ +Requires(pre): %{kernel_prereq}\ +Requires(pre): %{initrd_prereq}\ +Requires(pre): linux-firmware >= 20130724-29.git31f6b30\ +Requires(preun): systemd >= 200\ +%{expand:%%{?kernel%{?1:_%{1}}_conflicts:Conflicts: %%{kernel%{?1:_%{1}}_conflicts}}}\ +%{expand:%%{?kernel%{?1:_%{1}}_obsoletes:Obsoletes: %%{kernel%{?1:_%{1}}_obsoletes}}}\ +%{expand:%%{?kernel%{?1:_%{1}}_provides:Provides: %%{kernel%{?1:_%{1}}_provides}}}\ +# We can't let RPM do the dependencies automatic because it'll then pick up\ +# a correct but undesirable perl dependency from the module headers which\ +# isn't required for the kernel proper to function\ +AutoReqProv: no\ +%{nil}
%package headers Summary: Header files for the Linux kernel for use by glibc @@ -837,53 +845,65 @@ Provides: kernel%{?1:-%{1}}-modules-extra = %{version}-%{release}%{?1:+%{1}}\ Provides: installonlypkg(kernel-module)\ Provides: kernel%{?1:-%{1}}-modules-extra-uname-r = %{KVERREL}%{?1:+%{1}}\ Requires: kernel-uname-r = %{KVERREL}%{?1:+%{1}}\ +Requires: kernel-drivers-uname-r = %{KVERREL}%{?1:+%{1}}\ AutoReqProv: no\ %description -n kernel%{?variant}%{?1:-%{1}}-modules-extra\ This package provides less commonly used kernel modules for the %{?2:%{2} }kernel package.\ %{nil}
# +# This macro creates a kernel-<subpackage>-drivers package. +# %%kernel_drivers_package <subpackage> <pretty-name> +# +%define kernel_drivers_package() \ +%package %{?1:%{1}-}drivers\ +Summary: kernel modules to match the %{?2:%{2}-}core kernel\ +Group: System Environment/Kernel\ +Provides: kernel%{?1:-%{1}}-drivers-%{_target_cpu} = %{version}-%{release}\ +Provides: kernel-drivers-%{_target_cpu} = %{version}-%{release}%{?1:+%{1}}\ +Provides: kernel-drivers = %{version}-%{release}%{?1:+%{1}}\ +Provides: installonlypkg(kernel-module)\ +Provides: kernel-drivers-uname-r = %{KVERREL}%{?1:+%{1}}\ +Requires: kernel-uname-r = %{KVERREL}%{?1:+%{1}}\ +AutoReqProv: no\ +%description -n kernel%{?variant}%{?1:-%{1}}-drivers\ +This package provides commonly used kernel modules for the %{?2:%{2}-}core kernel package.\ +%{nil} + +# # This macro creates a kernel-<subpackage> and its -devel and -debuginfo too. # %%define variant_summary The Linux kernel compiled for <configuration> # %%kernel_variant_package [-n <pretty-name>] <subpackage> # %define kernel_variant_package(n:) \ -%package %1\ +%package %{?1:%{1}-}core\ Summary: %{variant_summary}\ Group: System Environment/Kernel\ -%kernel_reqprovconf\ -%{expand:%%kernel_devel_package %1 %{!?-n:%1}%{?-n:%{-n*}}}\ +Provides: kernel-%{?1:%{1}-}core-uname-r = %{KVERREL}%{?1:+%{1}}\ +%{expand:%%kernel_reqprovconf}\ +%{expand:%%kernel_devel_package %{?1:%{1}} %{!?{-n}:%{1}}%{?{-n}:%{-n*}}}\ +%{expand:%%kernel_drivers_package %{?1:%{1}} %{!?{-n}:%{1}}%{?{-n}:%{-n*}}}\ %if %{with_extra}\ -%{expand:%%kernel_modules_extra_package %1 %{!?-n:%1}%{?-n:%{-n*}}}\ +%{expand:%%kernel_modules_extra_package %{?1:%{1}} %{!?{-n}:%{1}}%{?{-n}:%{-n*}}}\ %endif\ -%{expand:%%kernel_debuginfo_package %1}\ +%{expand:%%kernel_debuginfo_package %{?1:%{1}}}\ %{nil}
- -# First the auxiliary packages of the main kernel package. -%kernel_devel_package -%if %{with_extra} -%kernel_modules_extra_package -%endif -%kernel_debuginfo_package - - # Now, each variant package.
%define variant_summary The Linux kernel compiled for SMP machines %kernel_variant_package -n SMP smp -%description smp +%description smp-core This package includes a SMP version of the Linux kernel. It is required only on machines with two or more CPUs as well as machines with hyperthreading technology.
Install the kernel-smp package if your machine uses two or more CPUs.
- %ifnarch armv7hl %define variant_summary The Linux kernel compiled for PAE capable machines %kernel_variant_package %{pae} -%description %{pae} +%description %{pae}-core This package includes a version of the Linux kernel with support for up to 64GB of high memory. It requires a CPU with Physical Address Extensions (PAE). The non-PAE kernel can only address up to 4GB of memory. @@ -891,7 +911,7 @@ Install the kernel-PAE package if your machine has more than 4GB of memory. %else %define variant_summary The Linux kernel compiled for Cortex-A15 %kernel_variant_package %{pae} -%description %{pae} +%description %{pae}-core This package includes a version of the Linux kernel with support for Cortex-A15 devices with LPAE and HW virtualisation support %endif @@ -900,7 +920,7 @@ Cortex-A15 devices with LPAE and HW virtualisation support %define variant_summary The Linux kernel compiled with extra debugging enabled for PAE capable machines %kernel_variant_package %{pae}debug Obsoletes: kernel-PAE-debug -%description %{pae}debug +%description %{pae}debug-core This package includes a version of the Linux kernel with support for up to 64GB of high memory. It requires a CPU with Physical Address Extensions (PAE). The non-PAE kernel can only address up to 4GB of memory. @@ -913,7 +933,7 @@ on kernel bugs, as some of these options impact performance noticably.
%define variant_summary The Linux kernel compiled with extra debugging enabled %kernel_variant_package debug -%description debug +%description debug-core The kernel package contains the Linux kernel (vmlinuz), the core of any Linux operating system. The kernel handles the basic functions of the operating system: memory allocation, process allocation, device @@ -923,6 +943,16 @@ This variant of the kernel has numerous debugging options enabled. It should only be installed when trying to gather additional information on kernel bugs, as some of these options impact performance noticably.
+# And finally the main -core package + +%define variant_summary The Linux kernel +%kernel_variant_package +%description core +The kernel package contains the Linux kernel (vmlinuz), the core of any +Linux operating system. The kernel handles the basic functions +of the operating system: memory allocation, process allocation, device +input and output, etc. +
%prep # do a few sanity-checks for --with *only builds @@ -1595,6 +1625,55 @@ BuildKernel() { %{SOURCE17} $RPM_BUILD_ROOT/lib/modules/$KernelVer %{SOURCE16} %endif
+ # + # Generate the kernel-core and kernel-drivers files lists + # + + # Copy the System.map file for depmod to use, and create a backup of the + # full module tree so we can restore it after we're done filtering + cp System.map $RPM_BUILD_ROOT/. + pushd $RPM_BUILD_ROOT + mkdir restore + cp -r lib/modules/$KernelVer/* restore/. + + # don't include anything going into k-m-e in the file lists + rm -rf lib/modules/$KernelVer/extra + + # Find all the module files and filter them out into the core and drivers + # lists. This actually removes anything going into -drivers from the dir. + find lib/modules/$KernelVer/kernel -name *.ko | sort -n > modules.list + cp $RPM_SOURCE_DIR/filter-*.sh . + %{SOURCE99} modules.list %{_target_cpu} + rm filter-*.sh + + # Run depmod on the resulting module tree and make sure it isn't broken + depmod -b . -aeF ./System.map $KernelVer + # remove files that will be auto generated by depmod at rpm -i time + pushd $RPM_BUILD_ROOT/lib/modules/$KernelVer/ + rm -f modules.{alias*,builtin.bin,dep*,*map,symbols*,devname,softdep} + popd + + # Go back and find all of the various directories in the tree. We use this + # for the dir lists in kernel-core + find lib/modules/$KernelVer/kernel -type d | sort -n > module-dirs.list + + # Cleanup + rm System.map + cp -r restore/* lib/modules/$KernelVer/. + rm -rf restore + popd + + # Make sure the files lists start with absolute paths or rpmbuild fails. + # Also add in the dir entries + sed -e 's/^lib*//lib/' $RPM_BUILD_ROOT/k-d.list > ../kernel${Flavour:+-${Flavour}}-drivers.list + sed -e 's/^lib*/%dir /lib/' $RPM_BUILD_ROOT/module-dirs.list > ../kernel${Flavour:+-${Flavour}}-core.list + sed -e 's/^lib*//lib/' $RPM_BUILD_ROOT/modules.list >> ../kernel${Flavour:+-${Flavour}}-core.list + + # Cleanup + rm -f $RPM_BUILD_ROOT/k-d.list + rm -f $RPM_BUILD_ROOT/modules.list + rm -f $RPM_BUILD_ROOT/module-dirs.list + %if %{signmodules} # Save the signing keys so we can sign the modules in __modsign_install_post cp signing_key.priv signing_key.priv.sign${Flav} @@ -1863,11 +1942,28 @@ fi\
# # This macro defines a %%post script for a kernel*-modules-extra package. +# It also defines a %%postun script that does the same thing. # %%kernel_modules_extra_post [<subpackage>] # %define kernel_modules_extra_post() \ %{expand:%%post %{?1:%{1}-}modules-extra}\ /sbin/depmod -a %{KVERREL}%{?1:+%{1}}\ +%{nil}\ +%{expand:%%postun %{?1:%{1}-}modules-extra}\ +/sbin/depmod -a %{KVERREL}%{?1:+%{1}}\ +%{nil} + +# +# This macro defines a %%post script for a kernel*-drivers package.%dir /lib/modules/%{KVERREL}%{?2:+%{2}}\ -/lib/modules/%{KVERREL}%{?2:+%{2}}/kernel\ +%dir /lib/modules/%{KVERREL}%{?2:+%{2}}/kernel\ /lib/modules/%{KVERREL}%{?2:+%{2}}/build\ /lib/modules/%{KVERREL}%{?2:+%{2}}/source\ /lib/modules/%{KVERREL}%{?2:+%{2}}/updates\ @@ -2035,7 +2137,8 @@ fi /etc/ld.so.conf.d/kernel-%{KVERREL}%{?2:+%{2}}.conf\ %endif\ /lib/modules/%{KVERREL}%{?2:+%{2}}/modules.*\ -%ghost /boot/initramfs-%{KVERREL}%{?2:+%{2}}.img\ +%{expand:%%files -f kernel-%{?2:%{2}-}drivers.list %{?2:%{2}-}drivers}\ +%defattr(-,root,root)\ %{expand:%%files %{?2:%{2}-}devel}\ %defattr(-,root,root)\ /usr/src/kernels/%{KVERREL}%{?2:+%{2}}\
On Tue, 2014-04-01 at 12:50 -0400, Josh Boyer wrote:
--- /dev/null +++ b/filter-i686.sh @@ -0,0 +1,5 @@ +#! /bin/bash
+driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mfd mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb"
+singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject hid-sensor-hub hid-sensor-magn-3d hid-sensor-incl-3d hid-sensor-als.ko
Just hid-sensor-als here?
hid-sensor-gyro-3d hid-sensor-iio-common hid-sensor-accel-3d hid-sensor-trigger hid-sensor-als"
There's hid-sensor-als again. So just drop hid-sensor-als.ko?
I don't know how to review this. But I did spot this typo.
Paul Bolle
On Tue, Apr 1, 2014 at 1:37 PM, Paul Bolle pebolle@tiscali.nl wrote:
On Tue, 2014-04-01 at 12:50 -0400, Josh Boyer wrote:
--- /dev/null +++ b/filter-i686.sh @@ -0,0 +1,5 @@ +#! /bin/bash
+driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mfd mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb"
+singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject hid-sensor-hub hid-sensor-magn-3d hid-sensor-incl-3d hid-sensor-als.ko
Just hid-sensor-als here?
hid-sensor-gyro-3d hid-sensor-iio-common hid-sensor-accel-3d hid-sensor-trigger hid-sensor-als"
There's hid-sensor-als again. So just drop hid-sensor-als.ko?
I don't know how to review this. But I did spot this typo.
Yep, I'll fix that. Thanks for spotting it.
As for "how to review", basically the *dirs variables are lists of entire directories to move. singlemods are single modules to move. There's not a ton of review other than looking at the resulting builds and making sure the depmod call succeeds without error and fixing the lists if it doesn't.
Which reminds me that I need to flip this to fail the build if that depmod call fails after people sign off on things.
josh
On Tue, 2014-04-01 at 12:50 -0400, Josh Boyer wrote:
--- /dev/null +++ b/filter-aarch64.sh @@ -0,0 +1,5 @@ +#! /bin/bash
+driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb"
+singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject "
Another attempt at reviewing: why is this not sorted alphabetically?
And wouldn't this be easier to maintain if this was split into two plain text files (per arch) with one name per line?
Paul Bolle
On Tue, Apr 1, 2014 at 1:45 PM, Paul Bolle pebolle@tiscali.nl wrote:
On Tue, 2014-04-01 at 12:50 -0400, Josh Boyer wrote:
--- /dev/null +++ b/filter-aarch64.sh @@ -0,0 +1,5 @@ +#! /bin/bash
+driverdirs="atm auxdisplay bcma bluetooth fmc infiniband isdn leds media memstick message mmc mtd nfc ntb pcmcia platform power ssb staging uio uwb"
+singlemods="ntb_netdev iscsi_ibft iscsi_boot_sysfs iscsi_tcp megaraid pmcraid qla1280 9pnet_rdma svcrdma xprtrdma hid-picolcd hid-prodikeys hwa-hc hwpoison-inject "
Another attempt at reviewing: why is this not sorted alphabetically?
Because I built the list as things depsolved or didn't. There's no main reason it couldn't be alphabetized. I don't immediately see any benefits of doing so, but it's easy enough to change.
And wouldn't this be easier to maintain if this was split into two plain text files (per arch) with one name per line?
I don't think so. Why 2 files per arch? The filter-<arch> files are overrides for the settings in filter-modules.sh. Unless there's a need to override, they are fairly small already.
One name per line could be done I guess but I don't see it being more or less maintainable.
josh
On Tue, Apr 01, 2014 at 12:50:33PM -0400, Josh Boyer wrote:
This creates kernel-core and kernel-drivers subpackages. The kernel package remains as a meta-package the requires both of the subpackages. This allows most installs to continue on as-is with upgrades working.
The contents of the kernel-core and kernel-drivers subpackages are determined at build time through the filter-modules.sh script. This script "removes" pre-defined subsystems and modules and generates a filelist for the %files section of each subpackage. The contents of each are per-arch, with arch override files taken into account. This allows us to make the split useful for varying arches.
I think there needs to be a little bit more documentation surrounding this. If one simply cracks open a filter-$arch.sh file, they really don't get any clue how these two variables are used, where things listed end up, etc. A blurb in each of them explaining "files added to these lists are excluded from kernel-core and stashed in kernel-drivers" (assuming I even got that much right from a quick read) would be useful. Somewhere down the road, I'm sure someone other than yourself will need to make a change to one of those lists. I'm pretty familiar with the kernel spec myself, and without some trial and error, poking at builds, I can't say for certain I actually understand exactly what ends up where.
Now, as for reusability... No clue if this is something that would be desired down the road in Red Hat Enterprise Linux (RHEL). Making this a toggle-able (e.g. --with split) option would be potentially nifty, but with all the macro crud, a hideous mess to implement. I can see this being potentially useful in the RHEL case too though -- people installing virt appliances, barebones servers, etc., might like the disk space savings not having tons of useless drivers installed too. I think we can make the case that its a good idea to split things up this way, and worst-case, its not horrendously hard to gut things back to the way they are now.
Some questions:
1) Is there a default destination for any "new" driver that shows up in the tree? I assume that will partly depend on where it ends up in the file system hierarchy, but maybe it is something that needs to be monitored to prevent sudden and unexpected bloat to -core (i.e., new subsystem gets enabled by way of some kconfig option, all winds up in -core, increasing its size significantly).
2) What criteria are being used to determine what goes into core? The Fedora feature page says core is "a minimum(-ish) set of drivers to only just be able to run in virtualized environments", which I hadn't read yet when I suggested maybe RHEL barebones servers would use this. I think I'd like to see a bit larger core than just-enough-for-virt myself. To me, what anaconda needs to install a system and get it on the network to receive updates maybe better describes what I'd consider core than just-enough-for-virt does. No clue how bit the delta is there though.
HTH,
On Mon, Apr 14, 2014 at 5:11 PM, Jarod Wilson jarod@redhat.com wrote:
On Tue, Apr 01, 2014 at 12:50:33PM -0400, Josh Boyer wrote:
This creates kernel-core and kernel-drivers subpackages. The kernel package remains as a meta-package the requires both of the subpackages. This allows most installs to continue on as-is with upgrades working.
The contents of the kernel-core and kernel-drivers subpackages are determined at build time through the filter-modules.sh script. This script "removes" pre-defined subsystems and modules and generates a filelist for the %files section of each subpackage. The contents of each are per-arch, with arch override files taken into account. This allows us to make the split useful for varying arches.
I think there needs to be a little bit more documentation surrounding this. If one simply cracks open a filter-$arch.sh file, they really don't get any clue how these two variables are used, where things listed end up, etc. A blurb in each of them explaining "files added to these lists are excluded from kernel-core and stashed in kernel-drivers" (assuming I even got that much right from a quick read) would be useful. Somewhere down the road, I'm sure someone other than yourself will need to make a change to one of those lists. I'm pretty familiar with the kernel spec myself, and without some trial and error, poking at builds, I can't say for certain I actually understand exactly what ends up where.
OK, totally fair. FWIW, you did get it correct for the most part.
Now, as for reusability... No clue if this is something that would be desired down the road in Red Hat Enterprise Linux (RHEL). Making this a toggle-able (e.g. --with split) option would be potentially nifty, but with all the macro crud, a hideous mess to implement. I can see this being
I considered wrapping it with something like %if with_split, but that does indeed get nasty. Also, within the context of Fedora, it would basically be an option that was either always on or always off. We couldn't really make it arch tunable because of potential changes to anaconda to expect kernel-core to exist, etc. Trying to deliver two different "kernel" packages seems pretty difficult in that regard.
potentially useful in the RHEL case too though -- people installing virt appliances, barebones servers, etc., might like the disk space savings not having tons of useless drivers installed too. I think we can make the case that its a good idea to split things up this way, and worst-case, its not horrendously hard to gut things back to the way they are now.
I was hoping it would be useful at some point, yeah. Whether that happens remains to be seen, but at least I'll try keeping things in mind as we go. If there are changes that would help going forward, I hope people bring those up as soon as possible when they think of them.
Some questions:
- Is there a default destination for any "new" driver that shows up in
the tree? I assume that will partly depend on where it ends up in the file system hierarchy, but maybe it is something that needs to be monitored to prevent sudden and unexpected bloat to -core (i.e., new subsystem gets enabled by way of some kconfig option, all winds up in -core, increasing its size significantly).
New drivers are mostly at the mercy of where they are in the subsystem directories. Any new driver that gets lumped into a subsys that's moved to -drviers will wind up there. For subsystems that aren't already lumped into -drivers, new drivers would wind up in -core. There's a depmod check to make sure we don't have -core wind up with unmet module dependencies, and that will be fatal during build (I still need to make that change).
I agree we need to watch the size issue, and we might be able to put a check on the module tree after we move stuff around. However, there's no strict limit on what that size should be. What I have here is a first approximation of a trimmed down size and it seems to be suitable to start with. Don was trying to advocate for a "quota" and I still think that's worthwhile in the longer run.
- What criteria are being used to determine what goes into core? The
Fedora feature page says core is "a minimum(-ish) set of drivers to only just be able to run in virtualized environments", which I hadn't read yet when I suggested maybe RHEL barebones servers would use this. I think I'd like to see a bit larger core than just-enough-for-virt myself. To me, what anaconda needs to install a system and get it on the network to receive updates maybe better describes what I'd consider core than just-enough-for-virt does. No clue how bit the delta is there though.
It's a bit more than just-virt. E.g. we have the radeon and intel video drivers in there, all the major rootfs filesystems (btrfs, xfs, ext4), and a variety of other drivers included. Using just kernel-core does boot a KVM guest without any missing functionality (unless you're doing passthru perhaps), so that is the main testcase but that doesn't mean it has to be the only thing that works. It likely isn't enough for a barebones server at the moment, but it isn't far off. Others have asked if kernel-core would be suitable for "embedded" devices, and the answer there is maybe. I wouldn't expect it to be suitable for something like a laptop, which would require all the various sound drivers and other niceties to make the hardware work.
josh
On Tue, Apr 1, 2014 at 12:50 PM, Josh Boyer jwboyer@fedoraproject.org wrote:
This creates kernel-core and kernel-drivers subpackages. The kernel package remains as a meta-package the requires both of the subpackages. This allows most installs to continue on as-is with upgrades working.
I've had this out for review for 3 weeks now. Aside from a need for further documentation, I haven't seen any major objections to doing this. The F21 Alpha change deadline is in July, but I would like to land this as soon as possible. With 3.15-rc2 out, it seems like a decent time to try and get this into rawhide.
Dave and Justin have both found no major faults with it. Peter has asked for some content changes for ARM, and those should be easy to make. I plan on documenting things more before pushing it out. I fully expect things will be found and fixed and improved over time as well.
With that in mind, I'd like to aim for getting this into rawhide by next Monday, April 28. If anyone has any showstopper things or major reservations, please speak up soon.
josh
On 04/01/2014 04:48 PM, Josh Boyer wrote:
I'd really appreciate it if people could look the patch over and provide further feedback and/or Acks.
Assuming you did not look into these things based on the affected scope on the feature page but you are ensuring no breakage in Dracut or Default enable services that expect modules to be available etc?
JBG
On Wed, Apr 02, 2014 at 11:56:16AM +0000, "Jóhann B. Guðmundsson" wrote:
On 04/01/2014 04:48 PM, Josh Boyer wrote:
I'd really appreciate it if people could look the patch over and provide further feedback and/or Acks.
Assuming you did not look into these things based on the affected scope on the feature page but you are ensuring no breakage in Dracut or Default enable services that expect modules to be available etc?
Given that a normal install will still be looking for kernel, and kernel is a metapackage that requires kernel-core and kernel-drivers, the installed set of content is identical to today. So Server, Workstation, existing installs should be fine.
For Cloud, that's up to them to sort out given that they wanted the smaller kernel.
josh
On 04/02/2014 12:12 PM, Josh Boyer wrote:
On Wed, Apr 02, 2014 at 11:56:16AM +0000, "Jóhann B. Guðmundsson" wrote:
On 04/01/2014 04:48 PM, Josh Boyer wrote:
I'd really appreciate it if people could look the patch over and provide further feedback and/or Acks.
Assuming you did not look into these things based on the affected scope on the feature page but you are ensuring no breakage in Dracut or Default enable services that expect modules to be available etc?
Given that a normal install will still be looking for kernel, and kernel is a metapackage that requires kernel-core and kernel-drivers, the installed set of content is identical to today. So Server, Workstation, existing installs should be fine.
The magic word should ;)
For Cloud, that's up to them to sort out given that they wanted the smaller kernel.
Is this split good enough to used by embedded or can we expect another split ( or shaving of core or drivers ) if there emerges an embedded WG?
JBG
On Wed, Apr 02, 2014 at 12:24:10PM +0000, "Jóhann B. Guðmundsson" wrote:
On 04/02/2014 12:12 PM, Josh Boyer wrote:
On Wed, Apr 02, 2014 at 11:56:16AM +0000, "Jóhann B. Guðmundsson" wrote:
On 04/01/2014 04:48 PM, Josh Boyer wrote:
I'd really appreciate it if people could look the patch over and provide further feedback and/or Acks.
Assuming you did not look into these things based on the affected scope on the feature page but you are ensuring no breakage in Dracut or Default enable services that expect modules to be available etc?
Given that a normal install will still be looking for kernel, and kernel is a metapackage that requires kernel-core and kernel-drivers, the installed set of content is identical to today. So Server, Workstation, existing installs should be fine.
The magic word should ;)
I tested it in a VM several times, going back and forth between existing kernels and kernels from the COPR. It worked as expected. I also tested just installing kernel-core and the VM booted as expected. I would not expect e.g. my laptop to boot with just kernel-core.
The Cloud WG really needs to drive as much of the testing as they can here. It's their Change request.
For Cloud, that's up to them to sort out given that they wanted the smaller kernel.
Is this split good enough to used by embedded or can we expect another split ( or shaving of core or drivers ) if there emerges an embedded WG?
Further tuning would be needed, perhaps with the exception of ARM. Really though, tranditional embedded usecases build per-board kernels so a distro kernel isn't really well suited to that at all. It's difficult to answer definitively without knowing further what embedded means.
josh
On 04/02/2014 12:44 PM, Josh Boyer wrote:
Further tuning would be needed, perhaps with the exception of ARM. Really though, tranditional embedded usecases build per-board kernels so a distro kernel isn't really well suited to that at all. It's difficult to answer definitively without knowing further what embedded means.
I noticed on a job description posted on devel that there seems to be an effort on the way pushing MIPS to become a primary arch in Fedora [1].
"The task is to take MIPS through to being a primary architecture in the Fedora/RHEL space, enabling further enterprise scale deployment of the MIPS architecture."
Afaik MIPS arch is mostly used on embedded systems so either a new line of servers are emerging or the intent is to use tablet/mobile devices to act as new form factor for "next generation of server technologies." so at least that architecture should be taken into consideration when making this split so we can limit any disruption/fallout that might come out of this as best we can to a single cycle.
JBG
1. http://www.imgtec.com/corporate/vacancy_detail.asp?VacancyID=2286
On Wed, Apr 02, 2014 at 01:06:45PM +0000, "Jóhann B. Guðmundsson" wrote:
On 04/02/2014 12:44 PM, Josh Boyer wrote:
Further tuning would be needed, perhaps with the exception of ARM. Really though, tranditional embedded usecases build per-board kernels so a distro kernel isn't really well suited to that at all. It's difficult to answer definitively without knowing further what embedded means.
I noticed on a job description posted on devel that there seems to be an effort on the way pushing MIPS to become a primary arch in Fedora [1].
"The task is to take MIPS through to being a primary architecture in the Fedora/RHEL space, enabling further enterprise scale deployment of the MIPS architecture."
Neat.
Afaik MIPS arch is mostly used on embedded systems so either a new line of servers are emerging or the intent is to use tablet/mobile devices to act as new form factor for "next generation of server technologies." so at least that architecture should be taken into consideration when making this split so we can limit any disruption/fallout that might come out of this as best we can to a single cycle.
MIPS would need to go through secondary arch status, just like all the rest of the architectures. We're not in a position to suddenly start worrying about kernels for an entire new architecture before anyone actually shows up to do work on that architecture. The job posting is encouraging, but it isn't a promise or actual work being done yet.
While I appreciate planning ahead, this is firmly in "we'll cross that bridge IF we come to it" territory.
josh
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 02 Apr 2014 13:06:45 +0000 "Jóhann B. Guðmundsson" johannbg@gmail.com wrote:
On 04/02/2014 12:44 PM, Josh Boyer wrote:
Further tuning would be needed, perhaps with the exception of ARM. Really though, tranditional embedded usecases build per-board kernels so a distro kernel isn't really well suited to that at all. It's difficult to answer definitively without knowing further what embedded means.
I noticed on a job description posted on devel that there seems to be an effort on the way pushing MIPS to become a primary arch in Fedora [1].
"The task is to take MIPS through to being a primary architecture in the Fedora/RHEL space, enabling further enterprise scale deployment of the MIPS architecture."
Afaik MIPS arch is mostly used on embedded systems so either a new line of servers are emerging or the intent is to use tablet/mobile devices to act as new form factor for "next generation of server technologies." so at least that architecture should be taken into consideration when making this split so we can limit any disruption/fallout that might come out of this as best we can to a single cycle.
I think what you will see here is the same as the arm push to be more general purpose computing machines. indeed the job poster even said they have a quad core 1ghz server with sata and 8gb ram for use with fedora development. but it is a bridge to cross down the road when more concrete things are in place.
Dennis
kernel@lists.fedoraproject.org
-
Dennis Gilmore -
Jarod Wilson -
Josh Boyer -
Jóhann B. Guðmundsson -
Paul Bolle