Hi guys,
sorry, I was mostly away the last two days so you had to fight with kernel-modules alone. I'll try to catch up and reply where I think it's needed.
Anyway, I created a example package with kernel-modules for ndiswrapper to play a bit with the discussed scheme. It
I maintain ndiswrapper (and some other kernel-module-packages) already for livna. I just chooesed it for this example cause it's small, builds fast and needs a userspace-program and a kernel-module.
I followed the example in the wiki and some posts on this list. I split the package in a userland package that creates a %{name}-kmsrc.rpm that is used by the kernel-module-package as source. Something like this would be neat package like nvidia als ati drivers where a single srpm is quite big if you want to rebuild only the kernel-module (yes, it seems a lot of users of livna users do that).
Find the specs and SRPMS at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/
Comments?
On Thu, 2005-06-30 at 15:52 +0200, Thorsten Leemhuis wrote:
Hi guys,
sorry, I was mostly away the last two days so you had to fight with kernel-modules alone. I'll try to catch up and reply where I think it's needed.
Anyway, I created a example package with kernel-modules for ndiswrapper to play a bit with the discussed scheme. It
I maintain ndiswrapper (and some other kernel-module-packages) already for livna. I just chooesed it for this example cause it's small, builds fast and needs a userspace-program and a kernel-module.
I followed the example in the wiki and some posts on this list. I split the package in a userland package that creates a %{name}-kmsrc.rpm that is used by the kernel-module-package as source. Something like this would be neat package like nvidia als ati drivers where a single srpm is quite big if you want to rebuild only the kernel-module (yes, it seems a lot of users of livna users do that).
I think the source should be a Source0 in kernel-module-ndiswrapper. I'm not very comfortable with confusing people with src.rpms that have no source in them. Yes, I know this means duplication of source code in two SRPMS, but disk is cheap. Nosrc.rpms are part of the slipperly slope to SuSE town. :/
Otherwise, I think you're right on target. (Ignoring the fact that I think ndiswrapper should probably be ExclusiveArch: i586 i686)
~spot
On Thu, 2005-06-30 at 09:07 -0500, Tom 'spot' Callaway wrote:
(Ignoring the fact that I think ndiswrapper should probably be ExclusiveArch: i586 i686)
Regarding ExclusiveArch in general (for module packages that can be built on whatever platform), what is it needed for in the proposal? http://fedoraproject.org/wiki/Extras/KernelModuleProposal
Does the build system decide that it _will_ (as opposed to won't) build something based on ExclusiveArch? That would sound backwards to me.
Am Donnerstag, den 30.06.2005, 09:07 -0500 schrieb Tom 'spot' Callaway:
On Thu, 2005-06-30 at 15:52 +0200, Thorsten Leemhuis wrote:
Hi guys,
sorry, I was mostly away the last two days so you had to fight with kernel-modules alone. I'll try to catch up and reply where I think it's needed.
Anyway, I created a example package with kernel-modules for ndiswrapper to play a bit with the discussed scheme. It
I maintain ndiswrapper (and some other kernel-module-packages) already for livna. I just chooesed it for this example cause it's small, builds fast and needs a userspace-program and a kernel-module.
I followed the example in the wiki and some posts on this list. I split the package in a userland package that creates a %{name}-kmsrc.rpm that is used by the kernel-module-package as source. Something like this would be neat package like nvidia als ati drivers where a single srpm is quite big if you want to rebuild only the kernel-module (yes, it seems a lot of users of livna users do that).
I think the source should be a Source0 in kernel-module-ndiswrapper.
No problem with that. Was just a idea to save space.
Otherwise, I think you're right on target. (Ignoring the fact that I think ndiswrapper should probably be ExclusiveArch: i586 i686)
x86_64 also, but yes, ppc was wrong here -- i just included it here cause it's an example ;)
On Thu, 2005-06-30 at 15:52 +0200, Thorsten Leemhuis wrote:
I split the package in a userland package that creates a %{name}-kmsrc.rpm that is used by the kernel-module-package as source. Something like this would be neat package like nvidia als ati drivers where a single srpm is quite big if you want to rebuild only the kernel-module
I don't see the point of the kmsrc package.
If you want to optimize SRPM download sizes, why not just supply a separate trimmed-down tarball containing only the kernel module bits in the kernel module SRPM (and include comments in the specfile how it was created from the upstream dist tarball, or include a script to do it in the source rpm)? And if you want, another similar trimmed one in the userland SRPM containing only the non-kernel-module parts.
Unless I'm missing something, using your approach, the user who wants to locally build only the kernel modules needs to have not only the SRPM that produces the modules, but also one extra binary package (which actually contains the sources).
Further, the sources and the build routines for the kernel modules are now split into two packages, which means also extra work and care from the maintainer.
Isn't this just more work for both maintainers and end users? Did I miss something?
Am Donnerstag, den 30.06.2005, 17:11 +0300 schrieb Ville Skyttä:
On Thu, 2005-06-30 at 15:52 +0200, Thorsten Leemhuis wrote:
I split the package in a userland package that creates a %{name}-kmsrc.rpm that is used by the kernel-module-package as source. Something like this would be neat package like nvidia als ati drivers where a single srpm is quite big if you want to rebuild only the kernel-module
I don't see the point of the kmsrc package.
As I said in the mail to spot -- I removed it.
If you want to optimize SRPM download sizes, why not just supply a separate trimmed-down tarball containing only the kernel module bits in the kernel module SRPM (and include comments in the specfile how it was created from the upstream dist tarball, or include a script to do it in the source rpm)?
Yes, that is my plan for the ati-fglrx package because the srpm is now 22M, but the src for the kernel-module is a lot smaller.
And if you want, another similar trimmed one in the userland SRPM containing only the non-kernel-module parts.
I don't see a reason for this.
Further, the sources and the build routines for the kernel modules are now split into two packages, which means also extra work and care from the maintainer.
In the case of ati-fglrx or nvidia-glx I think it's worth the trouble. For ndiswrapper it's probably not.
BTW, I updated the packages/specs at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ and also the proposal at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal I added it as real world example/second proposal with split packages for userland and kernel-module.
On Thu, 2005-06-30 at 17:11 +0300, Ville Skyttä wrote:
On Thu, 2005-06-30 at 15:52 +0200, Thorsten Leemhuis wrote:
I split the package in a userland package that creates a %{name}-kmsrc.rpm that is used by the kernel-module-package as source. Something like this would be neat package like nvidia als ati drivers where a single srpm is quite big if you want to rebuild only the kernel-module
I don't see the point of the kmsrc package.
ACK.
If you want to optimize SRPM download sizes, why not just supply a separate trimmed-down tarball containing only the kernel module bits in the kernel module SRPM (and include comments in the specfile how it was created from the upstream dist tarball, or include a script to do it in the source rpm)? And if you want, another similar trimmed one in the userland SRPM containing only the non-kernel-module parts.
Actually, I fail to see where this approach _really_ reduces bandwidth requirements.
Users who are rebuilding the rpms, should learn to download the srpm once. Afterwards, they'll have it "on disk" and don't have to re-download it again.
I would expect the high rate of downloads of kernel-module-*src.rpms you might be seeing at Livna to originate from delays between kernel releases in FC and kernel-module releases at Livna.
Further, the sources and the build routines for the kernel modules are now split into two packages, which means also extra work and care from the maintainer.
Agreed. Also, it might not always be possible or not possible with further effort, sometimes for technical reasons (Makefiles, build infrastructure etc.), sometimes for licensing reasons ("Unmodified sources").
Isn't this just more work for both maintainers and end users?
Not for end users. They'd only see a kernel-module*.src.rpm and will not know about the fact it had been split out elsewhere. But .. no matter if the source are split into userland/kernel srpms they will have to rebuild _one_ srpm.
Ralf
Am Donnerstag, den 30.06.2005, 18:47 +0200 schrieb Ralf Corsepius:
On Thu, 2005-06-30 at 17:11 +0300, Ville Skyttä wrote:
On Thu, 2005-06-30 at 15:52 +0200, Thorsten Leemhuis wrote:
If you want to optimize SRPM download sizes, why not just supply a separate trimmed-down tarball containing only the kernel module bits in the kernel module SRPM (and include comments in the specfile how it was created from the upstream dist tarball, or include a script to do it in the source rpm)? And if you want, another similar trimmed one in the userland SRPM containing only the non-kernel-module parts.
Actually, I fail to see where this approach _really_ reduces bandwidth requirements.
Sourcecode for the kernel-module parts livna ati-fglrx package: around 4,5 MByte for x86 and x86_64 together. Whole SRPM (includes x86_64 and x86): 22 MByte.
Users who are rebuilding the rpms, should learn to download the srpm once.
Every two month when ati releases a new driver.
Afterwards, they'll have it "on disk" and don't have to re-download it again.
Sure.
I would expect the high rate of downloads of kernel-module-*src.rpms you might be seeing at Livna to originate from delays between kernel releases in FC and kernel-module releases at Livna.
People use those to build against rawhide, kernels from updates-testing and even self-compiled kernels (not possible with the scheme we currently discuss here for extras). If I broke something there I got bug-reports shortly after release...
[...]
CU thl
Hi *
Am Donnerstag, den 30.06.2005, 15:52 +0200 schrieb Thorsten Leemhuis:
Anyway, I created a example package with kernel-modules for ndiswrapper to play a bit with the discussed scheme. It
I updated the example found online at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ and in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
Changes to the kernel-module-package:
- These macro definitions are now found in the top of the spec file: %{!?kver: %define kver %(uname -r)} %define ksrc %{_usrsrc}/kernels/%{kver}-%{_target_cpu} %define moddir /lib/modules/%{kver}/kernel/misc %define mainpkgname ndiswrapper %define mainpkgversion 1.2 %define mainpgkrelease 1 Yes, in other packages I would not like these "mainpkg*" definitions on the top, but in this type of package I think they are helpful.
- The main kernel-module package is now named "kernel-module-%{mainpkgname}-base" (if someones knows something better then "base" tell me). The kernel-module itself is now placed in %package -n kernel-module-%{mainpkgname} So only one SRPM is created no matter how much different kernel-modules are build. Also the CVS tags don't depend on the local kernel-version (I did not try but it got to this idea when I saw "Sacrifice no-args-builds-for-current-kernel for benefit of CVS tag sanity." https://www.redhat.com/archives/fedora-extras-commits/2005-July/msg00229.htm... )
- Now: Release: %{mainpgkrelease}.%(echo %{kver} | tr - _) Provides: kernel-module = %{kver} as suggested by spot on this list yesterday
- Avoid some problems when kernel-module and kernel are deleted in one rpm transaction (anyone knows a better way to fix? "Requires(postun): kernel-%{_target_cpu} = %{kver}" does not work): -- Only depmod when System.map is still there %postun -n kernel-module-%{mainpkgname} [ -e "/boot/System.map-%{kver}" ] && \ /sbin/depmod -e -F /boot/System.map-%{kver} %{kver}> /dev/null || : -- In %files: %dir /lib/modules/%{kver}/ so that dir is removed during remove.
CU thl
P.S: I'm on a trip the next weeks -- I don't know how often and how long I'll have access to the Internet. So don't expect a fast reply from me during that time. And if you want to propose a kernel-module scheme that I would not like: this is you chance! ;-)
On Sun, 2005-07-03 at 16:11 +0200, Thorsten Leemhuis wrote:
- These macro definitions are now found in the top of the spec file:
%{!?kver: %define kver %(uname -r)} %define ksrc %{_usrsrc}/kernels/%{kver}-%{_target_cpu} %define moddir /lib/modules/%{kver}/kernel/misc %define mainpkgname ndiswrapper %define mainpkgversion 1.2 %define mainpgkrelease 1 Yes, in other packages I would not like these "mainpkg*" definitions on the top, but in this type of package I think they are helpful.
I think you're mostly right. mainpkgversion and mainpkgrelease are redundant, however. Just use Version and Release for that.
- The main kernel-module package is now named
"kernel-module-%{mainpkgname}-base" (if someones knows something better then "base" tell me).
Perhaps instead of -base, it could be -source. I'm ambivalent on that, really.
The kernel-module itself is now placed in %package -n kernel-module-%{mainpkgname} So only one SRPM is created no matter how much different kernel-modules are build.
This makes sense, good thinking.
- Avoid some problems when kernel-module and kernel are deleted in one
rpm transaction (anyone knows a better way to fix? "Requires(postun): kernel-%{_target_cpu} = %{kver}" does not work): -- Only depmod when System.map is still there %postun -n kernel-module-%{mainpkgname} [ -e "/boot/System.map-%{kver}" ] && \ /sbin/depmod -e -F /boot/System.map-%{kver} %{kver}> /dev/null || : -- In %files: %dir /lib/modules/%{kver}/ so that dir is removed during remove.
The kernel package already owns that dir, I don't see why this is here. rpm removal ordering should take the module out first, then the kernel. Is that not the case?
All in all, very good work. I think your proposal is shaping up to work well. Thanks for the time.
~spot
Am Sonntag, den 03.07.2005, 09:21 -0500 schrieb Tom 'spot' Callaway:
On Sun, 2005-07-03 at 16:11 +0200, Thorsten Leemhuis wrote:
- These macro definitions are now found in the top of the spec file:
%{!?kver: %define kver %(uname -r)} %define ksrc %{_usrsrc}/kernels/%{kver}-%{_target_cpu} %define moddir /lib/modules/%{kver}/kernel/misc %define mainpkgname ndiswrapper %define mainpkgversion 1.2 %define mainpgkrelease 1 Yes, in other packages I would not like these "mainpkg*" definitions on the top, but in this type of package I think they are helpful.
I think you're mostly right. mainpkgversion and mainpkgrelease are redundant, however. Just use Version and Release for that.
/me thinks about that -- can't remember what my idea here was in the beginning.
Removed.
- The main kernel-module package is now named
"kernel-module-%{mainpkgname}-base" (if someones knows something better then "base" tell me).
Perhaps instead of -base, it could be -source. I'm ambivalent on that, really.
Yeah, source might be better. Changed.
[...]
- Avoid some problems when kernel-module and kernel are deleted in one
rpm transaction (anyone knows a better way to fix? "Requires(postun): kernel-%{_target_cpu} = %{kver}" does not work): -- Only depmod when System.map is still there %postun -n kernel-module-%{mainpkgname} [ -e "/boot/System.map-%{kver}" ] && \ /sbin/depmod -e -F /boot/System.map-%{kver} %{kver}> /dev/null || : -- In %files: %dir /lib/modules/%{kver}/ so that dir is removed during remove.
The kernel package already owns that dir, I don't see why this is here. rpm removal ordering should take the module out first, then the kernel. Is that not the case?
No:
[thl@notebook ~]$ ls /lib/modules/ 2.6.12-1.1387_FC4 [thl@notebook ~]$ sudo rpm -ivh /home/rpmbuild/rpmbuild/RPMS/i686/kernel-module-ndiswrapper-1.2-1.2.6.11_1.1369_FC4.i686.rpm /mnt/server/all/usr/www/fedora/linux/core/4/i386/os/Fedora/RPMS/kernel-2.6.11-1.1369_FC4.i686.rpm --oldpackage Preparing... ########################################### [100%] 1:kernel ########################################### [ 50%] 2:kernel-module-ndiswrapp########################################### [100%] [thl@notebook ~]$ sudo rpm -e kernel-2.6.11-1.1369_FC4 kernel-module-ndiswrapper-1.2-1.2.6.11_1.1369_FC4.i686 [thl@notebook ~]$ ls /lib/modules/ 2.6.11-1.1369_FC4 2.6.12-1.1387_FC4 [thl@notebook ~]$ ls /lib/modules/2.6.11-1.1369_FC4/ [thl@notebook ~]$ rpm -qpl /mnt/server/all/usr/www/fedora/linux/core/4/i386/os/Fedora/RPMS/kernel-2.6.11-1.1369_FC4.i686.rpm | grep '/lib/modules/2.6.11-1.1369_FC4$' /lib/modules/2.6.11-1.1369_FC4 [thl@notebook ~]$ sudo rpm -ivh /home/rpmbuild/rpmbuild/RPMS/i686/kernel-module-ndiswrapper-1.2-1.2.6.11_1.1369_FC4.i686.rpm /mnt/server/all/usr/www/fedora/linux/core/4/i386/os/Fedora/RPMS/kernel-2.6.11-1.1369_FC4.i686.rpm --oldpackage Preparing... ########################################### [100%] 1:kernel ########################################### [ 50%] 2:kernel-module-ndiswrapp########################################### [100%] [thl@notebook ~]$ sudo rpm -e kernel-module-ndiswrapper-1.2-1.2.6.11_1.1369_FC4.i686 [thl@notebook ~]$ sudo rpm -e kernel-2.6.11-1.1369_FC4 [thl@notebook ~]$ ls /lib/modules/ 2.6.12-1.1387_FC4 [thl@notebook ~]$
Don't ask me why. Anyone any idea what's wrong here? Probably a stupid error of mine I don't see.
All in all, very good work. I think your proposal is shaping up to work well. Thanks for the time.
np -- what would be of more interest: skvidal, could something like http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ker... work together with yum or a slightly modified version of yum? Or is there anything obvious wrong here that would confuse yum?
On Sun, 2005-07-03 at 17:25 +0200, Thorsten Leemhuis wrote:
Am Sonntag, den 03.07.2005, 09:21 -0500 schrieb Tom 'spot' Callaway:
- The main kernel-module package is now named
"kernel-module-%{mainpkgname}-base" (if someones knows something better then "base" tell me).
Perhaps instead of -base, it could be -source. I'm ambivalent on that, really.
Yeah, source might be better. Changed.
(Still has the -debuginfo problem.)
Don't ask me why. Anyone any idea what's wrong here?
rpm's erasure ordering is known to still have problems (not only the CLI; yum exhibits the same problem). This problem is not limited to kernel module packages nor leftover dirs; for example %postun scriptlets are affected too (/boot/System.map-%{kver} may have disappeared in this case). Dunno if PreReq instead of Requires, or Requires + Requires(postun) instead of Requires would help.
Am Sonntag, den 03.07.2005, 19:34 +0300 schrieb Ville Skyttä:
On Sun, 2005-07-03 at 17:25 +0200, Thorsten Leemhuis wrote:
Am Sonntag, den 03.07.2005, 09:21 -0500 schrieb Tom 'spot' Callaway:
- The main kernel-module package is now named
"kernel-module-%{mainpkgname}-base" (if someones knows something better then "base" tell me).
Perhaps instead of -base, it could be -source. I'm ambivalent on that, really.
Yeah, source might be better. Changed.
(Still has the -debuginfo problem.)
See other mail
Don't ask me why. Anyone any idea what's wrong here?
rpm's erasure ordering is known to still have problems (not only the CLI; yum exhibits the same problem). This problem is not limited to kernel module packages nor leftover dirs; for example %postun scriptlets are affected too (/boot/System.map-%{kver} may have disappeared in this case). Dunno if PreReq instead of Requires, or Requires + Requires(postun) instead of Requires would help.
I tried Requires(postun): /boot/System.map-%{kver} Requires(postun): kernel-%{_target_cpu} = %{kver} before and tried with PreReq now also -- Nothing helped.
On Sun, 2005-07-03 at 09:21 -0500, Tom 'spot' Callaway wrote:
On Sun, 2005-07-03 at 16:11 +0200, Thorsten Leemhuis wrote:
- These macro definitions are now found in the top of the spec file:
%{!?kver: %define kver %(uname -r)} %define ksrc %{_usrsrc}/kernels/%{kver}-%{_target_cpu} %define moddir /lib/modules/%{kver}/kernel/misc %define mainpkgname ndiswrapper %define mainpkgversion 1.2 %define mainpgkrelease 1 Yes, in other packages I would not like these "mainpkg*" definitions on the top, but in this type of package I think they are helpful.
I think you're mostly right. mainpkgversion and mainpkgrelease are redundant, however. Just use Version and Release for that.
Yep.
A policy or at least guidelines exactly where to place the modules below /lib/modules/%{kver} would be nice. See the TODO item in the Wiki page.
Personally I think /lib/modules/%{kver}/kernel/... is bad, these are not modules shipped with the kernel so intruding that space feels wrong. "updates" is bad too, these aren't updates to in-kernel modules. Upstream docs (see kbuild/modules.txt in the kernel source tree) talk about "extra", but IIRC in the past people have had the interpretation that it shouldn't be used for packaged modules.
Anyway, /lib/modules/%{kver}/extra/%{mainpkgname} using the above definitions gets my vote, assuming there won't be problems with module-init-tools or other stuff with that.
The kernel-module itself is now placed in %package -n kernel-module-%{mainpkgname} So only one SRPM is created no matter how much different kernel-modules are build.
This makes sense, good thinking.
Except as described in my earlier mail to this thread, if the main package's NVR doesn't change between rebuilds, we have a problem with -debuginfo for these builds. https://www.redhat.com/archives/fedora-packaging/2005-June/msg00055.html
Am Sonntag, den 03.07.2005, 19:12 +0300 schrieb Ville Skyttä:
On Sun, 2005-07-03 at 09:21 -0500, Tom 'spot' Callaway wrote:
On Sun, 2005-07-03 at 16:11 +0200, Thorsten Leemhuis wrote:
I think you're mostly right. mainpkgversion and mainpkgrelease are redundant, however. Just use Version and Release for that.
Yep.
Removed already :))
A policy or at least guidelines exactly where to place the modules below /lib/modules/%{kver} would be nice.
Yes.
My proposal: Put it at the same place where the normal module would be places by a normal "make install" of that package. That has several benefits:
- If a external Documenation says "Look for the module in /lib/$(uname -r)/somewhere it is exactly found there where the Documentation says. No special fedora way
- People sometimes try to compile a module on their own and go for the rpm later (or the other way round). They might end with two different modules in the kernel tree -- witch one wins? This can't happen if the modules are installed at their usual place.
Personally I think /lib/modules/%{kver}/kernel/... is bad, these are not modules shipped with the kernel so intruding that space feels wrong.
I don't share that opinion. I think it's confusing for no real reason. If we have a gnome-app or enhancement it's installed in the gnome-space in the filesystem also.
"updates" is bad too, these aren't updates to in-kernel modules.
Yes.
The kernel-module itself is now placed in %package -n kernel-module-%{mainpkgname} So only one SRPM is created no matter how much different kernel-modules are build.
This makes sense, good thinking.
Except as described in my earlier mail to this thread, if the main package's NVR doesn't change between rebuilds, we have a problem with -debuginfo for these builds. https://www.redhat.com/archives/fedora-packaging/2005-June/msg00055.html
Uhh, sorry, I knew I was forgetting something. Yes, that should be fixed. So what are our options:
1) create the debug-pkg ourself and don't rely on the internal rpm solution. 2) use a spec similar two my first proposal where the actual source of the kernel-module is in an external rpm that is a BuildRequire. Resulting SRPMS are small then. 3) No sub-pkg -- gives a lot of SRPMS that might take a lot of space... 4) <your idea here>
If 1) is easy I'll vote for that. Otherwise I'd like to give 2) a try. 3) is a no go when I think of kernel-modules like ati-fglrx where each SRPM package would take around 4,5 MByte currently. Especially if we want to rebuild kernel-module for nearly all kernels that ever were published (someone suggested that somewhere in this thread iirc).
On Sun, 2005-07-03 at 19:10 +0200, Thorsten Leemhuis wrote:
Am Sonntag, den 03.07.2005, 19:12 +0300 schrieb Ville Skyttä:
A policy or at least guidelines exactly where to place the modules below /lib/modules/%{kver} would be nice.
Yes.
My proposal: [...]
I forgot to mention that I don't have strong opinions on this, as long as some understandable guidelines exist. Installing into whatever upstream uses by default does have its merits like you pointed out.
Except as described in my earlier mail to this thread, if the main package's NVR doesn't change between rebuilds, we have a problem with -debuginfo for these builds. https://www.redhat.com/archives/fedora-packaging/2005-June/msg00055.html
Uhh, sorry, I knew I was forgetting something. Yes, that should be fixed. So what are our options:
- create the debug-pkg ourself and don't rely on the internal rpm
solution.
Urgh, have you looked at glibc.spec? We really don't want that in every kernel module package. Perhaps it's acceptable if we could just call a script somewhere and be done with it.
But the fundamental problem isn't actually limited to -debuginfo, but affects SRPMs to some extent as well (same-NEVR'd SRPM from different builds - what to do with them?). See below.
- use a spec similar two my first proposal where the actual source of
the kernel-module is in an external rpm that is a BuildRequire.
I still don't like this too much in principle, but all the alternatives seem to have shortcomings of their own as well, so it seems we need to pick the least bad alternative. And this is a definite candidate. If I understand correctly, the binary-containing-source package could be theoretically reused in DKMS and similar systems - that would make this approach even more attractive.
- No sub-pkg -- gives a lot of SRPMS that might take a lot of space...
Theoretically, we can choose to not ship these SRPMS at all. Their rebuild results vary depending on the passed in (or detected, in case of local rebuilds) kver anyway; what it was at build time won't affect the results. So essentially they're just the same things except for the filename of the SRPM. OTOH, not shipping them sounds fundamentally wrong.
In the meantime, status of kernel-module-thinkpad in CVS: - The SRPM duplication (non?)problem persists. - It can be built for the running kernel without specifying kver, but the -debuginfo problem and the problem of getting same-NEVR'd source packages from different (specfile based) builds bites then. This would not affect us, I assume we will always pass kver. And the problem would be trivially solved by just removing the possibility to build without specifying kver.
I'm starting to think that Thorsten's 2) above would actually solve all our problems, even if it's hackish, not the pretties one and it's a bit hard to understand.
Am Sonntag, den 03.07.2005, 19:10 +0200 schrieb Thorsten Leemhuis:
Am Sonntag, den 03.07.2005, 19:12 +0300 schrieb Ville Skyttä:
On Sun, 2005-07-03 at 09:21 -0500, Tom 'spot' Callaway wrote:
On Sun, 2005-07-03 at 16:11 +0200, Thorsten Leemhuis wrote:
The kernel-module itself is now placed in %package -n kernel-module-%{mainpkgname} So only one SRPM is created no matter how much different kernel-modules are build.
This makes sense, good thinking.
Except as described in my earlier mail to this thread, if the main package's NVR doesn't change between rebuilds, we have a problem with -debuginfo for these builds. https://www.redhat.com/archives/fedora-packaging/2005-June/msg00055.html
Uhh, sorry, I knew I was forgetting something. Yes, that should be fixed. So what are our options:
- create the debug-pkg ourself and don't rely on the internal rpm
solution.
[...]
If 1) is easy I'll vote for that.
I tried, was not that hard (if I didn't miss anything). Results are found at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
On Sun, 2005-07-03 at 21:01 +0200, Thorsten Leemhuis wrote:
- create the debug-pkg ourself and don't rely on the internal rpm
solution.
[...]
If 1) is easy I'll vote for that.
I tried, was not that hard (if I didn't miss anything). Results are found at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
I like this approach the best. The only change is that the -debuginfo package needs to Require: kernel-module-%{mainpkgname} (its not any good without it).
As to location, I'm very inclined to standardize on: /lib/modules/%{kver}/extra/%{mainpkgname}
To address Thorsten's concerns:
- Lots of external documentation conflicts with what we package in Fedora Core or Fedora Extras. rpm -ql kernel-module-openafs works very well. :)
- Double existing modules. Same thing happens if you compile Apache by hand, you get two copies of Apache on the system. There may EVEN be cases in which you want to do this for kernel modules. The modutils look for the first module which matches that name in the tree. By using extras/, we trump the kernel/ drivers directory.
I think it is important to point out which drivers came with the FC kernel, and which ones are provided by FE addon packages. Using the "extra" namespace is the cleanest way to do this.
~spot
On Mon, 2005-07-04 at 09:30 -0500, Tom 'spot' Callaway wrote:
On Sun, 2005-07-03 at 21:01 +0200, Thorsten Leemhuis wrote:
- create the debug-pkg ourself and don't rely on the internal rpm
solution.
[...]
If 1) is easy I'll vote for that.
I tried, was not that hard (if I didn't miss anything). Results are found at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
I like this approach the best.
I like that too, but the dilemma with the same-NEVR'd source rpm persists. I'm not sure if it's a design goal or a design flaw, but the little (ha!) pedant in me says it's the latter. To clarify:
- kernel-module-foo-1.0-1.src.rpm in repo - check out the package from CVS, build for a new kernel -> get another kernel-module-foo-1.0-1.src.rpm which != the original
What to do with the new source rpm? Discarding would be ugly, and overwriting the existing srpm in the repo even uglier. Rebuilding from an existing srpm in the repo would help (or just pretending it was done, and discarding the new one :)).
Allowing the source rpm's NEVR to change between rebuilds as usual and always shipping them would not have this problem at the (negligible IMHO) cost of more disk space consumption. That approach would not need any special -debuginfo treatment either.
In this scenario, we'd just need to figure out how we'd like the CVS tags to be. Turning things upside down and explicitly specifying a default %{kver} assignment in specfiles when new kernels are released (+ the smp etc variants each separately) and allowing local rebuilders override it with a --define would solve that easily, with the added benefit that the build system wouldn't have to know about passing any special --defines to these builds. Whoever requests the builds would just have to be able to specify the target architecture(s). As an example, I've changed kernel-module-thinkpad in Extras CVS (devel) to do the above (currently for the UP build).
Mmh, maybe I should just shut up about this now :). Does the above make any sense?
The only change is that the -debuginfo package needs to Require: kernel-module-%{mainpkgname} (its not any good without it).
Note: -debuginfo packages created the usual way don't have that dependency. But that's probably because it could be hard to implement in the automagic that generates them nowadays. If such a dependency would be added, it should be fully versioned, though.
As to location, I'm very inclined to standardize on: /lib/modules/%{kver}/extra/%{mainpkgname}
Works for me.
On Tue, 2005-07-05 at 22:34 +0300, Ville Skyttä wrote:
On Mon, 2005-07-04 at 09:30 -0500, Tom 'spot' Callaway wrote:
On Sun, 2005-07-03 at 21:01 +0200, Thorsten Leemhuis wrote:
- create the debug-pkg ourself and don't rely on the internal rpm
solution.
[...]
If 1) is easy I'll vote for that.
I tried, was not that hard (if I didn't miss anything). Results are found at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
I like this approach the best.
I like that too, but the dilemma with the same-NEVR'd source rpm persists. I'm not sure if it's a design goal or a design flaw, but the little (ha!) pedant in me says it's the latter. To clarify:
- kernel-module-foo-1.0-1.src.rpm in repo
- check out the package from CVS, build for a new kernel -> get another kernel-module-foo-1.0-1.src.rpm which != the original
Why would the src.rpm not be the same as the original? The spec file and source tarball should be consistent, and not affected by a rebuild.
~spot
On Tue, 2005-07-05 at 15:33 -0500, Tom 'spot' Callaway wrote:
On Tue, 2005-07-05 at 22:34 +0300, Ville Skyttä wrote:
- kernel-module-foo-1.0-1.src.rpm in repo
- check out the package from CVS, build for a new kernel -> get another kernel-module-foo-1.0-1.src.rpm which != the original
Why would the src.rpm not be the same as the original? The spec file and source tarball should be consistent, and not affected by a rebuild.
There are variables like build host, build time, file timestamps, file modes, --define's passed to the srpm build, possibly other buildsys configuration variations etc. All of which are sort of cosmetic, but nevertheless result in a different source rpm.
On Tue, 2005-07-05 at 23:47 +0300, Ville Skyttä wrote:
On Tue, 2005-07-05 at 15:33 -0500, Tom 'spot' Callaway wrote:
On Tue, 2005-07-05 at 22:34 +0300, Ville Skyttä wrote:
- kernel-module-foo-1.0-1.src.rpm in repo
- check out the package from CVS, build for a new kernel -> get another kernel-module-foo-1.0-1.src.rpm which != the original
Why would the src.rpm not be the same as the original? The spec file and source tarball should be consistent, and not affected by a rebuild.
There are variables like build host, build time, file timestamps, file modes, --define's passed to the srpm build, possibly other buildsys configuration variations etc. All of which are sort of cosmetic, but nevertheless result in a different source rpm.
I'm really not worried about cosmetic changes. None of these things should affect the binary packages generated from that src.rpm.
~spot
On Tue, 2005-07-05 at 16:12 -0500, Tom 'spot' Callaway wrote:
On Tue, 2005-07-05 at 23:47 +0300, Ville Skyttä wrote:
There are variables like build host, build time, file timestamps, file modes, --define's passed to the srpm build, possibly other buildsys configuration variations etc. All of which are sort of cosmetic, but nevertheless result in a different source rpm.
I'm really not worried about cosmetic changes. None of these things should affect the binary packages generated from that src.rpm.
--defines end up in various dependencies of the source rpm, which does not matter as long as one doesn't use its dependencies for anything, but the specfile's instead. (This is not limited to these packages.)
The original question remains though; what to do with the srpms? Discard or overwrite the ones already in the repo? My +1 to the former, or more generally: never overwrite any package in the repository.
On Wed, 2005-07-06 at 10:00 +0300, Ville Skyttä wrote:
On Tue, 2005-07-05 at 16:12 -0500, Tom 'spot' Callaway wrote:
On Tue, 2005-07-05 at 23:47 +0300, Ville Skyttä wrote:
There are variables like build host, build time, file timestamps, file modes, --define's passed to the srpm build, possibly other buildsys configuration variations etc. All of which are sort of cosmetic, but nevertheless result in a different source rpm.
I'm really not worried about cosmetic changes. None of these things should affect the binary packages generated from that src.rpm.
--defines end up in various dependencies of the source rpm, which does not matter as long as one doesn't use its dependencies for anything, but the specfile's instead. (This is not limited to these packages.)
The original question remains though; what to do with the srpms? Discard or overwrite the ones already in the repo? My +1 to the former, or more generally: never overwrite any package in the repository.
Personally, since the buildsystem is going to have to treat kernel-module-* packages differently, I'd like to see it build them like this:
When a make build is done in kernel-module-foo/FC-3/, the buildsystem assembles the sources and makes a SRPM. It then looks at a list (either generated at buildtime, or preexistant) of the released kernels for FC-3, and iterates through each of them, running rpmbuild --rebuild --define "kver $VERSION". At the end of the loop, we should have all the binary packages and a single SRPM.
~spot
On Wed, Jul 06, 2005 at 08:24:46AM -0500, Tom 'spot' Callaway wrote:
On Wed, 2005-07-06 at 10:00 +0300, Ville Skyttä wrote:
On Tue, 2005-07-05 at 16:12 -0500, Tom 'spot' Callaway wrote:
On Tue, 2005-07-05 at 23:47 +0300, Ville Skyttä wrote:
There are variables like build host, build time, file timestamps, file modes, --define's passed to the srpm build, possibly other buildsys configuration variations etc. All of which are sort of cosmetic, but nevertheless result in a different source rpm.
I'm really not worried about cosmetic changes. None of these things should affect the binary packages generated from that src.rpm.
--defines end up in various dependencies of the source rpm, which does not matter as long as one doesn't use its dependencies for anything, but the specfile's instead. (This is not limited to these packages.)
The original question remains though; what to do with the srpms? Discard or overwrite the ones already in the repo? My +1 to the former, or more generally: never overwrite any package in the repository.
Personally, since the buildsystem is going to have to treat kernel-module-* packages differently, I'd like to see it build them like this:
When a make build is done in kernel-module-foo/FC-3/, the buildsystem assembles the sources and makes a SRPM. It then looks at a list (either generated at buildtime, or preexistant) of the released kernels for FC-3, and iterates through each of them, running rpmbuild --rebuild --define "kver $VERSION". At the end of the loop, we should have all the binary packages and a single SRPM.
~spot
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260 Fedora Extras Steering Committee Member (RPM Standards and Practices) Aurora Linux Project Leader: http://auroralinux.org Lemurs, llamas, and sparcs, oh my!
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
+1
Personally, since the buildsystem is going to have to treat kernel-module-* packages differently, I'd like to see it build them like this:
When a make build is done in kernel-module-foo/FC-3/, the buildsystem assembles the sources and makes a SRPM. It then looks at a list (either generated at buildtime, or preexistant) of the released kernels for FC-3, and iterates through each of them, running rpmbuild --rebuild --define "kver $VERSION". At the end of the loop, we should have all the binary packages and a single SRPM.
Also, if possible, it would be awesome if the buildsystem could automatically do a build when a new kernel goes into FC (for that kernel only). Barring that, it would be nice if there was some way for kernel-module-* maintainers to be notified of the new kernel update in advance.
~spot
On Wed, 2005-07-06 at 09:39 -0500, Tom 'spot' Callaway wrote:
Personally, since the buildsystem is going to have to treat kernel-module-* packages differently, I'd like to see it build them like this:
When a make build is done in kernel-module-foo/FC-3/, the buildsystem assembles the sources and makes a SRPM. It then looks at a list (either generated at buildtime, or preexistant) of the released kernels for FC-3, and iterates through each of them, running rpmbuild --rebuild --define "kver $VERSION". At the end of the loop, we should have all the binary packages and a single SRPM.
That still doesn't answer the question, so at the risk of parroting, I'm going to try once again because I don't know if people understand the question and the problem:
Yes, it's a single SRPM resulting from _that_ build. It's very likely that there would be another one with the same NEVR from an earlier build sitting in the repository already. More clarification:
- in the beginning, we have only kernel X - rebuild a kernel-module-foo package for it, results: - k-m-foo-1.0-1.src.rpm - k-m-foo-1.0-1.X...rpm (all smp, xen0 etc variants and archs) - the above get pushed to the repository - then, kernel Y is released, rebuild k-m-foo (which has had no changes in the meantime) for it, results: - k-m-foo-1.0-1.src.rpm - k-m-foo-1.0-1.Y...rpm (all smp, xen0 etc variants and archs)
Do we discard the new k-m-foo-1.0-1.src.rpm or overwrite the one which is already in the repository?
Or do we solve this with a policy (assuming the process you described above would be implemented): "always bump the release of the "main" package of a module package before building it for a new kernel"? That'd result in only one srpm per module package per released kernel "family".
Also, if possible, it would be awesome if the buildsystem could automatically do a build when a new kernel goes into FC (for that kernel only). Barring that, it would be nice if there was some way for kernel-module-* maintainers to be notified of the new kernel update in advance.
Sure. But I'm becoming worried about this discussion already placing requirements on the build system, which will delay our ability to ship any module packages even further. We could start with something much simpler; several suggestions have already been outlined in this thread. The day "make build" would allow specifying the arch, we could start building and shipping the packages, and could improve the process incrementally.
On Wed, 2005-07-06 at 18:16 +0300, Ville Skyttä wrote:
- in the beginning, we have only kernel X
- rebuild a kernel-module-foo package for it, results:
- k-m-foo-1.0-1.src.rpm
- k-m-foo-1.0-1.X...rpm (all smp, xen0 etc variants and archs)
- the above get pushed to the repository
- then, kernel Y is released, rebuild k-m-foo (which has had no changes in the meantime) for it, results:
- k-m-foo-1.0-1.src.rpm
- k-m-foo-1.0-1.Y...rpm (all smp, xen0 etc variants and archs)
Do we discard the new k-m-foo-1.0-1.src.rpm or overwrite the one which is already in the repository?
I vote we discard the new one.
Or do we solve this with a policy (assuming the process you described above would be implemented): "always bump the release of the "main" package of a module package before building it for a new kernel"? That'd result in only one srpm per module package per released kernel "family".
It would also result in unnecessary package upgrades.
Also, if possible, it would be awesome if the buildsystem could automatically do a build when a new kernel goes into FC (for that kernel only). Barring that, it would be nice if there was some way for kernel-module-* maintainers to be notified of the new kernel update in advance.
Sure. But I'm becoming worried about this discussion already placing requirements on the build system, which will delay our ability to ship any module packages even further. We could start with something much simpler; several suggestions have already been outlined in this thread. The day "make build" would allow specifying the arch, we could start building and shipping the packages, and could improve the process incrementally.
The build system has other issues. Right now, I don't think it can deal with ExclusiveArch correctly. If it did, then we could just have "make build" parse ExclusiveArch for kernel-module packages, and build for all those arches. I've CC'd seth and dan to see what they say.
~spot
On Wed, 2005-07-06 at 10:28 -0500, Tom 'spot' Callaway wrote:
On Wed, 2005-07-06 at 18:16 +0300, Ville Skyttä wrote:
Or do we solve this with a policy (assuming the process you described above would be implemented): "always bump the release of the "main" package of a module package before building it for a new kernel"? That'd result in only one srpm per module package per released kernel "family".
It would also result in unnecessary package upgrades.
How so? We wouldn't be building the module package again for older kernels unless there are some real changes in it. To illustrate (leaving "tr - _" needed for the release tag aside for a moment):
kernel-%{kver1} # eg. kver1 = 2.6.11-1.1369_FC4 k-m-foo-1.0-1.%{kver1}
kernel-%{kver2} # eg. kver2 = 2.6.12-1.1387_FC4 k-m-foo-1.0-1.%{kver2}
In the above, k-m-foo-1.0-1.%{kver2} must not upgrade k-m-foo-1.0-1.%{kver1} in the usual sense anyway, instead they both need to be installed. So this needs to be special cased in depsolvers, and bumping the first digit of the release tag of k-m-foo when built for kernel %{kver2} should not have any effect on the special case.
Also note that a side effect of stuffing %{kver} into the release (or version) tag means that we might be in for surprises if we trust rpm's version comparison algorithm to sort the "same" module packages for different kernels consistently with the corresponding kernel versions. For example:
Version: 1.0 Release: 1.2.6.12_1.1234
Version: 1.0 Release: 1.2.6.12.1_1.2345
The former is newer as far as rpm is concerned, it compares "1234" in the former against "1" in the latter.
If the kernel packages' version number always has the same number of "segments" (currently 3), and its Epoch is never bumped, the inconsistency would never occur though.
On Wed, 2005-07-06 at 21:48 +0300, Ville Skyttä wrote:
On Wed, 2005-07-06 at 10:28 -0500, Tom 'spot' Callaway wrote:
On Wed, 2005-07-06 at 18:16 +0300, Ville Skyttä wrote:
Or do we solve this with a policy (assuming the process you described above would be implemented): "always bump the release of the "main" package of a module package before building it for a new kernel"? That'd result in only one srpm per module package per released kernel "family".
It would also result in unnecessary package upgrades.
How so? We wouldn't be building the module package again for older kernels unless there are some real changes in it. To illustrate (leaving "tr - _" needed for the release tag aside for a moment):
The buildsystem has no way of knowing this, nor do users. The only reasonable way is to rebuild for all existant kernels when we increment release. This means that if a new kernel comes out, all my old kernels get new kernel-module-foo packages downloaded and installed, when all i needed was a kernel-module-foo for my new kernel.
~spot
On Wed, 2005-07-06 at 15:00 -0500, Tom 'spot' Callaway wrote:
On Wed, 2005-07-06 at 21:48 +0300, Ville Skyttä wrote:
How so? We wouldn't be building the module package again for older kernels unless there are some real changes in it. To illustrate (leaving "tr - _" needed for the release tag aside for a moment):
The buildsystem has no way of knowing this, nor do users. The only reasonable way is to rebuild for all existant kernels when we increment release.
Well, we obviously disagree here, as well as on a number of other issues in this thread. But it seems I got my concerns across, so I'll happily switch back to the "waits patiently" mode now.
The build system has other issues. Right now, I don't think it can deal with ExclusiveArch correctly. If it did, then we could just have "make build" parse ExclusiveArch for kernel-module packages, and build for all those arches. I've CC'd seth and dan to see what they say.
does this imply that if you ExclusiveArch to i386 that it should build for i386, i686, athlon, automatically?
-sv
seth vidal (skvidal@phy.duke.edu) said:
The build system has other issues. Right now, I don't think it can deal with ExclusiveArch correctly. If it did, then we could just have "make build" parse ExclusiveArch for kernel-module packages, and build for all those arches. I've CC'd seth and dan to see what they say.
does this imply that if you ExclusiveArch to i386 that it should build for i386, i686, athlon, automatically?
Realistically, you'd want to either:
a) ExclusiveArch: <all sub arches the kernel builds>
or
b) have a magic list somewhere of things that builds for 'wherever there's a kernel'
Bill
On Thu, 2005-07-07 at 11:05 -0400, seth vidal wrote:
The build system has other issues. Right now, I don't think it can deal with ExclusiveArch correctly. If it did, then we could just have "make build" parse ExclusiveArch for kernel-module packages, and build for all those arches. I've CC'd seth and dan to see what they say.
does this imply that if you ExclusiveArch to i386 that it should build for i386, i686, athlon, automatically?
No, but I don't know what the buildsystem will do if it sees:
ExclusiveArch: i586 i686 ppc ppc64 x86_64
~spot
On Tue, Jul 05, 2005 at 11:47:43PM +0300, Ville Skyttä wrote:
There are variables like build host, build time, file timestamps, file modes, --define's passed to the srpm build, possibly other buildsys configuration variations etc. All of which are sort of cosmetic, but nevertheless result in a different source rpm.
This is really no worse than what we've already got. From the FC4 src rpms:
$ rpm -qp --qf '%{arch}\n' *.src.rpm --nodigest --nosignature|sort|uniq i386 ia64 noarch ppc ppc64 ppc64iseries x86_64
Am Montag, den 04.07.2005, 09:30 -0500 schrieb Tom 'spot' Callaway:
On Sun, 2005-07-03 at 21:01 +0200, Thorsten Leemhuis wrote:
- create the debug-pkg ourself and don't rely on the internal rpm
solution.
[...]
If 1) is easy I'll vote for that.
I tried, was not that hard (if I didn't miss anything). Results are found at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
I like this approach the best.
:)
The only change is that the -debuginfo package needs to Require: kernel-module-%{mainpkgname} (its not any good without it).
Done, with V-R
As to location, I'm very inclined to standardize on: /lib/modules/%{kver}/extra/%{mainpkgname}
I changed it -- but I still would prefer the directory used by upstream. Just a note: "extra =! extras" so other 3rd party repos probably could and should use this also.
I'll update the Wiki and the example found on my homepage when I have online-access the next time.
Am Donnerstag, den 07.07.2005, 09:13 +0200 schrieb Thorsten Leemhuis:
Am Montag, den 04.07.2005, 09:30 -0500 schrieb Tom 'spot' Callaway:
On Sun, 2005-07-03 at 21:01 +0200, Thorsten Leemhuis wrote:
I tried, was not that hard (if I didn't miss anything). Results are found at http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kernel-module-example/ in the wiki at http://www.fedoraproject.org/wiki/Extras/KernelModuleProposal
Another small update: the kernel-module is now installed as 0644 as proposed by Ville. There is no need for 744 (rumors circulate that there were problems with 644 in the past but nobody remembers the details).
On Sun, Jul 03, 2005 at 07:10:41PM +0200, Thorsten Leemhuis wrote:
My proposal: Put it at the same place where the normal module would be places by a normal "make install" of that package. That has several benefits:
[benefits snipped]
Drawback -- some packages (like OpenAFS, go figure) want to put the module in a really stupid place by default. We should follow the FHS and Fedora/RH precedent of fixing install location of files when the package is non-standard.
packaging@lists.fedoraproject.org
-
Bill Nottingham -
Jack Neely -
Matthew Miller -
Ralf Corsepius -
seth vidal -
Thorsten Leemhuis -
Tom Callaway -
Ville Skyttä