On Tue, Jul 18, 2006 at 01:59:34PM +0200, Eric Tanguy wrote:
kernels=`rpm -qf /boot/vmlinuz-* | grep -v "^file .* is not owned by any package"` uname_rs=`rpm -ql $kernels | grep ^/boot/vmlinuz- | sed -e's,^/boot/vmlinuz-(.*)$,\1,'` for kmdl in `rpm -qa *kmdl* | sed -e's,-kmdl-.*,-kmdl,' | sort -u`; do for uname_r in $uname_rs; do package=${kmdl}-$uname_r rpm -q $package > /dev/null 2>&1 || echo $package done done | xargs -r smart install -y
Ok this script seems to use smart. is it working with yum ?
Yes, I think "smart install -y" -> "yum -y install" is the only change needed. And for apt it would be "apt-get -y install".
How is this problem handled by livna because it works fine using this repo : the kernel modules are installed and not updated
It's known that there are issues with this scheme, or any scheme that will merge module and kernel versions.
rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work.
yum/smart/apt can be tought to be more clever than rpm and try to do the proper thing, which they will only be able to do if the merged versions are distangled again, e.g. hidden in Provides: and the like.
So in order for a merged-version scheme to work there is special handling neccessary (and for atrpms' system, too, but read on). I prefer to have rpm itself already do something sensible and since it can only upgrade one versioning system (e.g. either the kernel's or the module's) it's preferable to allow it to upgrade the module within the same kernel and not touch the cross-kernel boundaries. "No harm done" like accidentially removing kmdls for other kernels or coinstalling several kmdls for the same kernel. You "only" need to reinstall the missing kmdl upon each kernel upgrade.
It's far easier to get this into the brains of yum/smart/apt (e.g. translate the 9 lines of bash above into plugings/hooks for these depsolvers) or simply use the 9liner above than have a scheme which is broken at the level of rpm.
Incidentially one on my personal targets is to get thie discussed in the fedora packaging group and review the currently scheme. Of course, I'm proposing adoption of ATrpms' kmdl scheme :)
http://www.fedoraproject.org/wiki/Packaging/GuidelinesTodo
when i install a new kernel and are removed when the kernel is removed.
How the dependencies to the kernel are installed (which is responsible for having an automatic removal on kernel removal) is a different story, which the version merging does not break. The issue is with the system (system=rpm and(or any higher depsolver tool, e.g. yum/smart/apt) being able to distinguish different foo-kmdls as _upgrades_ vs _coinstalls_.
dropping fedora-list from CC
Hi All!
Seems Axel wants to bring kmod onto the topic again -- I can understand his situation so let's get this discussed...
Axel Thimm schrieb:
On Tue, Jul 18, 2006 at 01:59:34PM +0200, Eric Tanguy wrote:
How is this problem handled by livna because it works fine using this repo : the kernel modules are installed and not updated It's known that there are issues with this scheme, or any scheme that will merge module and kernel versions.
rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work.
Well, yes, coinstall will fail because this would result in a file conflict. But rpm -U works fine (but removes the older version) with the current Extras kmod scheme and "yum install foo" AFAICS works fine, too.
yum/smart/apt can be tought to be more clever than rpm and try to do the proper thing, which they will only be able to do if the merged versions are distangled again, e.g. hidden in Provides: and the like.
So in order for a merged-version scheme to work there is special handling neccessary (and for atrpms' system, too, but read on).
All special handling I'm aware of is that all modules that have
Provides: kernel-modules
are installed by yum and not updated. Similar how all packages providing "kernel" are installed and not updated. Yum support that for a long time (two years?) already (IIRC). The latgest version seems to also have some magic to updat kmod packages if there would be conflics otherwise.
I prefer to have rpm itself already do something sensible and since it can only upgrade one versioning system (e.g. either the kernel's or the module's) it's preferable to allow it to upgrade the module within the same kernel and not touch the cross-kernel boundaries. "No harm done" like accidentially removing kmdls for other kernels or coinstalling several kmdls for the same kernel. You "only" need to reinstall the missing kmdl upon each kernel upgrade.
It's far easier to get this into the brains of yum/smart/apt (e.g. translate the 9 lines of bash above into plugings/hooks for these depsolvers) or simply use the 9liner above than have a scheme which is broken at the level of rpm.
Well, we (fedora.us and Livna in this case) had a scheme that used the output from "uname -r" in the name similar how atrpms still uses it AFAICS. We (e.g. those involved in the kmod discussion for Extras) were not satisfied with it AFICS.
The decision against using the uname-output in the name was one of the first ones when the Extras kmod-standard was discussed. It was a rather quick decision IIRC to not use it.
Incidentially one on my personal targets is to get thie discussed in the fedora packaging group and review the currently scheme. Of course, I'm proposing adoption of ATrpms' kmdl scheme :)
Is you scheme documented somewhere properly so we can look at it again? Please also post a URL to the macros for FC5 (maybe rawhide, too) -- we probably need them if we want to look at the scheme.
BTW, livna uses the Extras scheme for all kmod-packages since FC5 now. AFICS it works a lot better than the old fedora.us scheme. There are still some things that could need to be improved:
- yum should install kmods for all latest kernels that are installed -- e.g. is one has a smp and a xen0 kernel installed yum should install kmods for both when running "yum install kmod-foo"
- plague needs support for kmod-srpms to get rid of the hard coded kversion and kvariants
when i install a new kernel and are removed when the kernel is removed.
How the dependencies to the kernel are installed (which is responsible for having an automatic removal on kernel removal) is a different story, which the version merging does not break. The issue is with the system (system=rpm and(or any higher depsolver tool, e.g. yum/smart/apt) being able to distinguish different foo-kmdls as _upgrades_ vs _coinstalls_.
As I said, Provides: kernel-modules and yum will install them. When you run rpm directly you are on your own -- it similar with the kernel itself.
CU thl
On Wed, Jul 19, 2006 at 07:40:38PM +0200, Thorsten Leemhuis wrote:
Seems Axel wants to bring kmod onto the topic again -- I can understand his situation so let's get this discussed...
Axel Thimm schrieb:
On Tue, Jul 18, 2006 at 01:59:34PM +0200, Eric Tanguy wrote:
How is this problem handled by livna because it works fine using this repo : the kernel modules are installed and not updated It's known that there are issues with this scheme, or any scheme that will merge module and kernel versions.
rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work.
Well, yes, coinstall will fail because this would result in a file conflict. But rpm -U works fine (but removes the older version) with the current Extras kmod scheme and "yum install foo" AFAICS works fine, too.
No, rpm -U would remove both kernel modules from both kernels and only install the selected kernel module, you end up with one kernel losing the kernel module w/o replacement.
rpm -U will always leave only one kernel module package behind unless these packages are disambiguated in their name by extending the name to contain the kernel's uname -r.
Well, we (fedora.us and Livna in this case) had a scheme that used the output from "uname -r" in the name similar how atrpms still uses it AFAICS. We (e.g. those involved in the kmod discussion for Extras) were not satisfied with it AFICS.
The decision against using the uname-output in the name was one of the first ones when the Extras kmod-standard was discussed. It was a rather quick decision IIRC to not use it.
I wished I had been involved at that time to argue against this unfortunate change, but probably it was during the "cold war" where most people including myself had run out of batteries and could not try to save the world anymore :)
Anyway it's never too late, I'm optimistic about the new world order. :)
Incidentially one on my personal targets is to get thie discussed in the fedora packaging group and review the currently scheme. Of course, I'm proposing adoption of ATrpms' kmdl scheme :)
Is you scheme documented somewhere properly so we can look at it again? Please also post a URL to the macros for FC5 (maybe rawhide, too) -- we probably need them if we want to look at the scheme.
I committed myself to prepare some documentation after asking in the packaging group whether there is general interest to review this part of the guidelines.
Not only on the actual usage but also on the rationale behind it hoping that some design decisions will become more apparent. In this sense this thread is perhaps a tad too early, but it was triggered by some user discussion and led naturally to explaining some parts of it with mentioning the review proposal. But since it got started (which is a good thing) we should not let it drown again. I'll try to prepare the docs ASAP.
BTW, livna uses the Extras scheme for all kmod-packages since FC5 now. AFICS it works a lot better than the old fedora.us scheme. There are still some things that could need to be improved:
But if it's broken already at the rpm level how can it be better? You can start teaching yum special handling, then smart and finally apt and maybe up2date since the scheme is to be used in RHEL at some point in time. But is this really the way to go?
Admittedly the kmdl scheme or the old livna scheme with the uname -r in name also need some special handling, but it's easier to add the coinstall logic (e.g. 9 lines of bash code were already anough) and the drawbacks w/o that special handling are far less severe (e.g. you never run into the situation that you might blow away kernel modules of unrelated kernels with a simple rpm -U).
To summarize:
o schemes w/o uname -r in name - break at rpm level - depsolver special handling is therefore a *must* otherwise old working kernel may get their kernel modules nuked. And even if all depsolvers will be patched to take this into account you are still broken on rpm level.
o schemes w/ uname -r in name - no coinstalls at rpm level - depsolver special handling is *nice-to-have* otherwise new kernels won't "inherit" kernel modules installed for old ones. In the worst case users have to manually reinstall kmdl after/during each kernel upgrade.
The latter "worst-case" scenario is what ATrpms ships for a couple of years now. kmdl at ATrpms be it for multimedia (ivtv, v4l, nvidia) or for wireless (ipw*, madwifi) have a large userbase that seem to cope well with reinstalling kmdls after a kernel upgrade (it's just the 9-liner bash scriplet after all). So field experiance if you'd like to call it, already shows that uname-r-in-name schemes are indeed consumable.
How the dependencies to the kernel are installed (which is responsible for having an automatic removal on kernel removal) is a different story, which the version merging does not break. The issue is with the system (system=rpm and(or any higher depsolver tool, e.g. yum/smart/apt) being able to distinguish different foo-kmdls as _upgrades_ vs _coinstalls_.
As I said, Provides: kernel-modules and yum will install them. When you run rpm directly you are on your own -- it similar with the kernel itself.
No, with the kernel it's always just rpm -i, never rpm -U, that's all a user needs to know. With kernel modules w/o uname-r-in-name you can easily run into a situation where you need something in between like the example above: Coinstall wrt to an installed kernel module package for another kernel, but upgrade any installed kernel module package for the matching kernel.
In order for a depsolver to get that logic installed you need to reverse engineer the two dimensional versions of modules and kernel out of the kernel module package and define a completly new "upgrade/coinstall" mechanism with is definitely not any more rpm CLI compatible. And losing that only because kernel modules packages w/ uname-r-in-name look ugly isn't worth it.
On Wednesday 19 July 2006 18:37, Axel Thimm wrote:
I wished I had been involved at that time to argue against this unfortunate change, but probably it was during the "cold war" where most people including myself had run out of batteries and could not try to save the world anymore :)
Anyway it's never too late, I'm optimistic about the new world order.
This becomes important as kernel ABI starts working, and you can build a module for a given kernel ABI and have it work on any new (or old) kernel that maintains the same ABI. No longer will modules be locked to a specific kernel version, and thus uname -r becomes irrelevant. I do believe Jon Masters is giving a talk on this at OLS.
Axel Thimm schrieb:
On Wed, Jul 19, 2006 at 07:40:38PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work.
Well, yes, coinstall will fail because this would result in a file conflict. But rpm -U works fine (but removes the older version) with the current Extras kmod scheme and "yum install foo" AFAICS works fine, too.
No, rpm -U would remove both kernel modules from both kernels and only install the selected kernel module, you end up with one kernel losing the kernel module w/o replacement.
rpm -U will always leave only one kernel module package behind unless these packages are disambiguated in their name by extending the name to contain the kernel's uname -r.
I don't want to reply to the other aspects of this mail -- I don't think it makes to much sense now and prefer to wait for the docs from Axel. But it seems we talked pass each other in above section so I'll try to give an example to show the behavior with the current Extras scheme (note this is not tested, only written down how it works afaik):
$ rpm -q kernel kernel-2.6.17-1.2145_FC5 kernel-2.6.17-1.2157_FC5 kernel-smp-2.6.17-1.2157_FC5 $ rpm -qa kmod-ntfs* kmod-ntfs-2.1.27-1.2.6.17_1.2145_FC5 kmod-ntfs-2.1.27-1.2.6.17_1.2157_FC5 kmod-ntfs-smp-2.1.27-1.2.6.17_1.2157_FC5
kmod-ntfs{,-smp,xen0,...}-2.1.27-2.2.6.17_1.2157_FC5 show up in the repo and user runs yum update. After that:
$ rpm -qa kmod-ntfs* kmod-ntfs-2.1.27-2.2.6.17_1.2157_FC5 kmod-ntfs-smp-2.1.27-2.2.6.17_1.2157_FC5
E.g. yes, kmod-ntfs-2.1.27-1.2.6.17_1.2145_FC5 got removed. No, both the up and the smp-kernel still have their modules.
CU thl
On Thu, Jul 20, 2006 at 05:58:02PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Wed, Jul 19, 2006 at 07:40:38PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work.
Well, yes, coinstall will fail because this would result in a file conflict. But rpm -U works fine (but removes the older version) with the current Extras kmod scheme and "yum install foo" AFAICS works fine, too.
No, rpm -U would remove both kernel modules from both kernels and only install the selected kernel module, you end up with one kernel losing the kernel module w/o replacement.
rpm -U will always leave only one kernel module package behind unless these packages are disambiguated in their name by extending the name to contain the kernel's uname -r.
I don't want to reply to the other aspects of this mail -- I don't think it makes to much sense now and prefer to wait for the docs from Axel. But it seems we talked pass each other in above section so I'll try to give an example to show the behavior with the current Extras scheme (note this is not tested, only written down how it works afaik):
No, I don't think we talked past each other, you are giving a perfect example outlining the design bug of any non-uname-r-in-name scheme.
$ rpm -q kernel kernel-2.6.17-1.2145_FC5 kernel-2.6.17-1.2157_FC5 kernel-smp-2.6.17-1.2157_FC5 $ rpm -qa kmod-ntfs* kmod-ntfs-2.1.27-1.2.6.17_1.2145_FC5 kmod-ntfs-2.1.27-1.2.6.17_1.2157_FC5 kmod-ntfs-smp-2.1.27-1.2.6.17_1.2157_FC5
kmod-ntfs{,-smp,xen0,...}-2.1.27-2.2.6.17_1.2157_FC5 show up in the repo and user runs yum update. After that:
$ rpm -qa kmod-ntfs* kmod-ntfs-2.1.27-2.2.6.17_1.2157_FC5 kmod-ntfs-smp-2.1.27-2.2.6.17_1.2157_FC5
E.g. yes, kmod-ntfs-2.1.27-1.2.6.17_1.2145_FC5 got removed.
Which is the bug in the scheme. You don't want to only support the rpm-latest kernel, just as you want to give freedom to the users to have multiple kernel rpms installed (of different versions) as it too often happens that the new kernel may not work on their systems.
E.g. every policy you follow for the kernel rpms themselves needs to be followed for the per-kernel installed kernel modules. So the kernel module of the "old" kernels need to stay untouched, by rpm and any higher-level packaging manager/depsolver.
Ignoring the bug on rpm CLI level and trying to fix all depsolvers out there is IMO wrong. rpm CLI needs to do a reasonable operation when presented these kernel module packages, not "accidentally" nuke away parts for the working previous (and perhaps already running) kernel.
The next horror scenario is to decide to keep this scheme and start working around it in each depsolver leaving some out like for example the increasing in popularity smart depsolver. smart upgrade would then happily nuke your display driver kernel module of the running kernel! Same for apt and up2date.
In fact the above is not to be layed in the future, but it's the current situation w/ for example livna, or not? Aren't there any livna reports that they lost their nvidia driver for old kernels while updateing their system?
No, both the up and the smp-kernel still have their modules.
And the reason is kernel flavour disambiguation through the name. Add the kernel's version to the name and you'll fix the bug mentioned above. And suddenly it's a uname-r-in-name scheme again and very close to kmdl systems (so let's pick that design and be all happy).
So, do we at least agree that the current kernel modules scheme breaks on rpm CLI level? Neither rpm -i, nor rpm -U could get you out of the above (typical!) situation.
Axel Thimm schrieb:
On Thu, Jul 20, 2006 at 05:58:02PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Wed, Jul 19, 2006 at 07:40:38PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work.
Well, yes, coinstall will fail because this would result in a file conflict. But rpm -U works fine (but removes the older version) with the current Extras kmod scheme and "yum install foo" AFAICS works fine, too.
No, rpm -U would remove both kernel modules from both kernels and only install the selected kernel module, you end up with one kernel losing the kernel module w/o replacement. rpm -U will always leave only one kernel module package behind unless these packages are disambiguated in their name by extending the name to contain the kernel's uname -r.
I don't want to reply to the other aspects of this mail -- I don't think it makes to much sense now and prefer to wait for the docs from Axel. But it seems we talked pass each other in above section so I'll try to give an example to show the behavior with the current Extras scheme (note this is not tested, only written down how it works afaik):
No, I don't think we talked past each other, you are giving a perfect example outlining the design bug of any non-uname-r-in-name scheme.
Well, you said "Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) [...]e.g. at least one kernel stay w/o any kmdl[...]" above -- and xen0 and non-xen0 work fine with the scheme, both have a module.
$ rpm -q kernel kernel-2.6.17-1.2145_FC5 kernel-2.6.17-1.2157_FC5 kernel-smp-2.6.17-1.2157_FC5 $ rpm -qa kmod-ntfs* kmod-ntfs-2.1.27-1.2.6.17_1.2145_FC5 kmod-ntfs-2.1.27-1.2.6.17_1.2157_FC5 kmod-ntfs-smp-2.1.27-1.2.6.17_1.2157_FC5
kmod-ntfs{,-smp,xen0,...}-2.1.27-2.2.6.17_1.2157_FC5 show up in the repo and user runs yum update. After that:
$ rpm -qa kmod-ntfs* kmod-ntfs-2.1.27-2.2.6.17_1.2157_FC5 kmod-ntfs-smp-2.1.27-2.2.6.17_1.2157_FC5
E.g. yes, kmod-ntfs-2.1.27-1.2.6.17_1.2145_FC5 got removed.
Which is the bug in the scheme.
Well, seems to be a bug for you. Okay, noted. I'd consider is as an minor annoyance. And it seems Jack Neely is working on getting it fixed.
You don't want to only support the rpm-latest kernel,
To support only the latest kernel was a decided early in the kmod discussion. That might have influenced the kmod standard.
BTW, when we initially discussed the kmod standard there was also (IIRC) a strong resistance from multiple important and well known people at redhat against overloading %{NAME} with the output of "uname -r". I doubt that option changed. Spot? Jeremy? f13?
[...]
In fact the above is not to be layed in the future, but it's the current situation w/ for example livna, or not? Aren't there any livna reports that they lost their nvidia driver for old kernels while updateing their system?
Nobody reported a bug yet. And the nvidia driver is a good example: we even have explicit "Obsoletes" in the spec to make sure old modules always get removed because nvidia-1.2.3 only works with kmod-nvidia-1.2.3, but not with kmod-nvidia-1.2.2
No, both the up and the smp-kernel still have their modules.
And the reason is kernel flavour disambiguation through the name. Add the kernel's version to the name and you'll fix the bug mentioned above. And suddenly it's a uname-r-in-name scheme again and very close to kmdl systems (so let's pick that design and be all happy).
And it also needs special support in depsolvers so you get all kernel-modules together with a kernel update.
And you need to maintain a lot of obsoletes to get old kmods removed that are incompatible with the updated userland packages. Or you need to build kmods for each and every kernel that ever shipped -- that would take to long (and they are often not available anymore).
So, do we at least agree that the current kernel modules scheme breaks on rpm CLI level?
No.
Neither rpm -i, nor rpm -U could get you out of the above (typical!) situation.
I don't consider it a big problem.
CU thl
On Fri, 2006-07-21 at 17:51 +0200, Thorsten Leemhuis wrote:
BTW, when we initially discussed the kmod standard there was also (IIRC) a strong resistance from multiple important and well known people at redhat against overloading %{NAME} with the output of "uname -r". I doubt that option changed. Spot? Jeremy? f13?
Yep. I'm still very strongly against overloading %{NAME}. We don't permit any other package to do this for any reason, and I don't want to start now. Name is not for versioning.
~spot
On Fri, Jul 21, 2006 at 12:06:39PM -0500, Tom 'spot' Callaway wrote:
On Fri, 2006-07-21 at 17:51 +0200, Thorsten Leemhuis wrote:
BTW, when we initially discussed the kmod standard there was also (IIRC) a strong resistance from multiple important and well known people at redhat against overloading %{NAME} with the output of "uname -r". I doubt that option changed. Spot? Jeremy? f13?
Yep. I'm still very strongly against overloading %{NAME}. We don't permit any other package to do this for any reason, and I don't want to start now. Name is not for versioning.
Well, it's good to know as that kills any review attempt of the current scheme.
On Fri, Jul 21, 2006 at 07:57:46PM +0200, Axel Thimm wrote:
On Fri, Jul 21, 2006 at 12:06:39PM -0500, Tom 'spot' Callaway wrote:
On Fri, 2006-07-21 at 17:51 +0200, Thorsten Leemhuis wrote:
BTW, when we initially discussed the kmod standard there was also (IIRC) a strong resistance from multiple important and well known people at redhat against overloading %{NAME} with the output of "uname -r". I doubt that option changed. Spot? Jeremy? f13?
Yep. I'm still very strongly against overloading %{NAME}. We don't permit any other package to do this for any reason, and I don't want to start now. Name is not for versioning.
Well, it's good to know as that kills any review attempt of the current scheme.
Does it make sense to gather all people against a uname -r in name scheme to try to persuade otherwise? Either in the regular #fedora-packaging meetings or on a special meeting?
I really thing there is a flaw and the uname-r-in-name is the only way out, and I'd try to persuade people about that. Maybe they could be poited to this mail thread as well, as everything is in principle layed out here.
Or, if you know it's a lost cause, let's silence this topic, which would be a rather sad outcome.
On Friday 21 July 2006 14:35, Axel Thimm wrote:
I really thing there is a flaw and the uname-r-in-name is the only way out, and I'd try to persuade people about that. Maybe they could be poited to this mail thread as well, as everything is in principle layed out here.
With the kabi stuff coming along, there is no longer a need to lock a given module to a given kernel version, just an ABI version. The kernel could easily be bumped a version but the ABI that a particular module needs would remain unchanged, and thus the module will continue to work. Locking to a uname will be pointless at this point.
Jon Masters is giving (gave?) a talk about this at OLS and was discussing these things at the kernel summit I do believe. I strongly feel we wait for him to return from OLS so that we can include him in the discussion surrounding packaging of external kernel modules.
As it stands, the development kernel does automatically provide an ABI checksum for each module subdirectory, and rpm knows about it. Requires on an ABI supposedly works today.
On Sat, Jul 22, 2006 at 09:45:51AM -0400, Jesse Keating wrote:
On Friday 21 July 2006 14:35, Axel Thimm wrote:
I really thing there is a flaw and the uname-r-in-name is the only way out, and I'd try to persuade people about that. Maybe they could be poited to this mail thread as well, as everything is in principle layed out here.
With the kabi stuff coming along, there is no longer a need to lock a given module to a given kernel version, just an ABI version. The kernel could easily be bumped a version but the ABI that a particular module needs would remain unchanged, and thus the module will continue to work. Locking to a uname will be pointless at this point.
Jon Masters is giving (gave?) a talk about this at OLS and was discussing these things at the kernel summit I do believe. I strongly feel we wait for him to return from OLS so that we can include him in the discussion surrounding packaging of external kernel modules.
As it stands, the development kernel does automatically provide an ABI checksum for each module subdirectory, and rpm knows about it. Requires on an ABI supposedly works today.
kABI will not really help, as it only measures what has changed in the ABI from on kernel release to the next, checking to see whether an old kernel module can be safely recycled. It will not magically force kernel developers to introduce a stable ABI, function signatures and other symbols will change just as frequent.
And the areas where kABI would help is where the kernel has reached some level of maturity where indeed the ABI has become stable. But these are not the typical subsystems external kernel modules are built for.
Currenlty the most frequent cases of kernel modules are such usually requiring v4l2, ieee82011 or vm subsystems. And these are currently guaranteed to change from kernel release to kernel release. And once these stabilize and other areas of the kernel become interesting you'll have the same situation there. Currently (the last 1-2 years) every kernel release breaks 70-80% of external kernel modules at build level already, and kABI would only confirm this.
So what is really needed is a kernel developers' committment to stabilize the kernel ABI, and we are really far away from such a point in time. And even if that were tomorrow, we need to consider legacy support for a couple of FC releases and also RHEL for a couple of years more.
But kABI is definitely the way towards a stable kernel ABI, just not within the timeframe we are interested in, e.g. <= 2 years.
So for the current discussion this won't help us further.
On Sunday 23 July 2006 11:57, Axel Thimm wrote:
kABI will not really help, as it only measures what has changed in the ABI from on kernel release to the next, checking to see whether an old kernel module can be safely recycled. It will not magically force kernel developers to introduce a stable ABI, function signatures and other symbols will change just as frequent.
And the areas where kABI would help is where the kernel has reached some level of maturity where indeed the ABI has become stable. But these are not the typical subsystems external kernel modules are built for.
Currenlty the most frequent cases of kernel modules are such usually requiring v4l2, ieee82011 or vm subsystems. And these are currently guaranteed to change from kernel release to kernel release. And once these stabilize and other areas of the kernel become interesting you'll have the same situation there. Currently (the last 1-2 years) every kernel release breaks 70-80% of external kernel modules at build level already, and kABI would only confirm this.
There are kernel updates for new rebase, there are kernel updates for security, there are kernel updates for specific bug fixes. There are a lot of cases where the ABI would not change for particular drivers. SCSI, Video, yes even wireless. Any naming scheme that will be discussed should take the KABI system into account and use that. Even if the ABI changes just as frequently as kernel version we should still use the ABI so that the same naming and packaging scheme will work across Fedora Core current releases, maint releases (Legacy), and RHEL (and rebuilds) releases.
On Sun, Jul 23, 2006 at 12:12:58PM -0400, Jesse Keating wrote:
On Sunday 23 July 2006 11:57, Axel Thimm wrote:
kABI will not really help, as it only measures what has changed in the ABI from on kernel release to the next, checking to see whether an old kernel module can be safely recycled. It will not magically force kernel developers to introduce a stable ABI, function signatures and other symbols will change just as frequent.
And the areas where kABI would help is where the kernel has reached some level of maturity where indeed the ABI has become stable. But these are not the typical subsystems external kernel modules are built for.
Currenlty the most frequent cases of kernel modules are such usually requiring v4l2, ieee82011 or vm subsystems. And these are currently guaranteed to change from kernel release to kernel release. And once these stabilize and other areas of the kernel become interesting you'll have the same situation there. Currently (the last 1-2 years) every kernel release breaks 70-80% of external kernel modules at build level already, and kABI would only confirm this.
There are kernel updates for new rebase, there are kernel updates for security, there are kernel updates for specific bug fixes. There are a lot of cases where the ABI would not change for particular drivers. SCSI, Video, yes even wireless. Any naming scheme that will be discussed should take the KABI system into account and use that. Even if the ABI changes just as frequently as kernel version we should still use the ABI so that the same naming and packaging scheme will work across Fedora Core current releases, maint releases (Legacy), and RHEL (and rebuilds) releases.
Well, add to the above that the kABI isn't going to give you an orderable single entry like uname-r does (but maybe noone cares, the kernel module packaging at least wouldn't), and that no user will understand the mapping between his kernel, whose uname -r he knows, and a kABI checksum.
But in principle if one day kABI checksums gain a popularity/visibilty like uname-r has today on the user's side, then I agree, that uname-r in the name could be replaced with a kABI checksum. In the kmdl scheme this would be a rather trivial change.
But you're not going to make friends with people already losing lunch on embedding uname-r in the name. :)
On Sunday 23 July 2006 14:12, Axel Thimm wrote:
Well, add to the above that the kABI isn't going to give you an orderable single entry like uname-r does (but maybe noone cares, the kernel module packaging at least wouldn't), and that no user will understand the mapping between his kernel, whose uname -r he knows, and a kABI checksum.
But in principle if one day kABI checksums gain a popularity/visibilty like uname-r has today on the user's side, then I agree, that uname-r in the name could be replaced with a kABI checksum. In the kmdl scheme this would be a rather trivial change.
Perhaps I fail to see the problem. Once you have an ABI to use for requires and such, can't you use someting more simple in the version or release rather than a uname-r in the name?
On Sun, Jul 23, 2006 at 02:20:13PM -0400, Jesse Keating wrote:
On Sunday 23 July 2006 14:12, Axel Thimm wrote:
Well, add to the above that the kABI isn't going to give you an orderable single entry like uname-r does (but maybe noone cares, the kernel module packaging at least wouldn't), and that no user will understand the mapping between his kernel, whose uname -r he knows, and a kABI checksum.
But in principle if one day kABI checksums gain a popularity/visibilty like uname-r has today on the user's side, then I agree, that uname-r in the name could be replaced with a kABI checksum. In the kmdl scheme this would be a rather trivial change.
Perhaps I fail to see the problem. Once you have an ABI to use for requires and such, can't you use someting more simple in the version or release rather than a uname-r in the name?
The kABI is a set of headers out of which you can generate a checksum. The checksum will by definition not be orderable, and also not memorizable. Similar to mercurial changeset ids.
On Sunday 23 July 2006 14:24, Axel Thimm wrote:
The kABI is a set of headers out of which you can generate a checksum. The checksum will by definition not be orderable, and also not memorizable. Similar to mercurial changeset ids.
Right, I understand that.
I'm far too late into the conversation I think. I'll wait for a complete proposal to come through.
On Fri, 21 Jul 2006, Tom 'spot' Callaway wrote:
On Fri, 2006-07-21 at 17:51 +0200, Thorsten Leemhuis wrote:
BTW, when we initially discussed the kmod standard there was also (IIRC) a strong resistance from multiple important and well known people at redhat against overloading %{NAME} with the output of "uname -r". I doubt that option changed. Spot? Jeremy? f13?
Yep. I'm still very strongly against overloading %{NAME}. We don't permit any other package to do this for any reason, and I don't want to start now. Name is not for versioning.
The name is used for versioning in several other packages for similar reasons (to *sanely* allow more than one version of the package simultaneously installed and updated), for example:
[pmatilai@cs181077098 ~]$ repoquery 'openssl*' openssl-perl-0:0.9.8a-5.2.i386 openssl097a-0:0.9.7a-4.2.1.i386 openssl-0:0.9.8a-5.2.i686 openssl-devel-0:0.9.8a-5.2.i386 openssl-0:0.9.8a-5.2.i386 [pmatilai@cs181077098 ~]$ repoquery 'libpng*' libpng-devel-2:1.2.8-2.2.1.i386 libpng10-0:1.0.18-3.2.1.i386 libpng-2:1.2.8-2.2.1.i386 libpng10-devel-0:1.0.18-3.2.1.i386
- Panu -
On Sat, 2006-07-22 at 23:39 -0700, Panu Matilainen wrote:
The name is used for versioning in several other packages for similar reasons (to *sanely* allow more than one version of the package simultaneously installed and updated), for example:
[pmatilai@cs181077098 ~]$ repoquery 'openssl*' openssl-perl-0:0.9.8a-5.2.i386 openssl097a-0:0.9.7a-4.2.1.i386 openssl-0:0.9.8a-5.2.i686 openssl-devel-0:0.9.8a-5.2.i386 openssl-0:0.9.8a-5.2.i386 [pmatilai@cs181077098 ~]$ repoquery 'libpng*' libpng-devel-2:1.2.8-2.2.1.i386 libpng10-0:1.0.18-3.2.1.i386 libpng-2:1.2.8-2.2.1.i386 libpng10-devel-0:1.0.18-3.2.1.i386
And I personally think those are abuses of %{NAME}. I'd much rather see the compat-* ideology used there instead of overloading Name.
Let me put it this way:
Unless someone can show me a solid write-up that shows that there is _NO_ way to handle kernel modules without overloading Name, I'm going to oppose it on principle.
~spot
On Sun, Jul 23, 2006 at 08:42:52AM -0500, Tom 'spot' Callaway wrote:
On Sat, 2006-07-22 at 23:39 -0700, Panu Matilainen wrote:
The name is used for versioning in several other packages for similar reasons (to *sanely* allow more than one version of the package simultaneously installed and updated), for example:
[pmatilai@cs181077098 ~]$ repoquery 'openssl*' openssl-perl-0:0.9.8a-5.2.i386 openssl097a-0:0.9.7a-4.2.1.i386 openssl-0:0.9.8a-5.2.i686 openssl-devel-0:0.9.8a-5.2.i386 openssl-0:0.9.8a-5.2.i386 [pmatilai@cs181077098 ~]$ repoquery 'libpng*' libpng-devel-2:1.2.8-2.2.1.i386 libpng10-0:1.0.18-3.2.1.i386 libpng-2:1.2.8-2.2.1.i386 libpng10-devel-0:1.0.18-3.2.1.i386
And I personally think those are abuses of %{NAME}. I'd much rather see the compat-* ideology used there instead of overloading Name.
compat-* is overloading the name just the same.
There are dozens of further examples, gcc<XYZ>, autoconf<XYZ>, automake<XYZ>, gtk2 (or let's say g*2), libstdc++so<XYZ>, mysqlclient<XYZ> and so on.
A quick check on current rawhide shows that at least 5% of the src.rpms are overloading the name.
I really had to fight with myself long before I accepted that there is no other way that uname-r-in-the-name. It hurts my eyes just as it does anyone's else, but it proves to be a neccessity.
Let me put it this way:
Unless someone can show me a solid write-up that shows that there is _NO_ way to handle kernel modules without overloading Name, I'm going to oppose it on principle.
It's in the posts by Thorsten and myself including an example even, the summary is that
There is no way to handle kernel modules for several kernels w/o using a uname-r disambiguation in the name on rpm level.
That's a fact. You either get to
a) ban usage of rpm CLI and patch all depsolvers to follow non-rpm ordering
b) stop supporting more than the very latest published kernel (e.g. allowing the current kernel to nuke/overwrite its own kernel modules while upgrading/installing a new kernel)
c) accept uname -r in the name
There is disagreement about how bad a) is. I consider it a blocker, Thorsten can live with it. So currently we are effectively living in a), only that noone knows that rpm CLI usage is disallowed with kernel module packages.
c) is the only technical sensible solution bringing us at 90% of the target with only one drawback: Ugly names. So what, technical aesthetics superseed what meets the eye. :)
On Sun, 2006-07-23 at 17:45 +0200, Axel Thimm wrote:
c) is the only technical sensible solution bringing us at 90% of the target with only one drawback: Ugly names. So what, technical aesthetics superseed what meets the eye. :)
It breaks bugzilla for starters. We deal with this for things like openssl and gcc because they're minimal. Maybe one or two other than the primary. With kmod, we'd be looking at a LOT more than one or two.
No other package puts the version for something it depends on in its Name.
So, what it boils down to is: rpm is not well built to handle the kernel module packaging case.
Can we fix rpm? I doubt it. Everytime I point out a weakness in rpm, I get told that its not going to be fixed. Since upstream is actively hostile towards us, we certainly can't expect them to help.
With all of that said:
I'll defer to Thorsten and Axel on this one, since they've been knee deep in this. If BOTH of you agree that the _ONLY_ way to have sane kernel module packages (without making rpm changes) is to overload Name, then I'll withdraw my objection to it. (I know Axel feels that way, do you Thorsten?)
~spot
Tom 'spot' Callaway schrieb:
[...] I'll defer to Thorsten and Axel on this one, since they've been knee deep in this.
scop should be asked, too. He's probably knee deep (or even deeper) into this, too.
If BOTH of you agree that the _ONLY_ way to have sane kernel module packages (without making rpm changes) is to overload Name, then I'll withdraw my objection to it. (I know Axel feels that way, do you Thorsten?)
Well, I stick to my opinion that "uname -r" in Name creates some problems on its own and not worth the trouble.
CU thl
On Sun, Jul 23, 2006 at 07:31:46PM +0200, Thorsten Leemhuis wrote:
Tom 'spot' Callaway schrieb:
[...] I'll defer to Thorsten and Axel on this one, since they've been knee deep in this.
scop should be asked, too. He's probably knee deep (or even deeper) into this, too.
If BOTH of you agree that the _ONLY_ way to have sane kernel module packages (without making rpm changes) is to overload Name, then I'll withdraw my objection to it. (I know Axel feels that way, do you Thorsten?)
Well, I stick to my opinion that "uname -r" in Name creates some problems on its own and not worth the trouble.
But please be as fair as to admit that w/o uname-r in name the problems are several magnitudes worse. rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
uname-r-in-name and the kmdl scheme isn't going to bring peace on earth, but it is already very close to the requirements on kernel module packages which no merging-versions-scheme can be.
On Sun, 2006-07-23 at 20:01 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 07:31:46PM +0200, Thorsten Leemhuis wrote:
Well, I stick to my opinion that "uname -r" in Name creates some problems on its own and not worth the trouble.
But please be as fair as to admit that w/o uname-r in name the problems are several magnitudes worse.
Having been involved in designing, using, and maintaining schemes and packages both with and without uname-in-name for years and discussing the pros and cons to death several times, my opinion is that the problems created by both are roughly equal, and certainly not different by order of magnitude.
The result of the last discussion round and yet another redesign of the kmod guidelines from scratch [0] has been accepted by a quite a few interested parties, reaching apparent critical mass so that it's actually possible to ship additional kernel modules in *some* usable and maintainable form in FE. I don't personally agree with every bit in the current design, but I see nothing critically wrong with it either. In particular, I see no need to challenge the achieved consensus with the controversial scheme change discussed in this thread, so -1.
rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
rpm -U behaves just as documented and just like with all other packages, including the kernels, ie. upgrades them. Yes, I'm aware of the nuances that might make some say it's not the same. Whatever, if you don't want that behaviour, don't use -U. kernel packages don't have uname-r-in-name either, and people are perfectly capable of upgrading their kernels with the rpm CLI.
Ditto, rpm -i behaves like for all other packages, it doesn't nuke or overwrite anything. Use --oldpackage in addition if you wish to deal with modules for old kernels.
Sure, it's a bit difficult but not impossible, just use -e and -i, to "safely" upgrade a kmod package for an old installed kernel. But I think shipping updated modules for old kernels is just not going to (and one could argue should not) happen in FC/FE anyway. Either way, I'm certainly not losing any sleep over that.
[0] Like I've said before, I'm not going to participate in more discussions about this unless specifically asked. I think I was asked in this thread, so here goes, but don't expect further replies.
On Sun, Jul 23, 2006 at 10:32:06PM +0300, Ville Skyttä wrote:
rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
rpm -U behaves just as documented and just like with all other packages, including the kernels, ie. upgrades them. Yes, I'm aware of the nuances that might make some say it's not the same. Whatever, if you don't want that behaviour, don't use -U. kernel packages don't have uname-r-in-name either, and people are perfectly capable of upgrading their kernels with the rpm CLI.
Ditto, rpm -i behaves like for all other packages, it doesn't nuke or overwrite anything. Use --oldpackage in addition if you wish to deal with modules for old kernels.
Just pick Thorsten's example where rpm -U will nuke the kernel module from another unrelated kernel and rpm -i will overwrite (coinstall over) the kernel module of the latest kernel.
To give the example again (for simplicity EVR is a single integer):
kernel-a with module foo-1 has kmod-foo-1-a kernel-b with module foo-1 has kmod-foo-1-b
(BTW this by itself already needs special support in all depsolvers to allow for multiple installs. smart for one cannot glob this, so you need to mention each kernel module in its config.)
Now an update module foo-2 brings in kmod-foo-2-b in the repo.
This *should* replace kmod-foo-1-b, but leave kmod-foo-1-a alone.
o rpm -i fails as it simply (partly) *overwrites* kmod-foo-1-b o rpm -U faile as it *nukes* kmod-foo-1-a
This will be the typical situation upon each kernel module upgrade. And w/o special handling in *each* depsolver depsolvers will automagically fail defaulting to nuking.
So, the fact remains: W/o uname-r-in-name you generate a very messy situation with compilcated special handling to avoid breakage and you cannot avoid breakage on rpm CLI level.
In contrast the kmdl scheme guarantees that old kernel modules will not be nuked and neither will anything be overwritten - in fact on rpm CLI level kmdls behave as a normal package - always use rpm -U. The only drawback of kmdl is that depsolvers don't coinstall for newly installed kernels. But this depsolver support is far more uncritical if missing and also easier to implement (like a 9-liner in bash).
On Sun, 2006-07-23 at 22:19 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 10:32:06PM +0300, Ville Skyttä wrote:
rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
rpm -U behaves just as documented and just like with all other packages, including the kernels, ie. upgrades them. Yes, I'm aware of the nuances that might make some say it's not the same. Whatever, if you don't want that behaviour, don't use -U. kernel packages don't have uname-r-in-name either, and people are perfectly capable of upgrading their kernels with the rpm CLI.
Ditto, rpm -i behaves like for all other packages, it doesn't nuke or overwrite anything. Use --oldpackage in addition if you wish to deal with modules for old kernels.
Just pick Thorsten's example where rpm -U will nuke the kernel module from another unrelated kernel and rpm -i will overwrite (coinstall over) the kernel module of the latest kernel.
But I thought it was already stated that we don't care about rpm used on the cli to handle these sorts of things. That we've assumed we're operating at a level above rpm for constructing the transaction set.
So if you think of rpm's direct use as not a concern what are the other issues with the current scheme?
-sv
On Sun, Jul 23, 2006 at 04:38:59PM -0400, seth vidal wrote:
On Sun, 2006-07-23 at 22:19 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 10:32:06PM +0300, Ville Skyttä wrote:
rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
But I thought it was already stated that we don't care about rpm used on the cli to handle these sorts of things. That we've assumed we're operating at a level above rpm for constructing the transaction set.
A scheme were two independent EVRs are merged together into one is broken and banning the tool that exhibits it and pasting it away from others isn't the solution. kmod-foo-1-a just has nothing to do with kmod-foo-<anything>-b (in the simplified notation).
On Sun, 2006-07-23 at 23:03 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 04:38:59PM -0400, seth vidal wrote:
On Sun, 2006-07-23 at 22:19 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 10:32:06PM +0300, Ville Skyttä wrote:
rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
But I thought it was already stated that we don't care about rpm used on the cli to handle these sorts of things. That we've assumed we're operating at a level above rpm for constructing the transaction set.
A scheme were two independent EVRs are merged together into one is broken and banning the tool that exhibits it and pasting it away from others isn't the solution. kmod-foo-1-a just has nothing to do with kmod-foo-<anything>-b (in the simplified notation).
okay. I can take that.
Here's my only complaint and it is a complaint for all sides equally:
I would like one and final standard decided on BEFORE FC6 is released. We need to get the code for yum complete so we can stop messing around with this stuff.
At this point I'm in favor of whatever gets it done.
-sv
On Sun, Jul 23, 2006 at 05:13:40PM -0400, seth vidal wrote:
On Sun, 2006-07-23 at 23:03 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 04:38:59PM -0400, seth vidal wrote:
On Sun, 2006-07-23 at 22:19 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 10:32:06PM +0300, Ville Skyttä wrote:
rpm -U/-i will nuke or overwrite kernel modules of the running kernel in a uname-r-less scheme.
But I thought it was already stated that we don't care about rpm used on the cli to handle these sorts of things. That we've assumed we're operating at a level above rpm for constructing the transaction set.
A scheme were two independent EVRs are merged together into one is broken and banning the tool that exhibits it and pasting it away from others isn't the solution. kmod-foo-1-a just has nothing to do with kmod-foo-<anything>-b (in the simplified notation).
okay. I can take that.
Here's my only complaint and it is a complaint for all sides equally:
I would like one and final standard decided on BEFORE FC6 is released. We need to get the code for yum complete so we can stop messing around with this stuff.
At this point I'm in favor of whatever gets it done.
Would you be interested in adding support for the kmdl scheme to yum (or a plugin) *before* a decision is made?
I'm suggesting this because kernel module handling is for many in this list too abstract to discuss on paper and a live demonstration using yum and kmdls would certainly help moving on on this subject here. A bash shell script doing it is 9 lines, so I hope for you it would be almost trivial to pluginize.
Even if the decision can't be made in time for FC6 (which effectively would mean to keep the current scheme) it would not be a waste of time as it is already in use at ATrpms, and many users want to see yum auto-coinstalling them upon new kernels.
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
-Toshio
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
-Toshio
On Tue, Jul 25, 2006 at 03:00:35PM -0700, Toshio Kuratomi wrote:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
The issue is with the package's name and evr, not its contents.
And there are no "normal" operations possible unless you use uname-r-in-name:
o non-kernel/non-kernel-module packages: Always UPGRADE (e.g. remove the old packages): rpm -U
o kernel packages: (co)INSTALL, don't remove the current kernel, possibly also not a couple more kernels: rpm -i [1]
o kernel-module packages: Do replace kernel modules for the same kernel, but don't touch kernel modules of other kernels: So it's a mixture of simultaneous UPGRADE/INSTALL operations.
The latter is also the problem. Given this two-dimensional evr-space where verticals and horizontals are to be handled with upgrades and (co)install operations you get into trouble with trying to project the space onto one dimension. You get the different operations needed mangled.
The only way out is to defuse one dimension, e.g. remove it from rpm's line of sight, and that is the kernel's version. Moving the uname-r into the package's name makes the packages behave properly both within one kernel and across different kernels with rpm -U, e.g. kmdls are to be handled like typical packages, for example (omitting releases, only versions for simplicity):
rpm -Uhv foo-kmdl-2.6.20-2.i686.rpm
will install kernel module foo in version 2 for kernel 2.6.20 w/o affecting kernel modules for kernels 2.6.19 or 2.6.21, but still replacing possibly previous versions of foo (say version 1) for kernel 2.6.20.
That's exactly the desired functionality on rpm CLI level already!
And all depsolvers already handle the part with upgrading the kmdls within the kernel. What they need to be tought to make the picture perfect is to install matching kmdls upon kernel upgrades.
[1] With yum's installn it's not quite true, there is a delayed rpm -e lingering around waiting for enough kernels to gather to remove the oldest.
Toshio Kuratomi schrieb:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
I like that idea -- especially when combined with the the kabi stuff. Yes, someone still could run "rpm -Uvh" and would loose older kmods, but yum and apt would do the right thing.
CU thl
On Sun, Jul 30, 2006 at 06:04:34PM +0200, Thorsten Leemhuis wrote:
Toshio Kuratomi schrieb:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
I like that idea -- especially when combined with the the kabi stuff. Yes, someone still could run "rpm -Uvh" and would loose older kmods, but yum and apt would do the right thing.
Why would suddenly yum/apt work better? The issues brought up with the current kernel module scheme didn't address the actual contents of the packages and Toshio's suggestion only discusses the contents.
So any raised issue on the naming/versioning of kernel module packages still remains as discussed - including non-rpm conformant behaviour of such packages with the rpm -U/-i nuke/conflict situation, which is still not "fixed" on yum level or any other depsolver's (and having looked into yum's plugin system which is quite nice BTW, I don't know whether it will be "fixable" at all)
"Fixing yum or the yum plugin" is in quotes, because it is not yum that has any deficiencies, but the current kernel packaging scheme.
The kmdl scheme works with yum already even w/o a plugin and a plugin under 100 lines enables coinstalls for new kernel packages.
Why would suddenly yum/apt work better? The issues brought up with the current kernel module scheme didn't address the actual contents of the packages and Toshio's suggestion only discusses the contents.
So any raised issue on the naming/versioning of kernel module packages still remains as discussed - including non-rpm conformant behaviour of such packages with the rpm -U/-i nuke/conflict situation, which is still not "fixed" on yum level or any other depsolver's (and having looked into yum's plugin system which is quite nice BTW, I don't know whether it will be "fixable" at all)
If the existing fedorakmod.py plugin in yum-utils does not handle installs, upgrades, and co-installs of kmod style kernel modules then I'd like a detailed bug report please.
Jack
"Fixing yum or the yum plugin" is in quotes, because it is not yum that has any deficiencies, but the current kernel packaging scheme.
The kmdl scheme works with yum already even w/o a plugin and a plugin under 100 lines enables coinstalls for new kernel packages. -- Axel.Thimm at ATrpms.net
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
On Mon, Aug 07, 2006 at 10:00:13AM -0400, Jack Neely wrote:
So any raised issue on the naming/versioning of kernel module packages still remains as discussed - including non-rpm conformant behaviour of such packages with the rpm -U/-i nuke/conflict situation, which is still not "fixed" on yum level or any other depsolver's (and having looked into yum's plugin system which is quite nice BTW, I don't know whether it will be "fixable" at all)
If the existing fedorakmod.py plugin in yum-utils does not handle installs, upgrades, and co-installs of kmod style kernel modules then I'd like a detailed bug report please.
AFAIU yum's plugin logic in order to perform according to the current rpm-level-broken kernel module scheme a plugin would have to undo parts of yum's transaction. E.g. yum does not offer a hook to manipulate its logic during the depsolving. Note that yum's depsolver logic is rpm conformant. Therefore the moment the fedorakmod.py takes over the transaction set may contain both nuking and conflicts which the plugin needs to address in retrospective now.
Coinstall support "solves" the first issue, but the second issue is only "solvable" by removing parts of the transaction. Unfortunately removing is an operation that needs to get back into the depsolver logic - unless the package to be removed has no further dependent or depending on packages. This may be true for most of the current few available kernel modules in this scheme, but there are others that have richer depedendency structure - most notably v4l2 and wifi related modules.
It doesn't look like the fedorakmod.py plugin takes that into account. It's still "fixable" by adding even more code, but it's already gaining code size and therefore maintenance costs for something that wasn't "broken" to begin with. This also shows that there may be even more bugs lurking, and this is only the support for yum, rpm still remains "broken" and so do all the other depsolvers.
It's nothing against Jack's work or even Thorsten's and Ville's work on the kernel module specs. It's just a core design element that proves to be bad and needs replacement asap.
I know I repeat myself, but:
There is the kmdl proposal which doesn't even need any special plugins to remain rpm-compliant for both rpm and all depsolvers and while support for coinstalls for new kernels is missing in all depsolvers it proved to be trivial to add a < 100 lines easy to maintain plugin to accomplish that. So there really is only the I-don't-like-uname-r- in-name issue left ...
On Mon, 2006-08-07 at 16:54 +0200, Axel Thimm wrote:
There is the kmdl proposal which doesn't even need any special plugins to remain rpm-compliant for both rpm and all depsolvers and while support for coinstalls for new kernels is missing in all depsolvers it proved to be trivial to add a < 100 lines easy to maintain plugin to accomplish that. So there really is only the I-don't-like-uname-r- in-name issue left ...
Axel, what are the differences between the "kmdl" proposal and the existing Fedora kernel module standard? Can you summarize for me?
~spot
On Mon, Aug 07, 2006 at 10:19:19AM -0500, Tom 'spot' Callaway wrote:
On Mon, 2006-08-07 at 16:54 +0200, Axel Thimm wrote:
There is the kmdl proposal which doesn't even need any special plugins to remain rpm-compliant for both rpm and all depsolvers and while support for coinstalls for new kernels is missing in all depsolvers it proved to be trivial to add a < 100 lines easy to maintain plugin to accomplish that. So there really is only the I-don't-like-uname-r- in-name issue left ...
Axel, what are the differences between the "kmdl" proposal and the existing Fedora kernel module standard? Can you summarize for me?
The main design differences are
o one src.rpm for both userland and kernel module subpackages. o full abstraction of names and embedded dependencies supporting among others a uname-r-in-name scheme.
The latter means that the kmdl macros used are flexible enough to even produce the current kernel module scheme. But they can also produce what is usually known as kmdl packages including the uname-r-in-name idiom and therefore remaining rpm conformant.
So kmdl is both
o an abstract interface hiding implementation details from the specfile o an explit implemenentation of these macros to fulfill rpm compliant versioning of kernel modules
This results in a flexible and powerful kernel module macro language and very small and maintainable forward-compatible specfiles (e.g. specfiles survive any furture implementation modification).
For a very small (but also very old) example see
http://dl.atrpms.net/all/arc4.spec
(This kmdl is used for *swan support on RHEL3 IIRC, e.g. the openswan kmdl depends on presence of the arc4 cipher. It was the smallest example I found, it has no userland parts but the placeholders are still there)
Axel Thimm schrieb:
On Mon, Aug 07, 2006 at 10:19:19AM -0500, Tom 'spot' Callaway wrote:
o one src.rpm for both userland and kernel module subpackages.
People disliked that when we created the current kmod standard because either you build the userland stuff for i586 and i686 or you have to add a lot of %if foo #build userland %endif %if bar #build module %endif into the spec file.
People also wanted proper debuginfo packages. This means that you have to create the debuginfo packages either manually or have to increase the release each time you build for a new kernel.
[...] For a very small (but also very old) example see
http://dl.atrpms.net/all/arc4.spec
From a quick glance:
-> no proper debuginfo packages -> has to be queued to the buildsys multiple time to build for UP, SMP, Xen, ... (it at least looks this way? Or am I wrong?)
Cu thl
On Mon, Aug 07, 2006 at 06:19:51PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Mon, Aug 07, 2006 at 10:19:19AM -0500, Tom 'spot' Callaway wrote:
o one src.rpm for both userland and kernel module subpackages.
People disliked that when we created the current kmod standard because either you build the userland stuff for i586 and i686 or you have to add a lot of %if foo #build userland %endif %if bar #build module %endif into the spec file.
It's better than to have two possibly divergent specs/src.rpms. Just imagine a cvs patch to apply to to both packages at once. You only have three %if/%else/%endif constructs in a common specfile saving quite a bit of redundant information and having a proper common changelog - maintenance and package overview is much better in a common specfile.
I think some of the design problems of the current kernel module scheme is due to allowing too much "external" pressure to get them through. I don't think/remember early drafts to have done this splitting of src.rpms just as I know the uname-r was removed due to external requests. The kernel module scheme was finally accepted, but at what price?
People also wanted proper debuginfo packages. This means that you have to create the debuginfo packages either manually or have to increase the release each time you build for a new kernel.
Proper debuginfo packages are produced at ATrpms - just not published due to lack of size.
[...] For a very small (but also very old) example see
http://dl.atrpms.net/all/arc4.specFrom a quick glance:
-> no proper debuginfo packages
Where do you see that in the specfile? FWIW debuginfos are no problem at all for kmdls.
-> has to be queued to the buildsys multiple time to build for UP, SMP, Xen, ... (it at least looks this way? Or am I wrong?)
Yes, and that's actually a *feature* I forgot to add to the design principles:
o kmdls are completely kernel flavour agnostic. There is no hardwiring up/smp/xen anywhere. Which is why the same specfile can be used unaltered for xenU/xen0/xen/PAE/kdump and whatever future kernel flavours may come up or even on RHEL flavours like hugemem. And it even works when the flavours change over the curse of time like "xen" suddenly appearing in FC5 alongside xen0/xenU.
The easy maintenance of the kmdl scheme based specfiles and buildsystems are the reason why ATrpms can offer that many kernel modules for that many distributions/releases/flavour at a very timely schedule after each kernel upgrade with only a fragment of one human resource :=)
Axel Thimm schrieb:
On Mon, Aug 07, 2006 at 06:19:51PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Mon, Aug 07, 2006 at 10:19:19AM -0500, Tom 'spot' Callaway wrote:
o one src.rpm for both userland and kernel module subpackages.
People disliked that when we created the current kmod standard because either you build the userland stuff for i586 and i686 or you have to add a lot of %if foo #build userland %endif %if bar #build module %endif into the spec file.
It's better than to have two possibly divergent specs/src.rpms.
Debatable. When the kmod standard was developed people were in favor of a split.
I used the above scheme for round about two years (E.g. FC2, FC3 and FC4) and I prefer the two srpms solution now that I got used to it (I didn't like it in the beginning).
Just imagine a cvs patch to apply to to both packages at once. You only have three %if/%else/%endif constructs in a common specfile saving quite a bit of redundant information and having a proper common changelog - maintenance and package overview is much better in a common specfile.
I don't think that's a real benefit.
I think some of the design problems of the current kernel module scheme is due to allowing too much "external" pressure to get them through. I don't think/remember early drafts to have done this splitting of src.rpms just as I know the uname-r was removed due to external requests. The kernel module scheme was finally accepted, but at what price?
It was a compromise and I must say it works a lot better than the stuff we had before (with uname-r in the name) and I'm quite satisfied with it.
People also wanted proper debuginfo packages. This means that you have to create the debuginfo packages either manually or have to increase the release each time you build for a new kernel.
Proper debuginfo packages are produced at ATrpms - just not published due to lack of size.
Okay. Looking closer. But that would also means you get a SRPM per kernel variant afaics? That will waste a lot of space on the servers (yes, technically all those SRPMS contain the same stuff, but there were people that want to find the corresponding SRPM by looking at %{SOURCERPM} -- that will fail if you don't upload all those SRPMS)
[...]
-> has to be queued to the buildsys multiple time to build for UP, SMP, Xen, ... (it at least looks this way? Or am I wrong?)
Yes, and that's actually a *feature* I forgot to add to the design principles:
o kmdls are completely kernel flavour agnostic. There is no hardwiring up/smp/xen anywhere.
That's also true for the current kmod stuff. We hardwire it currently for the buildsystem until it gains support for to pass the defines over. And new kernel flavour should in work without adjustments.
The easy maintenance of the kmdl scheme based specfiles and buildsystems are the reason why ATrpms can offer that many kernel modules for that many distributions/releases/flavour at a very timely schedule after each kernel upgrade with only a fragment of one human resource :=)
I still can't see it being much easier, sorry. Here and there it's a bit easier, yes, but here and there it's a bit more complicated -> overall it comes down to the same.
CU thl
On Tue, Aug 08, 2006 at 07:52:20AM +0200, Thorsten Leemhuis wrote:
You only have three %if/%else/%endif constructs in a common specfile saving quite a bit of redundant information and having a proper common changelog - maintenance and package overview is much better in a common specfile.
I don't think that's a real benefit.
Lower maintenance not a benefit?
I think some of the design problems of the current kernel module scheme is due to allowing too much "external" pressure to get them through. I don't think/remember early drafts to have done this splitting of src.rpms just as I know the uname-r was removed due to external requests. The kernel module scheme was finally accepted, but at what price?
It was a compromise and I must say it works a lot better than the stuff we had before (with uname-r in the name) and I'm quite satisfied with it.
I'm sorry, but you actively ignore the fact that was discussed over and over again in this thread that this scheme is *broken at rpm level*, dragging into the breakage all depsolvers which therefore *require* patching/plugins all one-by-one to avoid severe problems like nuking current working graphics drivers from the system. And you know that this will never get fixed on rpm level.
How can you be satisfied with this situation, when you know it can all be fixed? And not even know, but can see in practise. The kmdls are there since several years.
Okay. Looking closer. But that would also means you get a SRPM per kernel variant afaics? That will waste a lot of space on the servers
Yes, that would waste a lot of space. NO, THIS IS NOT THE CASE!!!
Please, Thorsten, it does look to me that you are desperately trying to find reasons against kmdls. You're really pushing me to the edge.
The waste of space are the current src.rpm pairs that are needed in your scheme. Twice the sources is redundant, a waste of space and error prone. so if you agree on saving space let's go kmdls, they only need half the space. ...
The easy maintenance of the kmdl scheme based specfiles and buildsystems are the reason why ATrpms can offer that many kernel modules for that many distributions/releases/flavour at a very timely schedule after each kernel upgrade with only a fragment of one human resource :=)
I still can't see it being much easier, sorry. Here and there it's a bit easier, yes, but here and there it's a bit more complicated -> overall it comes down to the same.
Sorry, we strongly disagree. What exactly is more complicated about kmdls? The fact that
o plugins are not *required*, but a nice-to-have? o these plugins are at most half the code that a plugin for the current scheme needs? o works at rpm level? o Only one src.rpm for everything cutting space and reducing maintenance
I can only see benfits with kmdls, and not nice-to-have stuff, but required functionality that the current scheme cannot offer.
For God's sake *the current scheme is broken* and we're trying to patch up yum with plugins to get back to rpm behaviour. And the patch/plugin is still not complete, and the same is needed for other depsolvers and still rpm CLI will be broken. Shouldn't that make all alarms sound? That's far from "overall it comes down to the same".
Hi!
Axel Thimm schrieb:
I'll leave the rest of the mail to the members of the packaging committee, but I'd like to say one thing to this para:
Please, Thorsten, it does look to me that you are desperately trying to find reasons against kmdls. You're really pushing me to the edge.
Sorry, that was not my intention. I just tried to lay down the details behind the decisions that lead to the current kmod standard.
CU thl
Hi Jack,
can you please answer on the statement below that wrt fedorakmod.py plugin if the wrongly tagged kernel module packages pull in other packages through dependencies you're hosed up?
Explenation for the others: That's because the plugin works in such a way that it needs to allow yum to perform the proper transaction setup which includes tagging kernel module packages for installation that should not had been tagged according to the kmod scheme (but that's not yum's fault, yum is just being rpm compatible).
So the plugin tries to remove again these packages before the transaction is commited. But it doesn't take into account that these packages may have further dependencies and may have pulled in more packages, which currently is not checked and I doubt that one can ever identify them.
Just to name real-world cases where kmdls pull in other kmdls and userland bits where this plugin would fail: GFS/cman/dlm, ivtv/v4ldvb, ipw3495{,d} packages, ipw*/ieee80211.
Please make a statement about that as this demonstrates that the plugin has unfixable flaws - which is neither yum's not the plugin's fault, it is just the mirroring of allowing an rpm-incompatible kernel module scheme.
I hope the correctness and maintenance issues here will persuade the last kmdl opponent. :)
On Mon, Aug 07, 2006 at 04:54:14PM +0200, Axel Thimm wrote:
On Mon, Aug 07, 2006 at 10:00:13AM -0400, Jack Neely wrote:
So any raised issue on the naming/versioning of kernel module packages still remains as discussed - including non-rpm conformant behaviour of such packages with the rpm -U/-i nuke/conflict situation, which is still not "fixed" on yum level or any other depsolver's (and having looked into yum's plugin system which is quite nice BTW, I don't know whether it will be "fixable" at all)
If the existing fedorakmod.py plugin in yum-utils does not handle installs, upgrades, and co-installs of kmod style kernel modules then I'd like a detailed bug report please.
AFAIU yum's plugin logic in order to perform according to the current rpm-level-broken kernel module scheme a plugin would have to undo parts of yum's transaction. E.g. yum does not offer a hook to manipulate its logic during the depsolving. Note that yum's depsolver logic is rpm conformant. Therefore the moment the fedorakmod.py takes over the transaction set may contain both nuking and conflicts which the plugin needs to address in retrospective now.
Coinstall support "solves" the first issue, but the second issue is only "solvable" by removing parts of the transaction. Unfortunately removing is an operation that needs to get back into the depsolver logic - unless the package to be removed has no further dependent or depending on packages. This may be true for most of the current few available kernel modules in this scheme, but there are others that have richer depedendency structure - most notably v4l2 and wifi related modules.
It doesn't look like the fedorakmod.py plugin takes that into account. It's still "fixable" by adding even more code, but it's already gaining code size and therefore maintenance costs for something that wasn't "broken" to begin with. This also shows that there may be even more bugs lurking, and this is only the support for yum, rpm still remains "broken" and so do all the other depsolvers.
It's nothing against Jack's work or even Thorsten's and Ville's work on the kernel module specs. It's just a core design element that proves to be bad and needs replacement asap.
I know I repeat myself, but:
There is the kmdl proposal which doesn't even need any special plugins to remain rpm-compliant for both rpm and all depsolvers and while support for coinstalls for new kernels is missing in all depsolvers it proved to be trivial to add a < 100 lines easy to maintain plugin to accomplish that. So there really is only the I-don't-like-uname-r- in-name issue left ...
Hi Jack,
On Sat, Aug 12, 2006 at 05:33:10PM +0200, Axel Thimm wrote:
can you please answer on the statement below that wrt fedorakmod.py plugin if the wrongly tagged kernel module packages pull in other packages through dependencies you're hosed up?
Please make a statement about that as this demonstrates that the plugin has unfixable flaws - which is neither yum's not the plugin's fault, it is just the mirroring of allowing an rpm-incompatible kernel module scheme.
I hope the correctness and maintenance issues here will persuade the last kmdl opponent. :)
I moved the issues to http://fedoraproject.org/wiki/AxelThimm/kmdls
Please check the raised issues and make a statement. The kmod+yum+anyplugin setup just cannot work and it's the best for the packaging folks to hear that from a third person, too. Thanks.
On Tue, Aug 15, 2006 at 01:32:20PM +0200, Axel Thimm wrote:
Hi Jack,
On Sat, Aug 12, 2006 at 05:33:10PM +0200, Axel Thimm wrote:
can you please answer on the statement below that wrt fedorakmod.py plugin if the wrongly tagged kernel module packages pull in other packages through dependencies you're hosed up?
Please make a statement about that as this demonstrates that the plugin has unfixable flaws - which is neither yum's not the plugin's fault, it is just the mirroring of allowing an rpm-incompatible kernel module scheme.
I hope the correctness and maintenance issues here will persuade the last kmdl opponent. :)
I moved the issues to http://fedoraproject.org/wiki/AxelThimm/kmdls
Please check the raised issues and make a statement. The kmod+yum+anyplugin setup just cannot work and it's the best for the packaging folks to hear that from a third person, too. Thanks. -- Axel.Thimm at ATrpms.net
Axel,
I apologize for not making your requested statement earlier. I hope this email proves to be acceptable.
I have read your Wiki article and I find several points that I do not agree with. I will focus on one of these points here. You state, in bold text, "[kmdl] doesn't have any design bugs." In fact, Seth, Jesse, Thorsten, myself and others have pointed out multiple times flaws in the kmdl kernel module packaging scheme. Some of these flaws equally affect both kmod and kmdl.
Having read your numerous posts to this list regarding kmdl, the wiki article you have authored, and your Yum plugin to support kmdl I see a common theme. You are unwilling to admit that your scheme may not be absolutely perfect and are unwilling to work with others to compromise. In fact, Thorsten attempted to compromise again with you this morning and you immediately refused.
The last two emails that you have sent in this thread addressed to me take this one step farther. You have requested that I make a statement regarding issues you have with the workings of the fedorakmod.py plugin. In and of itself, that is fine. However, you also tell me in both emails what my statement is to be and in the first justify why I will say such a statement. I find this attitude hostile and offensive.
My statement is thus: Many people have spent a large amount of time working with the kmod standard in developing the standard to this point and preparing FC6, FE6, and RHEL5 to make use of this standard. The kmod standard represents a significant step forward in kernel module packaging compared to older methods that I and many others have used in the past. The proposed kmdl scheme does not offer a significant improvement in design or practice compared to the kmod standard. It is simply a different selection of pros and cons.
Jack
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
Jack Neely wrote:
On Tue, Aug 15, 2006 at 01:32:20PM +0200, Axel Thimm wrote: You (Axel) are unwilling to admit that your scheme may not be absolutely perfect and are unwilling to work with others to compromise.
My $0.02 again, I think *both* sides, to a certain degree, are guilty of this.
-- Rex
On Tue, Aug 15, 2006 at 12:53:00PM -0500, Rex Dieter wrote:
Jack Neely wrote:
On Tue, Aug 15, 2006 at 01:32:20PM +0200, Axel Thimm wrote: You (Axel) are unwilling to admit that your scheme may not be absolutely perfect and are unwilling to work with others to compromise.
My $0.02 again, I think *both* sides, to a certain degree, are guilty of this.
You are probably right.
Can Axel come up with examples that make kmod and my code break miserably? Yes. We've all stated that the kmod scheme isn't perfect. I remember none of us being overly happy when we agreed to it.
However, I too can come up with examples that make kmdl and his Yum plugin break just as well. Several folks have discussed them. This hasn't swayed Axel.
I'm starting to think that Jesse is right. Kernel modules don't belong here. We have a couple good suggestions and sem-workable code for them. Hopefully they will be supplanted by the kabi stuff in the near future as I think that shows a lot of promise.
Perhaps offering several suggestions for packaging kernel modules to the community is what we do along with banning them from FC/FE. I don't like that at all but I'd agree to it.
Jack
-- Rex
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
On Tue, Aug 15, 2006 at 02:52:17PM -0400, Jack Neely wrote:
Can Axel come up with examples that make kmod and my code break miserably? Yes. We've all stated that the kmod scheme isn't perfect. I remember none of us being overly happy when we agreed to it.
However, I too can come up with examples that make kmdl and his Yum plugin break just as well. Several folks have discussed them. This hasn't swayed Axel.
Can you point me to that? In the context of kmdl + yum-plugin.
On Tue, 2006-08-15 at 14:52 -0400, Jack Neely wrote:
I'm starting to think that Jesse is right. Kernel modules don't belong here. We have a couple good suggestions and sem-workable code for them. Hopefully they will be supplanted by the kabi stuff in the near future as I think that shows a lot of promise.
Ok. A few comments here. I'm sorry I've not shouted too much before but I'm trying to listen rather than come into this group with too strong an opinion. I am very interested in discussing what we can do in the longer term about module packaging via IRC/whatever if it's useful toward reaching consensus, but I'm the new guy here so you tell me! :-)
At the moment, the kABI stuff is a small addition to kmods. I changed the kmodtool slightly (couple of lines of patch, attached a version) so that we depend upon kABI and remove the dependency on a particular kernel package version. All of that is in rawhide right now and has actually been for a while - but not so useful with no "stable" kABI.
I modified the kernel package build so that a new tool, kabitool, determines the groupings of ABI dependencies that are generated (it's all to work around limitations in RPM for the moment) and can take a config file that specifies these groupings too - so in the future, we could group things like the module struct, printk and other stuff that changes little but is a common dep. into its own dep. grouping. That would help when not having a stable kABI, but only a little.
Still, we don't have a stable kABI in Fedora (and that includes locking hierarchies that aren't automatically checked at the moment) so I am keen to avoid overselling ABI matching to the Fedora user base. I am, however working on some patches (as upstream maintainer of m-i-t) to allow depmod to be given explicit module priorities for multiple matching names under /lib/modules. This will allow us to override which modules will be selected by insmod/modprobe through a simple conf.d. I am hoping to get that stuff out into rawhide this week and available.
Jon.
On Tue, Aug 15, 2006 at 01:45:47PM -0400, Jack Neely wrote:
On Tue, Aug 15, 2006 at 01:32:20PM +0200, Axel Thimm wrote:
Hi Jack,
On Sat, Aug 12, 2006 at 05:33:10PM +0200, Axel Thimm wrote:
can you please answer on the statement below that wrt fedorakmod.py plugin if the wrongly tagged kernel module packages pull in other packages through dependencies you're hosed up?
Please make a statement about that as this demonstrates that the plugin has unfixable flaws - which is neither yum's not the plugin's fault, it is just the mirroring of allowing an rpm-incompatible kernel module scheme.
I hope the correctness and maintenance issues here will persuade the last kmdl opponent. :)
I moved the issues to http://fedoraproject.org/wiki/AxelThimm/kmdls
Please check the raised issues and make a statement. The kmod+yum+anyplugin setup just cannot work and it's the best for the packaging folks to hear that from a third person, too. Thanks.
Axel,
I apologize for not making your requested statement earlier. I hope this email proves to be acceptable.
I have read your Wiki article and I find several points that I do not agree with. I will focus on one of these points here. You state, in bold text, "[kmdl] doesn't have any design bugs." In fact, Seth, Jesse, Thorsten, myself and others have pointed out multiple times flaws in the kmdl kernel module packaging scheme. Some of these flaws equally affect both kmod and kmdl.
Please show me the *flaws*! The kmdl scheme perfectly blends into rpm semantics.
Having read your numerous posts to this list regarding kmdl, the wiki article you have authored, and your Yum plugin to support kmdl I see a common theme. You are unwilling to admit that your scheme may not be absolutely perfect and are unwilling to work with others to compromise. In fact, Thorsten attempted to compromise again with you this morning and you immediately refused.
That's wrong. I myself had stated often enough that I don't like uname-r-in-name. It is not "perfect" it is a neccessity given the current packaging limitations.
And how exactly does the kmdl plugin tell you that I'm unwilling to admit anything at all?
The last two emails that you have sent in this thread addressed to me take this one step farther. You have requested that I make a statement regarding issues you have with the workings of the fedorakmod.py plugin.
Yes, please and the request stands.
In and of itself, that is fine. However, you also tell me in both emails what my statement is to be and in the first justify why I will say such a statement. I find this attitude hostile and offensive.
Oh, no, please do make your statement on your behalf. You certainly don't need patronizing.
My statement is thus: Many people have spent a large amount of time working with the kmod standard in developing the standard to this point and preparing FC6, FE6, and RHEL5 to make use of this standard. The kmod standard represents a significant step forward in kernel module packaging compared to older methods that I and many others have used in the past. The proposed kmdl scheme does not offer a significant improvement in design or practice compared to the kmod standard. It is simply a different selection of pros and cons.
So where is the statement? You are talking abstractly about kmdls and kmods and I wrote explicitely about where and how fedorakmod.py is broken. I'd like a statement about the possibility of yum working properly with kmod.
The thesis that is given in the wiki is
- that fedorakmod is currently broken, - that any future attempt to fix it breaks it even more, - that it is therefore unfixable with limited man power
That's where I'd like you to either stand up and say Axel's halucinating or to concurr with the outlayed facts. That's far more valuable than quoting "kmod standard represents a significant step forward", we want technical facts, not a marketing blob :)
Please the request for making a statement about current and future yum support of kmods still stands. If you admit it's broken people will make easier a decision. If you prove it's not broken or if broken fixable with few steps then it will also help people decide.
Axel Thimm schrieb:
On Sun, Jul 30, 2006 at 06:04:34PM +0200, Thorsten Leemhuis wrote:
Toshio Kuratomi schrieb:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
I like that idea -- especially when combined with the the kabi stuff. Yes, someone still could run "rpm -Uvh" and would loose older kmods, but yum and apt would do the right thing.
Why would suddenly yum/apt work better?
Because
/lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
avoids that there are ever file conflicts between packages so yum will always be able to install the new module (just like the kernel -- you can of course still do rpm -Uvh manually, but I don't care because that's possible with the kernel, too).
CU thl
On Mon, Aug 07, 2006 at 06:22:52PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Sun, Jul 30, 2006 at 06:04:34PM +0200, Thorsten Leemhuis wrote:
Toshio Kuratomi schrieb:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
I like that idea -- especially when combined with the the kabi stuff. Yes, someone still could run "rpm -Uvh" and would loose older kmods, but yum and apt would do the right thing.
Why would suddenly yum/apt work better?
Because
/lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
avoids that there are ever file conflicts between packages so yum will always be able to install the new module (just like the kernel -- you can of course still do rpm -Uvh manually, but I don't care because that's possible with the kernel, too).
I understand, you push the problem from rpm/yum to module-init-tools, e.g. now /sbin/depmod needs to understand rpmvercmp to decide which MODULE-VERSION-RELEASE is newer?
The issue is still there, just not where it belong, in rpm and yum. It is now at module-init-tools level and suddenly /sbin/depmod would need to understand rpm ordering rules.
I don't think that's an improvement :)
Axel Thimm schrieb:
On Mon, Aug 07, 2006 at 06:22:52PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Sun, Jul 30, 2006 at 06:04:34PM +0200, Thorsten Leemhuis wrote:
Toshio Kuratomi schrieb:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
I like that idea -- especially when combined with the the kabi stuff. Yes, someone still could run "rpm -Uvh" and would loose older kmods, but yum and apt would do the right thing.
Why would suddenly yum/apt work better?
Because
/lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
avoids that there are ever file conflicts between packages so yum will always be able to install the new module (just like the kernel -- you can of course still do rpm -Uvh manually, but I don't care because that's possible with the kernel, too).
I understand, you push the problem from rpm/yum to module-init-tools, e.g. now /sbin/depmod needs to understand rpmvercmp to decide which MODULE-VERSION-RELEASE is newer?
Well, if we use the kerneldrivers stuff from jcm then /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko seems like the better place for modules than /lib/modules/KERNELVER/extra/MODULE/MODULE.ko because the latter path might be confusing when KERNELVER was already deinstalled.
The issue is still there, just not where it belong, in rpm and yum. It is now at module-init-tools level and suddenly /sbin/depmod would need to understand rpm ordering rules.
The kerneldrivers stuff will handle that in any case afaics. The question is: do we want to use it. There are also other open question if we want to use it, e.g. - when does a kmod get removed? - do we need to drop the hard dep on the kernel
I don't think that's an improvement :)
I didn't try it out yet. Maybe it's an improvement, maybe not.
CU thl
On Tue, Aug 08, 2006 at 07:40:51AM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Mon, Aug 07, 2006 at 06:22:52PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
On Sun, Jul 30, 2006 at 06:04:34PM +0200, Thorsten Leemhuis wrote:
Toshio Kuratomi schrieb:
Apologies for posting into the wrong subthread of this monster, I already deleted the relevant mail.
If one of the major issues with the current kmod spec is that neither rpm -U nor rpm -i work correctly, shouldn't that be corrected? If the module could install into something like this: /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
instead of: /lib/modules/KERNELVER/extra/MODULE/MODULE.ko
wouldn't that bring the behaviour of kmods inline with that of the kernel? (Use rpm -i for normal operations, rpm -U if you don't believe in Murphy).
I like that idea -- especially when combined with the the kabi stuff. Yes, someone still could run "rpm -Uvh" and would loose older kmods, but yum and apt would do the right thing.
Why would suddenly yum/apt work better?
Because
/lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko
avoids that there are ever file conflicts between packages so yum will always be able to install the new module (just like the kernel -- you can of course still do rpm -Uvh manually, but I don't care because that's possible with the kernel, too).
I understand, you push the problem from rpm/yum to module-init-tools, e.g. now /sbin/depmod needs to understand rpmvercmp to decide which MODULE-VERSION-RELEASE is newer?
Well, if we use the kerneldrivers stuff from jcm then /lib/modules/MODULE-VERSION-RELEASE/(KERNELVER|KABI)/MODULE.ko seems like the better place for modules than /lib/modules/KERNELVER/extra/MODULE/MODULE.ko because the latter path might be confusing when KERNELVER was already deinstalled.
I agree, but still something needs to manage the evr (where epoch is missing in the above path BTW) comparison of modules per kernel.
The issue is still there, just not where it belong, in rpm and yum. It is now at module-init-tools level and suddenly /sbin/depmod would need to understand rpm ordering rules.
The kerneldrivers stuff will handle that in any case afaics. The question is: do we want to use it. There are also other open question if we want to use it, e.g.
- when does a kmod get removed?
- do we need to drop the hard dep on the kernel
I don't think that's an improvement :)
I didn't try it out yet. Maybe it's an improvement, maybe not.
*Something* in the system needs to be able to compare different evrs of "MODULE-VERSION-RELEASE" and decide on *removing* the old module. It really *needs removal* and not simple overriding because the new and old kernel module package's contents may differ in the number/names of carried kernel module files.
Even if removing would not be needed and we could live with module-init-tools doing some magic you would still have to push the rpmvercmp logic to module-init-tools which upstream will not accept. And we don't gain anything: The problem must still be solved, it just looks like being out of sight.
So pushing this problem out of rpm's and yum's scope is very wrong and only creates more problems. This is not against the actual location of kernel module files. For the kabi forward-compatibility stuff thinking about new locations is OK, but it will not solve the issues we're talking about. It's realy purely orthogonal.
<rant> Did I mention that resitance is futile? This is the umpteenth attempt to find something to block kmdl adoption, that I need to argue against. If I would had put that energy into something else I would probably had solved the TOE by now.
Possibly my attempts to get a sane kernel module packaging scheme is futile ... </rant>
Axel Thimm schrieb:
On Tue, Aug 08, 2006 at 07:40:51AM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
<rant> Did I mention that resitance is futile? This is the umpteenth attempt to find something to block kmdl adoption, that I need to argue against. If I would had put that energy into something else I would probably had solved the TOE by now.
Possibly my attempts to get a sane kernel module packaging scheme is futile ...
</rant>
You are in the packaging committee, I'm not. If you convince the others in the committee that throwing away the current standard (that will be used in RHEL5 afaics) and that going "back" to something that's similar to the stuff we used in fedora.us/lvn is a good thing -> go ahead.
CU thl
Would you be interested in adding support for the kmdl scheme to yum (or a plugin) *before* a decision is made?
I'm suggesting this because kernel module handling is for many in this list too abstract to discuss on paper and a live demonstration using yum and kmdls would certainly help moving on on this subject here. A bash shell script doing it is 9 lines, so I hope for you it would be almost trivial to pluginize.
Even if the decision can't be made in time for FC6 (which effectively would mean to keep the current scheme) it would not be a waste of time as it is already in use at ATrpms, and many users want to see yum auto-coinstalling them upon new kernels. -- Axel.Thimm at ATrpms.net
If you'd like to contribute a Yum plugin or patches to one of the existing 2 they are in the yum-utils project. They're not hard to write and there are a handful of good examples now. My time is going toward the current proposal.
Panu does have a plugin in yum-utils to support a uname-r in name scheme for kernel modules. Looks like it needs to be updated a bit, but perhaps that could be a starting point for your work.
Jack
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
On Tue, Jul 25, 2006 at 11:31:29PM +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 05:13:40PM -0400, seth vidal wrote:
On Sun, 2006-07-23 at 23:03 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 04:38:59PM -0400, seth vidal wrote:
On Sun, 2006-07-23 at 22:19 +0200, Axel Thimm wrote:
On Sun, Jul 23, 2006 at 10:32:06PM +0300, Ville Skyttä wrote:
> rpm -U/-i will nuke or overwrite kernel modules of the running > kernel in a uname-r-less scheme.
But I thought it was already stated that we don't care about rpm used on the cli to handle these sorts of things. That we've assumed we're operating at a level above rpm for constructing the transaction set.
A scheme were two independent EVRs are merged together into one is broken and banning the tool that exhibits it and pasting it away from others isn't the solution. kmod-foo-1-a just has nothing to do with kmod-foo-<anything>-b (in the simplified notation).
okay. I can take that.
Here's my only complaint and it is a complaint for all sides equally:
I would like one and final standard decided on BEFORE FC6 is released. We need to get the code for yum complete so we can stop messing around with this stuff.
At this point I'm in favor of whatever gets it done.
Would you be interested in adding support for the kmdl scheme to yum (or a plugin) *before* a decision is made?
I'm suggesting this because kernel module handling is for many in this list too abstract to discuss on paper and a live demonstration using yum and kmdls would certainly help moving on on this subject here. A bash shell script doing it is 9 lines, so I hope for you it would be almost trivial to pluginize.
Even if the decision can't be made in time for FC6 (which effectively would mean to keep the current scheme) it would not be a waste of time as it is already in use at ATrpms, and many users want to see yum auto-coinstalling them upon new kernels.
On Wed, Jul 26, 2006 at 10:33:05AM -0400, Jack Neely wrote:
If you'd like to contribute a Yum plugin or patches to one of the existing 2 they are in the yum-utils project. They're not hard to write and there are a handful of good examples now. My time is going toward the current proposal.
Panu does have a plugin in yum-utils to support a uname-r in name scheme for kernel modules. Looks like it needs to be updated a bit, but perhaps that could be a starting point for your work.
Thanks, I looked it up and indeed writing a yum plugin for kmdls is trivial. I really like the design of the plugin/hook mechanism.
So I managed to get a fully featured kmdl plugin under 100 python lines (actually 99 ;). It even has two modes of operation:
yum update: Will update/coinstall kmdl of all old and new kernels
yum install: Will only do something with kmdls if a kernel is asked to be installed (and only for that kernel)
I differ between the two, because I don't want the user to be surprised with kernel module installations when he yum-installs openoffice :)
It also supports asynced kernel/kernel-module repo arrival. When a kernel update appears it will be installed even when the kmdls aren't yet available/built. If on the next yum update the previously missing kmdls appear, they will get installed at that time.
I'm attaching the file (kmdl.py) to this mail. Seth & Jack you have a look and if approved add it to yum/yum-util sources (whatever is more appropriate)? If you don't like it let me know what needs improvement.
To atrpms-devel: Please test the kmdl.py by copying it to /usr/lib/yum-plugins/kmdl.py
To fedora-packaging: Now we have full support on rpm and yum level for kmdls [*], and it's rather comprehendable and maintainable code. If we can get over the ugliness or uname-r-in-name we should really adopt that scheme.
[*] apt has kmdl support from lua, but during apt's time-off I think it regressed, but it can be easily resurrected, it will also be a couple of lua lines. smart support for any kernel module scheme seems to be the toughest because it's plugin scheme is not well documented and it looks like it misses some important hooks.
On Thu, 2006-07-27 at 01:07 +0200, Axel Thimm wrote:
Thanks, I looked it up and indeed writing a yum plugin for kmdls is trivial. I really like the design of the plugin/hook mechanism.
So I managed to get a fully featured kmdl plugin under 100 python lines (actually 99 ;). It even has two modes of operation:
Yah the plugin interface is pretty happy. I even gave a tutorial at lca on the subject:
http://linux.duke.edu/~skvidal/lca-yum-tutorial/
Menno and others did some great work on the interface.
yum update: Will update/coinstall kmdl of all old and new kernels
yum install: Will only do something with kmdls if a kernel is asked to be installed (and only for that kernel)
I differ between the two, because I don't want the user to be surprised with kernel module installations when he yum-installs openoffice :)
It also supports asynced kernel/kernel-module repo arrival. When a kernel update appears it will be installed even when the kmdls aren't yet available/built. If on the next yum update the previously missing kmdls appear, they will get installed at that time.
I'm attaching the file (kmdl.py) to this mail. Seth & Jack you have a look and if approved add it to yum/yum-util sources (whatever is more appropriate)? If you don't like it let me know what needs improvement.
I'll take a look shortly. thanks.
The plugins are generally put in the yum-utils package and each of them broken out into their own subpackage.
then adding them or not is easy for the end user.
If everything looks okay is that cool for you?
-sv
On Thu, Jul 27, 2006 at 06:12:39PM -0400, seth vidal wrote:
On Thu, 2006-07-27 at 01:07 +0200, Axel Thimm wrote:
Thanks, I looked it up and indeed writing a yum plugin for kmdls is trivial. I really like the design of the plugin/hook mechanism.
So I managed to get a fully featured kmdl plugin under 100 python lines (actually 99 ;). It even has two modes of operation:
Yah the plugin interface is pretty happy. I even gave a tutorial at lca on the subject:
http://linux.duke.edu/~skvidal/lca-yum-tutorial/
Menno and others did some great work on the interface.
I see, it looks like YumBase can do everything and one mustn't (shouldn't?) resort to the rpm module directly.
yum update: Will update/coinstall kmdl of all old and new kernels
yum install: Will only do something with kmdls if a kernel is asked to be installed (and only for that kernel)
I differ between the two, because I don't want the user to be surprised with kernel module installations when he yum-installs openoffice :)
It also supports asynced kernel/kernel-module repo arrival. When a kernel update appears it will be installed even when the kmdls aren't yet available/built. If on the next yum update the previously missing kmdls appear, they will get installed at that time.
I'm attaching the file (kmdl.py) to this mail. Seth & Jack you have a look and if approved add it to yum/yum-util sources (whatever is more appropriate)? If you don't like it let me know what needs improvement.
I'll take a look shortly. thanks.
The plugins are generally put in the yum-utils package and each of them broken out into their own subpackage.
then adding them or not is easy for the end user.
If everything looks okay is that cool for you?
Yes, thanks!
On Fri, 2006-07-28 at 14:07 +0200, Axel Thimm wrote:
http://linux.duke.edu/~skvidal/lca-yum-tutorial/
Menno and others did some great work on the interface.
I see, it looks like YumBase can do everything and one mustn't (shouldn't?) resort to the rpm module directly.
we've just tried to simplify the interface some. There's still much more room to improve and we've made some of those improvements in the 2.9.X devel branch.
-sv
On Sun, Jul 23, 2006 at 11:54:17AM -0500, Tom 'spot' Callaway wrote:
On Sun, 2006-07-23 at 17:45 +0200, Axel Thimm wrote:
c) is the only technical sensible solution bringing us at 90% of the target with only one drawback: Ugly names. So what, technical aesthetics superseed what meets the eye. :)
It breaks bugzilla for starters.
Kernel modules for foo should only have one bugzilla entry, namely "foo", e.g. the src.rpm's name. Just like there is no bugzilla netry for "foo-devel" and other subpackaging - you can safely consider kmdl subpackages of the same package.
No other package puts the version for something it depends on in its Name.
Well, we already gave some examples and there were gstreamer07 (or 08) plugins as well. When you have a very tight integration of core package and it's plugins (which is what the kernel and the kernel module packages are) *AND* you have the requirement that the core package can have multiple consurrent installs, then the plugins' association to the core packages need to be marked in the name. Specifically for kernel modules this means uname-r-in-the-name.
So, what it boils down to is: rpm is not well built to handle the kernel module packaging case.
Can we fix rpm? I doubt it. Everytime I point out a weakness in rpm, I get told that its not going to be fixed. Since upstream is actively hostile towards us, we certainly can't expect them to help.
No, we can't fix rpm. If there is ever an rpm-ng project revising package management in general we could address this on the blueprint stage, but as it stands neither rpm, nor deb can be patched up to overcome some deficiencies (and this is not the only one).
On Fri, Jul 21, 2006 at 05:51:10PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
I don't want to reply to the other aspects of this mail -- I don't think it makes to much sense now and prefer to wait for the docs from Axel.
Well, maybe there won't be any docs, we should really decide on the uname -r thing first, otherwise the docs will be invalid and unneccessary.
No, I don't think we talked past each other, you are giving a perfect example outlining the design bug of any non-uname-r-in-name scheme.
Well, you said "Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) [...]e.g. at least one kernel stay w/o any kmdl[...]" above -- and xen0 and non-xen0 work fine with the scheme, both have a module.
OK, forget about the kernel flavours, I noted that further below, but the bigger bug is still in place: Nuking of kernel modules of all previous kernels including the one currently running and installing the next kernel.
Well, seems to be a bug for you. Okay, noted. I'd consider is as an minor annoyance. And it seems Jack Neely is working on getting it fixed.
He can't fix it on rpm level, and therefore each any every depsolver needs patching, will he do that for all major 4 (yum/smart/apt/up2date), or will we have to say don't use depsolvers 2-4 and any other future depsolver/applet because they don't fix the breakage we introduced at rpm level?
And I stongly disagree with calling silently removing kernel modules for all installed kernels a "minor annoyance" (?). When upgrading to a new kernel one of the "old" kernels is by definition the one you are currently running. So nuking your display driver, maybe even your storage access (imagine ISV's 3w*, qla*) while upgrading is a critical bug IMO.
You don't want to only support the rpm-latest kernel,
To support only the latest kernel was a decided early in the kmod discussion. That might have influenced the kmod standard.
Maybe, but that's a wrong decision. It can only be decided that way if the previous kernel is also immediately obsoleted in the sense that it shouldn't even serve as a fallback anymore. That's a policy that can't go through. And having one policy for the kernel and another for modules is wrong, too. Imagine a policy of "we support the previous kernel for some transitions time, but only stripped from it's kernel modules?"
In fact the above is not to be layed in the future, but it's the current situation w/ for example livna, or not? Aren't there any livna reports that they lost their nvidia driver for old kernels while updateing their system?
Nobody reported a bug yet. And the nvidia driver is a good example: we even have explicit "Obsoletes" in the spec to make sure old modules always get removed because nvidia-1.2.3 only works with kmod-nvidia-1.2.3, but not with kmod-nvidia-1.2.2
That's different, you are referring to module's version, not the kernel's. It's the kernel's version (e..g uname -r) that gets in the way.
No, both the up and the smp-kernel still have their modules.
And the reason is kernel flavour disambiguation through the name. Add the kernel's version to the name and you'll fix the bug mentioned above. And suddenly it's a uname-r-in-name scheme again and very close to kmdl systems (so let's pick that design and be all happy).
And it also needs special support in depsolvers so you get all kernel-modules together with a kernel update.
No, that's an unfair comparison.
o Special support for kmdls is far easier to implement: No need to introduce special non-rpm versioning, half the work is already done by natural rpm ordering, only the coinstallation upon new kernel installs needs to be done. Remember the the 9-liner in bash? That's all there is to it, port that to python and let someone who has looked at yum plugins looks over it for a minute and you're ready. Due to it's simplicity it's is also rather bug-free from the beginning.
o Not having special support for kmdls isn't as critical: The worst case is that the new kernel has no initial kmdls. But upgrading of old kmdls works as expected and there is no danger of nuking away unrelated kernel modules.
IMO these make a very large difference in the quality, neccessity and ease of implementation of "required special support" in both schemes.
And you need to maintain a lot of obsoletes to get old kmods removed that are incompatible with the updated userland packages.
No, that's not neccessary, if you have the proper dependencies in place, which the kmdl scheme automatically has.
Or you need to build kmods for each and every kernel that ever shipped -- that would take to long (and they are often not available anymore).
No, that's also not neccessary. You only build for the kernels you care about, e.g. the ones that the buildsystem considers still supported in whatever sense (e.g. exist in updates-relased).
So, do we at least agree that the current kernel modules scheme breaks on rpm CLI level?
No.
Not being able to use rpm CLI, and if used using to either (partial) module overwrites or nuking of old kernel modules is no breakage?
Well, we really differ in opinion here, I consider this a big design flaw to drop direct rpm support or to allow for overwrites/nuking to happen.
Neither rpm -i, nor rpm -U could get you out of the above (typical!) situation.
I don't consider it a big problem.
So what's your cure of the problem? Disallow using /usr/bin/rpm? Moving it away and only allowing access through yum?
We know rpm needs higher level tools to really manage the packages including net access to packages, but until now it's been an iron rule to remain rpm compatible (which means rpm -U/-i/-F should be applicable).
Aynway if there is an unmovable blocker/veto by someone that the name may only contain kernel flavour, but no further uname -r, and he/they cannot be convinced otherwise the cause is lost right from the start.
Before losing more time into this, this pricipal issue should be resolved (maybe by a vote on next Thursday), as my doc work will be wasted (in this scope) right from the start.
Thorsten, besides commenting on this mail, would you have time next Thursday 1h before the fesco meeting to join #fedora-packaging for this?
Axel Thimm schrieb:
On Fri, Jul 21, 2006 at 05:51:10PM +0200, Thorsten Leemhuis wrote:
Axel Thimm schrieb:
[...] Thorsten, besides commenting on this mail, would you have time next Thursday 1h before the fesco meeting to join #fedora-packaging for this?
I don't know yet for sure if I have time then next Thursday, but I currently suppose I'm online then and in that case I of course can join that channel *if* the Committee wants to discuss this.
CU thl
packaging@lists.fedoraproject.org
-
Axel Thimm -
Jack Neely -
Jesse Keating -
Jon Masters -
Panu Matilainen -
Rex Dieter -
seth vidal -
Thorsten Leemhuis -
Tom Callaway -
Toshio Kuratomi -
Ville Skyttä