packaging August 2006

packaging@lists.fedoraproject.org

35 participants
40 discussions

Kernel Module Packaging Standard Teleconference
by Warren Togami 18 Aug '06

18 Aug '06

After weeks of discourse on IRC and mailing list discussions, we seem to be unable to make progress toward an amicable agreement. What seems to be the issue is that RPM does not allow any clean solution to the kernel module packaging problem. Both kmod and kmdl are comprised of ugly hacks in order to overcome the design limitations of RPM. Both kmod and kmdl have pros and cons, and neither is a clearly agreed by all parties as being technically superior. The FESCO side is disconcerted by this situation because a great deal of effort over years was put into making and ratifying today's kmod standard. On the other hand, there is some hope for middle-ground because of stated willingness to further revise kmod to overcome remaining limitations. It is the decision of me, Max, and Spot, to put a hold on publishing any further kmod modules pending the result the Kernel Module Packaging Standard conference call. Time/date of this call is to be determined, but we are aiming for this Friday, August 18th at a time that is workable for both American EDT and Europe. Perhaps 1PM EDT would work, the usual timeslot of FESCO meetings? Perhaps this timeslot wont work. If you feel it is important to attend but you are unable to make this time, please make it known in this thread. In order to make this meeting manageable, we must limit the number of attendees. Invited are the existing members of the Fedora Packaging Committee, Thorsten Leemhuis and certain Red Hat employees relevant to this discussion. Please talk to me directly if you have been an active contributor to this in the past but are not included in this invitee list. My personal feeling is that it is way too late to throw out kmod, OTOH it is important that we discuss each point to see if kmod can be improved. If the packaging committee cannot come to key agreements, then the matter will be escalated to the Fedora Project Board who will make the key decisions. Warren Togami wtogami(a)redhat.com

11 37

file-conflicts on updates problem
by Thorsten Leemhuis 18 Aug '06

18 Aug '06

Hi all! I was looking at the current kmod situation a bit again and tested some stuff and it seems I'm to dump to reproduce the "file-conflicts on updates problem" -- a problem that I saw myself some weeks ago once IIRC and that was multiple times brought into the current discussions by Axel. Seems I'm doing something wrong here. Can somebody tell me that that is? I uploaded two srpms for testing to http://www.leemhuis.info/files/fedorarpms/MISC.fdr/kmod/ for people to reproduce the problem locally. That's not a real kmod, but it uses the kmod-scheme and a "kernel module" for testing which is created by > dd if=/dev/urandom bs=1k count=500 > $RPM_BUILD_ROOT/lib/modules/%{kverrel}${kvariant}/extra/%{kmod_name}/foo.ko Look closer yourself below to understand my confusion. Note, all the yum calls are on FC5 with latest updates, but without the kmod plugin: Prepare the test rpms: > [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod-common.spec > [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod.spec --define 'kversion 2.6.17-1.2157_FC5' --define 'kvariants ""' --define 'thisrelease 1' --target i686 > [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod.spec --define 'kversion 2.6.17-1.2174_FC5' --define 'kvariants ""' --define 'thisrelease 1' --target i686 > [thl@notebook ~]$ rpmbuild -ba rpmbuild/SPECS/foo-kmod.spec --define 'kversion 2.6.17-1.2174_FC5' --define 'kvariants ""' --define 'thisrelease 2' --target i686 [...] Put them in a testrepo: > [thl@notebook ~]$ mkdir -p ~/tmp/testrepo > [thl@notebook ~]$ cp rpmbuild/RPMS/noarch/foo-kmod-common-1.1-1.noarch.rpm rpmbuild/RPMS/i686/kmod-foo-1.1-1.2.6.17_1.2157_FC5.i686.rpm ~/tmp/testrepo/ > [thl@notebook ~]$ createrepo ~/tmp/testrepo > 2/2 - kmod-foo-1.1-1.2.6.17_1.2157_FC5.i686.rpm > Saving Primary metadata > Saving file lists metadata > Saving other metadata Install the first one with yum: > [thl@notebook ~]$ sudo yum --enablerepo=testrepo install kmod-foo > Loading "installonlyn" plugin > Setting up Install Process > Setting up repositories [...] > Parsing package install arguments > Resolving Dependencies > --> Populating transaction set with selected packages. Please wait. > ---> Package kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 set to be installed > --> Running transaction check > --> Processing Dependency: foo-kmod-common >= 1.1 for package: kmod-foo > --> Restarting Dependency Resolution with new changes. > --> Populating transaction set with selected packages. Please wait. > ---> Package foo-kmod-common.noarch 0:1.1-1 set to be updated > --> Running transaction check > > Dependencies Resolved > > ============================================================================= > Package Arch Version Repository Size > ============================================================================= > Installing: > kmod-foo i686 1.1-1.2.6.17_1.2157_FC5 testrepo 504 k > Installing for dependencies: > foo-kmod-common noarch 1.1-1 testrepo 2.2 k > > Transaction Summary > ============================================================================= > Install 2 Package(s) > Update 0 Package(s) > Remove 0 Package(s) > Total download size: 506 k > Is this ok [y/N]: y > Downloading Packages: > Running Transaction Test > Finished Transaction Test > Transaction Test Succeeded > Running Transaction > Installing: foo-kmod-common ######################### [1/2] > Installing: kmod-foo ######################### [2/2] > WARNING: Module /lib/modules/2.6.17-1.2157_FC5/extra/foo/foo.ko is not an elf object > > Installed: kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 > Dependency Installed: foo-kmod-common.noarch 0:1.1-1 > Complete! > [thl@notebook ~]$ Copy the second one in the repo and update: > [thl@notebook ~]$ cp rpmbuild/RPMS/i686/kmod-foo-1.1-1.2.6.17_1.2174_FC5.i686.rpm ~/tmp/testrepo/ > [thl@notebook ~]$ createrepo ~/tmp/testrepo/ > 3/3 - kmod-foo-1.1-1.2.6.17_1.2174_FC5.i686.rpm > Saving Primary metadata > Saving file lists metadata > Saving other metadata > [thl@notebook ~]$ sudo rm -rf /var/cache/yum/testrepo/ > [thl@notebook ~]$ sudo yum --enablerepo=testrepo update > Loading "installonlyn" plugin > Setting up Update Process > Setting up repositories > macromedia [1/7] > livna [2/7] > testrepo [3/7] > testrepo 100% |=========================| 951 B 00:00 > updates-testing [4/7] > core [5/7] > updates [6/7] > extras [7/7] > Reading repository metadata in from local files > primary.xml.gz 100% |=========================| 1.0 kB 00:00 > testrepo : ################################################## 3/3 > Added 3 new packages, deleted 0 old in 0.05 seconds > Resolving Dependencies > --> Populating transaction set with selected packages. Please wait. > ---> Downloading header for kmod-foo to pack into transaction set. > kmod-foo-1.1-1.2.6.17_1.2 100% |=========================| 2.4 kB 00:00 > ---> Package kmod-foo.i686 0:1.1-1.2.6.17_1.2174_FC5 set to be installed > --> Running transaction check > > Dependencies Resolved > > ============================================================================= > Package Arch Version Repository Size > ============================================================================= > Installing: > kmod-foo i686 1.1-1.2.6.17_1.2174_FC5 testrepo 504 k > > Transaction Summary > ============================================================================= > Install 1 Package(s) > Update 0 Package(s) > Remove 0 Package(s) > Total download size: 504 k > Is this ok [y/N]: y > Downloading Packages: > Running Transaction Test > Finished Transaction Test > Transaction Test Succeeded > Running Transaction > Installing: kmod-foo ######################### [1/1] > WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object > > Installed: kmod-foo.i686 0:1.1-1.2.6.17_1.2174_FC5 > Complete! > [thl@notebook ~]$ Works as expected so far. But now: > [thl@notebook ~]$ cp rpmbuild/RPMS/i686/kmod-foo-1.1-2.2.6.17_1.2174_FC5.i686.rpm tmp/testrepo/ > [thl@notebook ~]$ sudo rm -rf /var/cache/yum/testrepo/ > [thl@notebook ~]$ createrepo ~/tmp/testrepo/ > 4/4 - kmod-foo-1.1-2.2.6.17_1.2174_FC5.i686.rpm > Saving Primary metadata > Saving file lists metadata > Saving other metadata > [thl@notebook ~]$ sudo yum --enablerepo=testrepo update > Loading "installonlyn" plugin > Setting up Update Process > Setting up repositories > macromedia [1/7] > livna [2/7] > testrepo [3/7] > testrepo 100% |=========================| 951 B 00:00 > updates-testing [4/7] > core [5/7] > updates [6/7] > extras [7/7] > Reading repository metadata in from local files > primary.xml.gz 100% |=========================| 1.1 kB 00:00 > testrepo : ################################################## 4/4 > Added 4 new packages, deleted 0 old in 0.10 seconds > Resolving Dependencies > --> Populating transaction set with selected packages. Please wait. > ---> Downloading header for kmod-foo to pack into transaction set. > kmod-foo-1.1-2.2.6.17_1.2 100% |=========================| 2.4 kB 00:00 > ---> Package kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5 set to be installed > --> Running transaction check > --> Populating transaction set with selected packages. Please wait. > ---> Package kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 set to be erased > --> Running transaction check > > Dependencies Resolved > > ============================================================================= > Package Arch Version Repository Size > ============================================================================= > Installing: > kmod-foo i686 1.1-2.2.6.17_1.2174_FC5 testrepo 504 k > Removing: > kmod-foo i686 1.1-1.2.6.17_1.2157_FC5 installed 500 k > > Transaction Summary > ============================================================================= > Install 1 Package(s) > Update 0 Package(s) > Remove 1 Package(s) > Total download size: 504 k > Is this ok [y/N]: y > Downloading Packages: > Running Transaction Test > Finished Transaction Test > Transaction Test Succeeded > Running Transaction > Removing : kmod-foo ######################### [1/2] > Installing: kmod-foo ######################### [2/2] > WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object > > Removed: kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 > Installed: kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5 > Complete! > [thl@notebook ~]$ Okay, why did that work? I don't know for real. Let's check some details: > [thl@notebook ~]$ rpm -qa kmod-foo > kmod-foo-1.1-2.2.6.17_1.2174_FC5 > kmod-foo-1.1-1.2.6.17_1.2174_FC5 > [thl@notebook ~]$ rpm -V kmod-foo-1.1-1.2.6.17_1.2174_FC5 > ..5....T /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko > [thl@notebook ~]$ rpm -V kmod-foo-1.1-2.2.6.17_1.2174_FC5 > [thl@notebook ~]$ Well, I'd call that "interesting". Seem the "file-conflicts on updates problem" doesn't exist. But while testing it I noticed something else. You probably noticed that in above transaction kmod-foo.i686 0:1.1-1.2.6.17_1.2157_FC5 got removed -- it seems that's not a rpm/yum feature. It's the installonly-plugin that does this. Let's get back to the state we were before: > [thl@notebook ~]$ sudo rpm -ivh /home/thl/tmp/testrepo/kmod-foo-1.1-1.2.6.17_1.2157_FC5.i686.rpm --oldpackage > Preparing... ########################################### [100%] > 1:kmod-foo ########################################### [100%] > WARNING: Module /lib/modules/2.6.17-1.2157_FC5/extra/foo/foo.ko is not an elf object > [thl@notebook ~]$ sudo rpm -e kmod-foo-1.1-1.2.6.17_1.2174_FC5 kmod-foo-1.1-2.2.6.17_1.2174_FC5 > [thl@notebook ~]$ sudo rpm -ivh /home/thl/tmp/testrepo/kmod-foo-1.1-1.2.6.17_1.2174_FC5.i686.rpm > Preparing... ########################################### [100%] > 1:kmod-foo ########################################### [100%] > WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object > [thl@notebook ~]$ Now disable the plugin: > [thl@notebook ~]$ sudo yum --noplugins --enablerepo=testrepo update > Setting up Update Process > Setting up repositories > macromedia [1/7] > livna [2/7] > testrepo [3/7] > updates-testing [4/7] > core [5/7] > updates [6/7] > extras [7/7] > Reading repository metadata in from local files > Resolving Dependencies > --> Populating transaction set with selected packages. Please wait. > ---> Downloading header for kmod-foo to pack into transaction set. > kmod-foo-1.1-2.2.6.17_1.2 100% |=========================| 2.4 kB 00:00 > ---> Package kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5 set to be installed > --> Running transaction check > > Dependencies Resolved > > ============================================================================= > Package Arch Version Repository Size > ============================================================================= > Installing: > kmod-foo i686 1.1-2.2.6.17_1.2174_FC5 testrepo 504 k > > Transaction Summary > ============================================================================= > Install 1 Package(s) > Update 0 Package(s) > Remove 0 Package(s) > Total download size: 504 k > Is this ok [y/N]: y > Downloading Packages: > Running Transaction Test > Finished Transaction Test > Transaction Test Succeeded > Running Transaction > Installing: kmod-foo ######################### [1/1] > WARNING: Module /lib/modules/2.6.17-1.2174_FC5/extra/foo/foo.ko is not an elf object > > Installed: kmod-foo.i686 0:1.1-2.2.6.17_1.2174_FC5 > Complete! > [thl@notebook ~]$ Disabling the installonly-plungin in /etc/yum/pluginconf.d/installonlyn.conf has the same effect. So the "old modules get removed on updates" problem in reality is a bug AFAICS. Or I'm probably doing something really wrong here -- but what? CU thl

1 0

Do we need a Rule "Docs should be packaged as %doc"?
by Thorsten Leemhuis 18 Aug '06

18 Aug '06

Hi! Subject says it all. Reason for this idea: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=199108#c33 Quote: >> > I will prefer not to move docs from /usr/share/gutenprint/doc to >> > /usr/share/doc/gutenprint. >> >> Hmmm, we don't seem to have anything in the guidelines for this AFAICS. But IMHO >> all docs should be marked as %doc and thus should land in >> /usr/share/doc/<packagename-version-release> (the proper place used by all other >> packages) >> >> Maybe we need to add such a rule :-/ > > Sure if you think like that. Primary looking at package said me that let that > doc files be in /usr/share/gutenprint/doc > Then i check under /usr/share on my system using > find . -name doc * and i got following output > ./sane/xsane/doc > ./cups/doc > ./apps/quanta/doc > ./sgml/docbook/xsl-stylesheets-1.69.1-5/htmlhelp/doc > ./scrollkeeper/doc > ./vim/vim70/doc > ./eclipse/plugins/org.python.pydev_0.9.3/PySrc/ThirdParty/logilab/common/doc > ./eclipse/plugins/org.python.pydev_0.9.3/PySrc/ThirdParty/logilab/pylint/doc > ./pear/doc > ./gutenprint/doc > where some of the entries belongs to Fedora Core packages. > > So it looks to me that either we have different strategy for Fedora Extras or we > have some Guidelines that will require a major changes when a package moves from > Fedora Extras to Fedora Core. Then i would like to see that Guidelines page. CU thl

5 5

mixed-use-of-spaces-and-tabs
by Jason L Tibbitts III 18 Aug '06

18 Aug '06

Does anyone else find this rpmlint warning annoying? Do we really care that much about how packagers make things line up, or whether they make things line up at all? I can see a problem with mixing tabs and spaces on a single line, but the warning triggers when it finds both three consecutive spaces and a single tab anywhere in the specfile. - J<

4 4

Friday 1PM EDT IRC Meeting
by Warren Togami 18 Aug '06

18 Aug '06

It seems that most of the stakeholders would prefer an IRC meeting instead of a teleconference call. We will meet in #fedora-packaging at 1PM EDT Friday (same time of day as FESCO meetings) to discuss this topic. Warren Togami wtogami(a)redhat.com

2 1

The .pc and pkgconfig issue
by Jason L Tibbitts III 17 Aug '06

17 Aug '06

I know a month or so ago we voted to add a bit to the guidelines reinforcing the necessity of -devel packages with .pc files requiring pkgconfig. I went to file a bug about a core package that wasn't doing this (which resulted in bustage of a package I was reviewing): https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=202280 It seems Matthias Clasen (mclasen(a)redhat.com) wants to debate this and I'm not sure what to do. I don't have any arguments other than directory ownership and "that's what the committee decided". - J<

4 4

Re: Attention kernel module project packagers!
by Thorsten Leemhuis 16 Aug '06

16 Aug '06

David Woodhouse schrieb: > On Tue, 2006-08-15 at 09:50 -0400, Jesse Keating wrote: >> Given that we don't want it on Core or Extras, I'm pretty happy to >> let random 3rd party packager do whatever they want for packaging >> modules. I'm not interested in dictating how they should handle >> this ugly hack. >> >> Your example about ntfs is not usable w/out the userland >> (ntfsprogs), which nobody wants to touch due to legal reasons, and >> would be obsoleted by FUSE anyway where the most recent ntfs >> support is done entirely in userspace. >> >> There are many more things the packaging committee can spend time >> worrying about. Packaging of kernel modules isn't one of them >> IMHO. > > Yeah, that's a fair point. However, it would be useful if those who > _do_ care about kernel module packages would come to an agreement > about how it should be done, and that can be documented somewhere > central to Fedora -- like on the Fedora wiki. > > We can modify our kernel RPM and yum if appropriate in order to > support that agreed method. That already happend -- FESCo worked out and agreed on a propsoal last winter http://www.fedoraproject.org/wiki/Packaging/KernelModules It's working fine. CU thl

9 11

Re: atrpms kernel modules
by Axel Thimm 16 Aug '06

16 Aug '06

On Tue, Jul 18, 2006 at 01:59:34PM +0200, Eric Tanguy wrote: > > kernels=`rpm -qf /boot/vmlinuz-* | grep -v "^file .* is not owned by any package"` > > uname_rs=`rpm -ql $kernels | grep ^/boot/vmlinuz- | sed -e's,^/boot/vmlinuz-$.*$$,\1,'` > > for kmdl in `rpm -qa \*kmdl\* | sed -e's,-kmdl-.*,-kmdl,' | sort -u`; > > do > > for uname_r in $uname_rs; do > > package=${kmdl}-$uname_r > > rpm -q $package > /dev/null 2>&1 || echo $package > > done > > done | xargs -r smart install -y > > Ok this script seems to use smart. is it working with yum ? Yes, I think "smart install -y" -> "yum -y install" is the only change needed. And for apt it would be "apt-get -y install". > How is this problem handled by livna because it works fine using > this repo : the kernel modules are installed and not updated > It's known that there are issues with this scheme, or any scheme > that will merge module and kernel versions. rpm for one can't cope with it. Suppose you have two active kernels (say 2.6.16 and 2.6.17 or xen0 and non-xen0 etc.) and already have foo-kmdl for both in version 1. Now foo-kmdl in version 2 is released. Both rpm -i (coinstall, no replacement of of foo-kmdl for the same kernel) and rpm -U (remove all foo-kmdl but the highest one, e.g. at least one kernel stay w/o any kmdl) won't work. yum/smart/apt can be tought to be more clever than rpm and try to do the proper thing, which they will only be able to do if the merged versions are distangled again, e.g. hidden in Provides: and the like. So in order for a merged-version scheme to work there is special handling neccessary (and for atrpms' system, too, but read on). I prefer to have rpm itself already do something sensible and since it can only upgrade one versioning system (e.g. either the kernel's or the module's) it's preferable to allow it to upgrade the module within the same kernel and not touch the cross-kernel boundaries. "No harm done" like accidentially removing kmdls for other kernels or coinstalling several kmdls for the same kernel. You "only" need to reinstall the missing kmdl upon each kernel upgrade. It's far easier to get this into the brains of yum/smart/apt (e.g. translate the 9 lines of bash above into plugings/hooks for these depsolvers) or simply use the 9liner above than have a scheme which is broken at the level of rpm. Incidentially one on my personal targets is to get thie discussed in the fedora packaging group and review the currently scheme. Of course, I'm proposing adoption of ATrpms' kmdl scheme :) http://www.fedoraproject.org/wiki/Packaging/GuidelinesTodo > when i install a new kernel and are removed when the kernel is > removed. How the dependencies to the kernel are installed (which is responsible for having an automatic removal on kernel removal) is a different story, which the version merging does not break. The issue is with the system (system=rpm and(or any higher depsolver tool, e.g. yum/smart/apt) being able to distinguish different foo-kmdls as _upgrades_ vs _coinstalls_. -- Axel.Thimm at ATrpms.net

11 63

Re: Attention kernel module project packagers!
by Jesse Keating 15 Aug '06

15 Aug '06

On Tuesday 15 August 2006 09:40, David Woodhouse wrote: > I'm not necessarily suggesting that we shouldn't have an agreed method > of building kernel module packages at all -- just that we shouldn't have > any such packages in Core or Extras. > > There _are_ relatively sane (and legal, unlike nvidia/ati stuff) cases > where one might want to build a separate module -- like the NTFS modules > in Livna, for example. And other 'new drivers' which aren't yet > upstream. Of course you're right when you agree with me that those new > drivers shouldn't be in Core or Extras -- but that doesn't mean we > shouldn't provide a way to package them at all. Given that we don't want it on Core or Extras, I'm pretty happy to let random 3rd party packager do whatever they want for packaging modules. I'm not interested in dictating how they should handle this ugly hack. Your example about ntfs is not usable w/out the userland (ntfsprogs), which nobody wants to touch due to legal reasons, and would be obsoleted by FUSE anyway where the most recent ntfs support is done entirely in userspace. There are many more things the packaging committee can spend time worrying about. Packaging of kernel modules isn't one of them IMHO. -- Jesse Keating Release Engineer: Fedora

1 0

Re: Attention kernel module project packagers!
by Jesse Keating 15 Aug '06

15 Aug '06

On Tuesday 15 August 2006 08:48, David Woodhouse wrote: > I think this argument is invalid. We should ban _all_ module packages > from Core and Extras anyway. I'm with David on this one. Packaging of modules is the path to insanity. It requires all kinds of weird hacks to how they are built, how they are named, how they are handled by rpm and depsolvers, they always lag, they always hold users behind when critical kernel fixes come out, etc, etc, etc... I personally feel (as does David) that if the module isn't good enough for upstream, then it would HAVE to live in the kernel package itself. If it's not good enough for the kernel package itself, then it isn't good enough for Fedora. (Same could be applied to RHEL, but that's a battle for a different list). Arguing over which ugly ass hack to apply to be able to package kernel modules is a bikeshed argument. -- Jesse Keating Release Engineer: Fedora

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

packaging August 2006