packaging July 2007

packaging@lists.fedoraproject.org

31 participants
24 discussions

smp_flags considered dangerous?
by Axel Thimm 04 Jul '07

04 Jul '07

Well, we know smp_flags lead to undeterministic builds that lead to o uncomparable build logs o difficult to trace bugs in the the build output o undeterministic triggering of build system bugs I had vtk [1] in the queue for a year, and one of the reviewers' demand was to add smp_flags, since "it worked that well". I added this for the sake of getting it reviewed and once the package was reviewed it wouldn't build on Fedora builders. And it was quite non-reproducible on my 4-way system, probably due to different number of make threads, different timings etc. It was also non-reproducable on the Fedora builders - it would fail but at different build stages, so I started to check what BR changed between invocations and the like. The benefits of smp_flags is that some very big builds are taking less time. So if say openoffice uses them (I have no idea) it would make no sense to forbid them for these packages. But for most packages that usually build under 15 minutes on a serial invocation the drawbacks are higher than the benefits (like package rebuilds simply breaking for no apparent reason like vtk did). But as *recommendation* we should switch from endorsing it to questioning it and recommending not to use it. [1] https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=199405 -- Axel.Thimm at ATrpms.net

7 18

build ID -debuginfo rpm convention
by Roland McGrath 03 Jul '07

03 Jul '07

Hi folks. Please see http://fedoraproject.org/wiki/Releases/FeatureBuildId for background (and read through all that before asking any technical questions on the details aside from the rpm packaging convention). The upshot is that with F8 tools each binary linked will contain something akin to a UUID, and core dumps will include the IDs of the executable and DSOs in the crashed process. Married with some conventions about storing and looking up ID bits, this enables nice automagic debuggability things including e.g. yum install foo-debuginfo for all the foos in your core file. For rpm builds, the final ID bits are chosen by debugedit. BZ 246404 tracks the debugedit change, which has not gone in yet (just went into rpm5.org though ;-) http://people.redhat.com/roland/build-id/ has all the patches. Here I want to talk about those conventions. This will be an upstream convention that the consuming tools (debuggers et al) will have as default behavior, not just a convention for Fedora or rpm. But I want to reach consensus on the integrated plan for Fedora repos before proposing a particular convention upstream. See http://fedoraproject.org/wiki/RolandMcGrath/BuildID#symlinks for the details of my initial proposal. In short, the debuginfo rpm covering /usr/bin/debug would include two symlinks like these: /usr/lib/debug/.build-id/7c/763b567137492087bc1e50809b1218 -> ../../../bin/foo /usr/lib/debug/.build-id/7c/763b567137492087bc1e50809b1218.debug -> ../../usr/bin/foo Consumers will use the .build-id/xx/x... convention to find the binaries with ID bits xxx... Fancy consumers will do: yum install /usr/lib/debug/.build-id/7c/763b567137492087bc1e50809b1218.debug The basic filesystem part of this is simple and efficient to have as the new standard convention in upstream consumers. This will wind up with two symlinks per binary installed in /usr/{bin,sbin,lib} etc., which seems reasonable enough for the maximum size of /usr/lib/debug/.build-id/ directories in an installed system. Is this reasonable for the packaging in the whole-distro perspective? The issues I can imagine might be problems are: * adds two (long) file names per installed binary to debuginfo repo Is this going to be an unreasonable bloat on the filelist metadata? * danglink symlink from foo-debuginfo rpm to file in foo rpm Is this kosher? The big benefit of this particular convention is that it requires no new infrastructure work. The extent of the implementation work in the packaging sphere is a patch to find-debuginfo.sh (below). Please let me know ASAP what you think about this plan. Thanks, Roland diff -r 4178c07c98c8 scripts/find-debuginfo.sh --- a/scripts/find-debuginfo.sh Fri Jun 29 14:12:44 2007 +0300 +++ b/scripts/find-debuginfo.sh Sun Jul 01 16:11:35 2007 -0700 @@ -11,16 +11,30 @@ SOURCEFILE=$BUILDDIR/debugsources.list debugdir="${RPM_BUILD_ROOT}/usr/lib/debug" -echo -n > $SOURCEFILE +> $SOURCEFILE +> $LISTFILE strip_to_debug() { eu-strip --remove-comment -f "$1" "$2" || : } +debug_link() +{ + tgt="$1" + tgt="${tgt/#..\/..\/..\/..\/..\/usr\//../../../../}" + tgt="${tgt/#..\/..\/..\/..\/lib\//../../../}" + tgt="${1/#..\/..\/..\/debug\//../../}" + ln -snf "$tgt" "${debugdir}/$2" + echo >> $LISTFILE "/usr/lib/debug/$2" +} + # Strip ELF binaries -for f in `find $RPM_BUILD_ROOT ! -path "${debugdir}/*.debug" -type f $ -perm -0100 -or -perm -0010 -or -perm -0001 $ -exec file {} \; | \ - sed -n -e 's/^$.*$:[ ]*.*ELF.*, not stripped/\1/p'` +find $RPM_BUILD_ROOT ! -path "${debugdir}/*.debug" -type f \ + $ -perm -0100 -or -perm -0010 -or -perm -0001 $ \ + -print | +file -N -f - | sed -n -e 's/^$.*$:[ ]*.*ELF.*, not stripped/\1/p' | +while read f do dn=$(dirname $f | sed -n -e "s#^$RPM_BUILD_ROOT##p") bn=$(basename $f .debug).debug @@ -30,12 +44,23 @@ do [ -f "${debugfn}" ] && continue echo extracting debug info from $f - /usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug -l "$SOURCEFILE" "$f" + id=`/usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \ + -i -l "$SOURCEFILE" "$f"` || exit + if [ -z "$id" ]; then + echo >&2 "No build ID note found in $f" + # Demand build IDs in all binaries for Fedora rebuild. + # Comment out this exit to be less anal about it. + exit 2 + else + id="${id:0:2}/${id:2}" + debug_link "../../../../..$dn/$(basename $f)" ".build-id/${id}" + fi # A binary already copied into /usr/lib/debug doesn't get stripped, # just has its file names collected and adjusted. case "$dn" in - /usr/lib/debug/*) continue ;; + /usr/lib/debug/*) + continue ;; esac mkdir -p "${debugdn}" @@ -46,12 +71,15 @@ do strip_to_debug "${debugfn}" "$f" chmod u-w "$f" fi + + [ -z "$id" ] || debug_link "../..$dn/$bn" ".build-id/${id}.debug" done mkdir -p ${RPM_BUILD_ROOT}/usr/src/debug -cat $SOURCEFILE | (cd $RPM_BUILD_DIR; LANG=C sort -z -u | cpio -pd0mL ${RPM_BUILD_ROOT}/usr/src/debug) +LANG=C sort -z -u $SOURCEFILE | +(cd $RPM_BUILD_DIR; cpio -pd0mL ${RPM_BUILD_ROOT}/usr/src/debug) # stupid cpio creates new directories in mode 0700, fixup find ${RPM_BUILD_ROOT}/usr/src/debug -type d -print0 | xargs -0 chmod a+rx -find ${RPM_BUILD_ROOT}/usr/lib/debug -type f | sed -n -e "s#^$RPM_BUILD_ROOT##p" > $LISTFILE +find ${RPM_BUILD_ROOT}/usr/lib/debug -type f | sed -n -e "s#^$RPM_BUILD_ROOT##p" >> $LISTFILE find ${RPM_BUILD_ROOT}/usr/src/debug -mindepth 1 -maxdepth 1 | sed -n -e "s#^$RPM_BUILD_ROOT##p" >> $LISTFILE

3 2

Multiple version naming overly restrictive?
by Toshio Kuratomi 03 Jul '07

03 Jul '07

Hey all, I just noticed this section of the packaging Guidelines: http://fedoraproject.org/wiki/Packaging/NamingGuidelines#MultiplePackages says: ''' For many reasons, it is sometimes advantageous to keep multiple versions of a package in Fedora to be installed simultaneously. When doing so, the package name should reflect this fact. The most recent version of a package should use the base name with no versions, and all other addons should note their version in the name. ''' I think specifying that the latest package version has [basename] and the others have [basename][version] may be overly restrictive. We have precedent in gtk+ vs gtk2 (long term), gcc (3.x) vs gcc4 (short term), gtkhtml vs gtkhtml2 vs gtkhtml3 (long term) for doing things the other way. Do we really want to restrict this? If not, we could amend the text like this: ''' One package should use the base name with no versions and all other addons should note their version in the name. ''' -Toshio

2 1

Meeting today
by Axel Thimm 03 Jul '07

03 Jul '07

I will probably not make it to the meeting, but I'd like to propose to reverse the recommendation to use smp_flags with the opposite (make the opt-out an opt-in) for the reasons outlined. Count my +1 for that. -- Axel.Thimm at ATrpms.net

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

packaging July 2007