buildsys February 2006

buildsys@lists.fedoraproject.org

14 participants
8 discussions

Start a nNew thread

Extras build system back up
by Dan Williams 26 Oct '06

26 Oct '06

Hi, We're back up and using the latest plague code. yum update your plague-client and you should be good to go. Dan

3 7

adding packages to mock chroot
by David Smith 27 Feb '06

27 Feb '06

When you create a mock chroot, all the packages you want to install in it need to be available from a yum repostory. I've got the situation where I'm trying to bootstrap a bit, so, I'd like to be able to: - build a chroot - build a small set of packages - add those packages to the chroot to be used by future builds Off the top of my head, possibilities include: (1) creating a base chroot, building the rpms, add the new packages to a yum repo, creating a second chroot that contains the new packages (2) creating the base chroot, building the rpms, editing the buildroots.xml file and adding in the new packages, updating the yum repo, then run "mock init" again (hoping that the new packages would get added to the chroot) Are there better ways of doing this? Thanks. -- David Smith dsmith(a)redhat.com Red Hat, Inc. http://www.redhat.com 256.217.0141 (direct) 256.837.0057 (fax)

3 2

bugzilla #164441 (mock-helper and basedir)
by Clark Williams 27 Feb '06

27 Feb '06

Hi all, I've been looking at how to change mock so that the build chroot's can be placed at arbitrary locations (as opposed to the current hardcoded /var/lib/mock). The main reason I want to do this is that I build target root filesystems for embedded targets in our compile farm and we use a big NFS filesystem as our main storage. I can access the build directory and our cross-toolchains from any system in our farm and nothing that's specific to a build is on any farm system (I can kickstart it at will and not lose anything). Yes, we've successfully redirected /var/lib/mock to the NFS storage with a mount --bind but I just find that kinda scary. I'm just not very confident in remounting an NFS filesystem to a local filesystem. The bottom line is that I'd like to remove dependencies on the build system's local filesystem and move it all to the NFS storage. So, the problem seems to be how to modify mock-helper so that it allows the move, yet doesn't become a handy-dandy-attack-point for some cracker wannabe. Can we say that the following is true? "The mock-helper program always begins execution *outside* of a chroot and is the only mechanism mock uses to *enter* a chroot." If so, then we can check to insure that the intended chroot directory: 1. Is not '/' 2. Is not a symlink to '/' 3. Is not in a list of special directories (/bin, /sbin, /lib, etc.) Are we "safe" if we insure that whatever command is executed by mock-helper is executed in a chroot directory that does not contain a system critical component? Clark -- Clark Williams <williams(a)redhat.com>

6 11

kmod arch, ExlusiveArch, and buildsys?
by Warren Togami 17 Feb '06

17 Feb '06

Hi Dan, I am supposed to ask you about your opinion about how to ask buildsys to build for specific archs for a kernel module. I believe that we should use ExclusiveArch with explicit listed archs, and buildsys should loop through each listed arch and build each. This would also work for multi-arch performance intensive userspace packages similar to what we do in glibc or openssl in Core. [1] Contributors may also list any other arch like s390x or alpha in ExclusiveArch, and buildsys simply ignores it if it isn't supported in Extras. I believe this is the best approach to take for Extras, because Core has been similar to this for many years. Do you have any opinion on this, or know of a better way to handle this? Here is some related discussion: <scop> the problem with exclusivearch is that let's say you list i586 i686 x86_64 ppc and someone wants to rebuild it for alpha -> won't build, even if the package/software would work <warren> scop, you can include other archs in the list, buildsys can simply ignore Extras non-supported archs. <scop> warren, it's abuse of the tag nevertheless <warren> scop, I'd personally disagree, but either way this is very simple and it already behaves similar to this in Core literally forever. <warren> more importantly the simplicity of this allows us to move forward sooner <warren> (assuming dcbw doesn't have a better idea) <scop> that I agree with Warren Togami wtogami(a)redhat.com [1] However we should disallow this, unless someone proves through benchmarks that there is a huge benefit of publishing both i386 and i686 packages and our tools support selection of the right arch directly. Currently I believe it doesn't.

3 2

Re: Sugg: Improve for mock (caching)
by Enrico Scholz 17 Feb '06

17 Feb '06

skvidal(a)phy.duke.edu (seth vidal) writes: >> The speedup gained by caching is not (primarily) related to the >> network connection but to the rpm installation and disk-IO. On the >> fedora.us server we used caching also and the initial buildroot was >> created in <1min (afair). >> >> Local tests at that time showed, that the initial buildroot can be >> created in 20 seconds when a tmpfs will be used instead of a real disk >> with ext3. > > So the numbers I'm curious about that you may have: > - remote repos - but lan-like connection times > - local-disk repos - chroot creation - all on ext3 > - tmpfs targets for the chroot creation and repos > > do you have anything like that? ok, a little bit late, but I implemented caching now so I can provide some real numbers: Traditional 'mock init' on a tmpfs takes | Init buildroot... | | real 1m37.445s | user 1m15.329s | sys 0m8.009s tar'ing this filesystem, and extracting it takes | Extracting cached buildroot... | | real 0m1.266s | user 0m0.188s | sys 0m0.944s Both operations are resulting into the same root-fs; the executed code was: | mock=( env http_proxy= mock --resultdir="$results" -r "$MOCK_ROOT" ) | mock_yum=( env http_proxy= /usr/sbin/mock-helper yum --installroot "$BUILD_DIR" ) | | function initBuildroot() { | "${mock[@]}" clean >/dev/null | | if test -e "$CACHE_NAME"; then | echo "Extracting cached buildroot..." | time /usr/sbin/mock-helper uncache "$MOCK_ROOT" | | echo "Updating metadata..." | "${mock_yum[@]}" check-update || : | else | lock | echo "Init buildroot..." | time "${mock[@]}" init >/dev/null | unlock | | echo "Cleaning up root..." | "${mock_yum[@]}" clean packages headers | echo "Creating cache..." | /usr/sbin/mock-helper cache "$MOCK_ROOT" | fi | } The 'cache' and 'uncache' operations of 'mock-helper' were added by me and are simple 'tar' wrappers. The resulting tarball is uncompressed and 280MB sized, both the rootfs and the tarball are located in the tmpfs. Environment: mock-machine: Intel Celeron 2.80GHz, 1500MB RAM repository-server: PII 266MHz, 196MB RAM, http-access to repo (Apache2) 100Mb/s LAN; 'wget' download rate between repo and mock machine appr. 7-8MB/s Enrico

2 1

Kernel-Modules in the buildsys
by Thorsten Leemhuis 13 Feb '06

13 Feb '06

Hi all! The kernel-module discussion on fedora-extras list was quite successful and we have a kernel-module scheme now where nobody complained loudly (at least until now). Two examples that are based on the latest results from the discussion are found in extras-cvs in the packages lirc-kmod and thinkpad-kmod. What we don't have yet: Support for the that kernel-module scheme in the buildsystem. What's needed? Roughly this: - we need to get the kernel-verrel to the spec-file/rpmbuild somehow - we need to get the kernel-variants to the spec-file/rpmbuild somehow Example: To rebuild lirc-kmod for the kernel 2.6.15-1.1831_FC4 and 2.6.15-1.1831_FC4smp on i686 one would run this command with the current scheme: $ rpmbuild --rebuild lirc-kmod-0.8.0-2.2.6.15_1.1831_FC4.src.rpm --define 'kver 2.6.15-1.1831_FC4' --define 'kvariants "" smp ' --target i686 So guys, how to we teach that the buildsystem? Do we handle that in mock? Or in plague? As a yum-plugin? I don't know plague and mock to much, so any help/suggestions appreciated! /me looks at the command and wonders why we use "kver" for kversion -- we either should use "kver" and "kvars" or "kversion" and "kvariants" -- Thorsten Leemhuis <fedora(a)leemhuis.info>

4 3

x86_64-devel koffice builds fail randomly (on random builders)
by Andreas Bierfert 09 Feb '06

09 Feb '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, when I pushed koffice 1.4.90-{1,2} everything worked fine. Not there was a bug with some desktop file and I fixed it by adding an %exclude to the file section (so no changes in the sources or the build part or the alike) but now the build keeps failing randomly. See the logs from try 2-5 (lost the first one). Any help would be very much welcome... - - Andreas This is from the 2nd try: creating libkiviocommon_la.all_cpp.cpp ... /bin/sh ../../libtool --silent --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. - -I. -I../.. -I./ui -Iui -I./config -Iconfig -I./kiviosdk -I./tools - -I../../lib/kofficeui -I../../lib/kofficeui -I../../lib/kofficecore - -I../../lib/kofficecore -I../../lib/store -I../../lib/store -I../../lib/kwmf - -I../../lib/kwmf -I../../lib/kopalette -I../../lib/kopalette - -I../../lib/kopalette -I../../lib/kopalette -I../../lib/kotext - -I../../lib/kotext -I../../lib/kformula -I/usr/include/kde - -I/usr/lib64/qt-3.3/include -I. -DQT_THREAD_SUPPORT -D_REENTRANT - -Wno-long-long -Wundef -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align - -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall - -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 - -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute - -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common - -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANS LATION -DHAVE_KNEWSTUFF -c -o libkiviocommon_la.all_cpp.lo libkiviocommon_la.all_cpp.cpp /bin/sh ../../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long -Wundef - -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion - -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall - -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 - -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute - -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common - -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT - -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -o libkiviocommon.la -rpath /usr/lib64 - -L/usr/lib64/qt-3.3/lib -L/usr/lib64 -L/usr/lib64 -no-undefined - -Wl,--no-undefined -Wl,--allow-shlib-undefined libkiviocommon_la.all_cc.lo libkiviocommon_la.all_cpp.lo ../../kivio/kiviopart/tools/libtools.la ../../kivio/kiviopart/kiviosdk/libkiviosdk.la ../../kivio/kiviopart/config/libkivioconfig.la ../../kivio/kiviopart/ui/libui.la ../../lib/kofficeui/libkofficeui.la ../../lib/kopainter/libkopainter.la ../../lib /kopalette/libkopalette.la ../../lib/kotext/libkotext.la -lpython2.4 -ldl -ldl -L/usr/lib64 : : No such file or directory : : No such file or directory libtool: link: `/usr/lib64/libkutils.la' is not a valid libtool archive make[4]: *** [libkiviocommon.la] Error 1 make[4]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kivio/kiviopart' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kivio/kiviopart' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kivio' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90' make: *** [all] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.25214 (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.25214 (%build) 3rd try: /bin/sh ../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long -Wundef - -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion - -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall - -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 - -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute - -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common - -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT - -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -o kpresenter -no-undefined - -L/usr/lib64/qt-3.3/lib -L/usr/lib64 -L/usr/lib64 kpresenter.la.o libkdeinit_kpresenter.la libtool: link: `/usr/lib64/libkdecore.la' is not a valid libtool archive make[3]: *** [kpresenter] Error 1 make[3]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kpresenter' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kpresenter' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90' make: *** [all] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.96654 (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.96654 (%build) 4th try: /bin/sh ../../../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long - -Wundef -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion - -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall - -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 - -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute - -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common - -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT - -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -fexceptions -o krossruby.la -rpath /usr/lib64/kde3 -module -lruby -L/usr/lib64/qt-3.3/lib -L/usr/lib64 - -L/usr/lib64 -avoid-version -module -no-undefined -Wl,--no-undefined - -Wl,--allow-shlib-undefined -version-info 1:0:0 krossruby_la.all_cpp.lo - -lqt-mt -lz -lpng -lz -lm -lXext -lX11 -lSM -lICE -lpthread -lkdecore ../api/libkrossapi.la ../main/libkrossmain.la libtool: link: `/usr/lib64/libkio.la' is not a valid libtool archive make[4]: *** [krossruby.la] Error 1 make[4]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/lib/kross/ruby' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/lib/kross' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/lib' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90' make: *** [all] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.10097 (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.10097 (%build) 5th try: bin/sh ../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long -Wundef - -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion - -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall - -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 - -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute - -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common - -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT - -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -o libkdgantt.la - -L/usr/lib64/qt-3.3/lib -L/usr/lib64 -L/usr/lib64 -no-undefined - -Wl,--no-undefined -Wl,--allow-shlib-undefined libkdgantt_la.all_cpp.lo -lkdeui libtool: link: `/usr/lib64/libkdefx.la' is not a valid libtool archive make[2]: *** [libkdgantt.la] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kdgantt' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90' make: *** [all] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.40707 (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.40707 (%build) - -- Andreas Bierfert | http://awbsworld.de | GPG: C58CF1CB andreas.bierfert(a)lowlatency.de | http://lowlatency.de | signed/encrypted phone: +49 2402 102373 | cell: +49 172 9789968 | mail preferred -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFD6wANQEQyPsWM8csRAsPMAJwIzdk8K2zR9AfJRkWnpkhwtN206wCfXBwJ sc6WBZYbexp7WyxRLduvyJc= =GFC8 -----END PGP SIGNATURE-----

1 0

pthread_sigmask module
by Jeff Sheltren 01 Feb '06

01 Feb '06

Hi, I am having some problems with the build server hanging (ie. there are no jobs running, but when I submit a job it sits with status 'waiting')... Since this is running on EL4 (python 2.3.4), I'm getting this message when I start up plague-builder: WARNING: You might want to install the pthread_sigmask module. Python versions earlier than 2.4 have problems with signals and spawned processes. I'm guessing this is related to my problem :) So the question is, where can I find this module? Thanks, Jeff

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys February 2006