buildsys October 2010

buildsys@lists.fedoraproject.org

12 participants
11 discussions

[BUG] Koji and Lustre
by Thomas Guthmann 09 Sep '11

09 Sep '11

Hey guys, We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py) when used with a Lustre filesystem. This function parses all attributes of every file of a build and is used when you want to display build.log/root.log through the web interface. Everything returned by listTaskOutput is returned through XML-RPC and as a result we had this : An error has occurred while processing your request. Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC limits'> Traceback (most recent call last): File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in publish_object return old_publish_object(req, object) File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py", line 412, in publish_object return publish_object(req,util.apply_fs_data(object, req.form, req=req)) File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line 439, in apply_fs_data return object(**args) File "/usr/share/koji-web/scripts/index.py", line 649, in getfile output = server.listTaskOutput(taskID, stat=True) File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468, in __call__ return self.__func(self.__name,args,opts) File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718, in _callMethod raise err Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC limits'> The issue comes from the st_dev value gathered by getStat (stat). In Lustre this value can be very high and that's why it complains. To fix that, we used the same condition than st_size, we cast the value as a string. See attached patch. Example (see 'Device:' the value after the '/'): # stat build.log File: `build.log' Size: 106021 Blocks: 208 IO Block: 2097152 regular file Device: e04ae70eh/3763005198d Inode: 721664 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache) Access: 2010-07-13 10:44:40.000000000 +1000 Modify: 2010-07-12 12:54:15.000000000 +1000 Change: 2010-07-12 12:54:52.000000000 +1000 # That's what is read by listTaskOutput relfilename=build.log ATTR=st_atime GETATTR=1278981616 relfilename=build.log ATTR=st_blksize GETATTR=2097152 relfilename=build.log ATTR=st_blocks GETATTR=208 relfilename=build.log ATTR=st_ctime GETATTR=1278903292 relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\ relfilename=build.log ATTR=st_gid GETATTR=48 relfilename=build.log ATTR=st_ino GETATTR=721664 relfilename=build.log ATTR=st_mode GETATTR=33188 relfilename=build.log ATTR=st_mtime GETATTR=1278903255 relfilename=build.log ATTR=st_nlink GETATTR=1 relfilename=build.log ATTR=st_rdev GETATTR=0 relfilename=build.log ATTR=st_size GETATTR=106021 relfilename=build.log ATTR=st_uid GETATTR=48 Hope it helps, lost a good amount of time on that one :) Cheers, Thomas

2 3

Strange mock build failure due to typo
by Paul Howarth 24 Jan '11

24 Jan '11

Hi, today I have been preparing an update to perl-Math-Pari and came across a very strange build failure whilst doing a local mock build on a Fedora 13 x86_64 host. My package built successfully on x86_64 but when I tried to build for i386, the build failed during %setup but without any diagnostics. The SRPM was installed but no attempt to install its build requirements was made. The root.log showed an exit status of 0 for all commands that had been run. After much experimentation bisecting the changes I had made, I discovered that a typo in the changelog entry was the culprit: I had set the year to 2100 instead of 2010. So it would appear that somewhere in the mock/yum/rpm stack there may be a year 2038 problem waiting to bite us (though I suspect there may not be too many 32-bit builds happening by then). Seriously though, it would be nice to have better diagnostics for this and perhaps an rpmlint check for changelog entries in the future? Paul.

2 1

[PATCH 1/3] Install build deps with yum-builddep.
by Ville Skyttä 04 Dec '10

04 Dec '10

No longer need to screen-scrape resolvedep and feed that to yum install, and we have a chance to get BuildConflicts handing "for free" (when RHBZ #614191 is done in yum(-builddep)). --- mock.spec.in | 2 +- py/mock/backend.py | 33 ++++++++++++++++++++++----------- 2 files changed, 23 insertions(+), 12 deletions(-) diff --git a/mock.spec.in b/mock.spec.in index e2580ca..0949975 100644 --- a/mock.spec.in +++ b/mock.spec.in @@ -18,7 +18,7 @@ Source: https://fedorahosted.org/mock/attachment/wiki/MockTarballs/%{name}-%{ver URL: http://fedoraproject.org/wiki/Projects/Mock BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n) BuildArch: noarch -Requires: python >= 2.4, yum >= 2.4, tar, pigz, python-ctypes, python-decoratortools, usermode +Requires: python >= 2.4, yum >= 2.4, yum-utils >= 1.1.9, tar, pigz, python-ctypes, python-decoratortools, usermode Requires: createrepo Requires(pre): shadow-utils BuildRequires: python-devel diff --git a/py/mock/backend.py b/py/mock/backend.py index ba0ad14..81a98d0 100644 --- a/py/mock/backend.py +++ b/py/mock/backend.py @@ -69,6 +69,7 @@ class Root(object): self.chroot_file_contents = config['files'] self.chroot_setup_cmd = config['chroot_setup_cmd'] self.yum_path = '/usr/bin/yum' + self.yum_builddep_path = '/usr/bin/yum-builddep' self.macros = config['macros'] self.more_buildreqs = config['more_buildreqs'] self.cache_topdir = config['cache_topdir'] @@ -444,23 +445,28 @@ class Root(object): """figure out deps from srpm. call yum to install them""" try: self.uidManager.becomeUser(0, 0) + + def _yum_and_check(cmd): + output = self._yum(cmd, returnOutput=1) + for line in output.split('\n'): + if line.lower().find('No Package found for'.lower()) != -1: + raise mock.exception.BuildError, "Bad build req: %s. Exiting." % line + + # first, install pre-existing deps and configured additional ones arg_string = self.preExistingDeps for hdr in mock.util.yieldSrpmHeaders(srpms, plainRpmOk=1): # get text buildreqs - a = mock.util.requiresTextFromHdr(hdr) - b = mock.util.getAddtlReqs(hdr, self.more_buildreqs) - for item in mock.util.uniqReqs(a, b): + for item in mock.util.getAddtlReqs(hdr, self.more_buildreqs): arg_string = arg_string + " '%s'" % item - - # everything exists, okay, install them all. - # pass build reqs (as strings) to installer if arg_string != "": - output = self._yum('resolvedep %s' % arg_string, returnOutput=1) - for line in output.split('\n'): - if line.lower().find('No Package found for'.lower()) != -1: - raise mock.exception.BuildError, "Bad build req: %s. Exiting." % line + # everything exists, okay, install them all. + # pass build reqs (as strings) to installer + _yum_and_check('resolvedep %s' % arg_string) # nothing made us exit, so we continue self._yum('install %s' % arg_string, returnOutput=1) + + # install actual build dependencies + _yum_and_check("builddep '%s'" % "' '".join(srpms)) finally: self.uidManager.restorePrivs() @@ -676,7 +682,12 @@ class Root(object): if not self.online: cmdOpts = "-C" - cmd = '%s --installroot %s %s %s' % (self.yum_path, self.makeChrootPath(), cmdOpts, cmd) + # invoke yum-builddep instead of yum if cmd is builddep + exepath = self.yum_path + if cmd.startswith("builddep "): + exepath = self.yum_builddep_path + cmd = cmd[len("builddep "):] + cmd = '%s --installroot %s %s %s' % (exepath, self.makeChrootPath(), cmdOpts, cmd) self.root_log.debug(cmd) output = "" try: -- 1.7.2.3

3 9

Koji policies
by ed_pollard＠us.ibm.com 26 Oct '10

26 Oct '10

I am attempting to setup koji in a test environment and would like to allow users to add packages to tags without having admin rights. I found http://fedoraproject.org/wiki/Koji/Policies but have been unable to configure a policy for my purposes. Any help would be greatly appreciated. ------ Edward Pollard ed_pollard(a)us.ibm.com

2 3

Mock alternating /etc/php.ini file
by David Hrbáč 20 Oct '10

20 Oct '10

Hi, I have the following scenario. I'm trying to build some php-pear package with Mock for CentOS 4.x. PHP settings within php.ini has memory_limit = 8MB. This memory limit is too low to build the package. I don't want to change php package itself. I have tried: a) package with sed to replace memory setting - fails with access denied on /etc/php.ini b) package containing own /etc/php.ini - conflicts with php-common Any ideas? Thanks, David Hrbáč

2 2

cannot open Providename index using db3 - Invalid argument (22)
by Vijay N. Majagaonkar 18 Oct '10

18 Oct '10

Hi, I am running out of strange problem at the time of creating ISO, ISO created successfully but it failed at the time of OS installation and this what i found in root.log file I am not able to under stand the what is the problem "cannot open Providename index using db3 - Invalid argument (22)" can someone please give me some point or direction to fix this, [Log] DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building isolinux directory DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: mkdosfs 2.11 (12 Mar 2005) DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb: /var/lib/rpm/Providename: unsupported hash version: 9 DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot open Providename index using db3 - Invalid argument (22) DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb: /var/lib/rpm/Providename: unsupported hash version: 9 DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot open Providename index using db3 - Invalid argument (22) DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: http://koji.fedora.com/packages/kernel/2.6.18/164.el5/x86_64/kernel-xen-2.6…: [Errno 12] Timeout: <urlopen error timed out> DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Trying other mirror. DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Traceback (most recent call last): DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File "/usr/bin/yumdownloader", line 293, in ? DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: util = YumDownloader() DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File "/usr/bin/yumdownloader", line 42, in __init__ DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: self.main() DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File "/usr/bin/yumdownloader", line 80, in main DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: self.downloadPackages(opts) DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File "/usr/bin/yumdownloader", line 214, in downloadPackages DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: path = repo.getPackage(download) DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File "/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 849, in getPackage DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: cache=cache DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File "/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 811, in _getFile DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: raise Errors.RepoError, errstr DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: yum.Errors.RepoError: failed to retrieve kernel/2.6.18/164.el5/x86_64/kernel-xen-2.6.18-164.el5.x86_64.rpm from anaconda-extrarepo-1 DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error was [Errno 12] Timeout: <urlopen error timed out> DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: kernel (kernel-xen) doesn't exist for x86_64. skipping DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building minstg.img [/Log] Thanks for the help V!jay

2 3

pungi error, cannot find xorg
by Narendra Sisodiya 16 Oct '10

16 Oct '10

Please have a look, I have attached error logs at http://gist.github.com/629446 -- ┌─────────────────────────┐ │ Narendra Sisodiya │ http://narendrasisodiya.com └─────────────────────────┘

1 2

Mock Error
by Narendra Sisodiya 15 Oct '10

15 Oct '10

creating 32 bit iso file on 64 bit machine using pungi and mock. but following error I am getting. [root@narendra mock]# mock -r fedora-13-i386 --clean INFO: mock.py version 1.1.5 starting... State Changed: init plugins INFO: selinux enabled State Changed: start State Changed: lock buildroot State Changed: clean [root@narendra mock]# mock -r fedora-13-i386 --init INFO: mock.py version 1.1.5 starting... State Changed: init plugins INFO: selinux enabled State Changed: start State Changed: lock buildroot State Changed: clean State Changed: init State Changed: lock buildroot Mock Version: 1.1.5 INFO: Mock Version: 1.1.5 INFO: enabled root cache State Changed: unpacking root cache INFO: enabled yum cache State Changed: cleaning yum metadata INFO: enabled ccache State Changed: running yum ERROR: Command failed. See logs for output. # ['/usr/sbin/groupadd', '-g', '478', 'mockbuild'] [root@narendra mock]# -- ┌─────────────────────────┐ │ Narendra Sisodiya │ http://narendrasisodiya.com └─────────────────────────┘

3 6

Pungi Error
by Narendra Sisodiya 10 Oct '10

10 Oct '10

I am running fedora 13 64 bit system I am trying to run pungi. $pungi -c test.ks I am getting this error /usr/lib/python2.6/site-packages/pykickstart/parser.py:673: DeprecationWarning: %packages does not end with %end. This syntax has been deprecated. It may be removed from future releases, which will result in a fatal error from kickstart. Please modify your kickstart file to use this updated syntax. warnings.warn(_("%s does not end with %%end. This syntax has been deprecated. It may be removed from future releases, which will result in a fatal error from kickstart. Please modify your kickstart file to use this updated syntax.") % "%packages", DeprecationWarning) Warning: Reusing existing destination directory. Traceback (most recent call last): File "/usr/bin/pungi", line 216, in <module> main() File "/usr/bin/pungi", line 86, in main mypungi._inityum() # initialize the yum object for things that need it File "/usr/lib/python2.6/site-packages/pypungi/__init__.py", line 166, in _inityum del self.ayum.prerepoconf AttributeError: prerepoconf -- ┌─────────────────────────┐ │ Narendra Sisodiya │ http://narendrasisodiya.com └─────────────────────────┘

3 5

Fwd: Creating Install media of fedora.
by Narendra Sisodiya 07 Oct '10

07 Oct '10

What are the tools available to install media. I want to create custom fedora 13 install DVD (not live). Also, which fedora version or ubuntu version you recommend to do this task, -- ┌─────────────────────────┐ │ Narendra Sisodiya │ http://narendrasisodiya.com └─────────────────────────┘

3 9

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys October 2010