buildsys August 2007

buildsys@lists.fedoraproject.org

11 participants
13 discussions

No such file or directory: u'/var/tmp/yum-root-O3By9L/anaconda/headers/ifd-egate-0.05-17.i386.hdr'
by Sergio Monteiro Basto 15 Sep '07

15 Sep '07

Hi, I use updated fedora 7 what I could do ? pungi -c /etc/pungi/f7-everything.i386 stops width : INFO:pypungi.pungi:Running /usr/lib/anaconda-runtime/pkgorder /srv/pungi/f7/7/Everything/i386/os i386 Fedora ERROR:pypungi.pungi:Got an error from /usr/lib/anaconda-runtime/pkgorder ERROR:pypungi.pungi:Traceback (most recent call last): File "/usr/lib/anaconda-runtime/pkgorder", line 169, in <module> addGroups(ds, ["core", "base", "text-internet"]) File "/usr/lib/anaconda-runtime/pkgorder", line 105, in addGroups processTransaction(ds) File "/usr/lib/anaconda-runtime/pkgorder", line 77, in processTransaction ds.populateTs(keepold=0) File "/usr/lib/python2.5/site-packages/yum/depsolve.py", line 201, in populateTs self.downloadHeader(txmbr.po) File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 863, in downloadHeader cache=repo.http_caching != 'none', File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 599, in getHeader cache=cache, File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 567, in _getFile http_headers=headers, File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 411, in urlgrab return self._mirror_try(func, url, kw) File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 397, in _mirror_try return func_ref( *(fullurl,), **kwargs ) File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 927, in urlgrab return self._retry(opts, retryfunc, url, filename) File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 845, in _retry r = apply(func, (opts,) + args, {}) File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 915, in retryfunc fo._do_grab() File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 1196, in _do_grab else: new_fo = open(self.filename, 'wb') IOError: [Errno 2] No such file or directory: u'/var/tmp/yum-root-O3By9L/anaconda/headers/ifd-egate-0.05-17.i386.hdr' Thanks, -- Sérgio M. B.

6 23

Re: [patch 1/4] Remove Python 2.5 usage
by Jesse Keating 02 Sep '07

02 Sep '07

On Fri, 31 Aug 2007 17:00:46 +0100 Mark McLoughlin <markmc(a)redhat.com> wrote: > An empty list of base classes wasn't allowed before Python 2.5 so > remove it since it seems to be the only Python 2.5 feature being > used. So here is where I get a little curious. Much of the rest of pungi relies upon specific APIs in things like yum, createrepo, anaconda, pykickstart, etc.. Much of the requirements are on versions only available on platforms that have python2.5. As such, making the tip of pungi run on something other than python2.5 just seems bound to fail to me. You /really/ don't want to be composing one release from another. If you're going to compose FC6, you need to use the FC6 version of pungi to do it. Mock can help. The rest of the changes look fine and I'll add them shortly. Curious what you're attempting to do with pungi on a < python2.5 platform... I'm certainly not thinking about backwards compat when developing future versions of pungi to compose future versions of Fedora... -- Jesse Keating Fedora -- All my bits are free, are yours?

2 2

[patch 1/4] Remove Python 2.5 usage
by Toshio Kuratomi 31 Aug '07

31 Aug '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > An empty list of base classes wasn't allowed before Python 2.5 so > remove it since it seems to be the only Python 2.5 feature being > used. What version of python are you taking as your minimum? If it's python2.2 then you'd be better off inheriting from object. Inheriting from object makes this a new-style class which has some features and behaviors that can make it superior to old-style classes. Revised patch: Index: pungi/pypungi/__init__.py =================================================================== - --- pungi.orig/pypungi/__init__.py +++ pungi/pypungi/__init__.py @@ -16,7 +16,7 @@ import logging import os import subprocess - -class PungiBase(): +class PungiBase(object): """The base Pungi class. Set up config items and logging here""" def __init__(self, config): -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFG2ECxX6yAic2E7kgRAsNfAJwJXRMOdu6HjYzqjGxPO6xXxMNVVgCfd8L0 c/wgEOGTft0e9/CuNbGmu0Q= =JVoi -----END PGP SIGNATURE-----

2 1

Re: [patch 0/4] Some miscellanous pungi patches
by Mark McLoughlin 31 Aug '07

31 Aug '07

Hey, Oh, yes, I forgot to say ... pungi/share/comps-cleanup.xml seems to be missing from the repo and the 1.0.2 doesn't seem to have been pushed to the repo either? Cheers, Mark.

1 0

[patch 4/4] Add a Config class
by Mark McLoughlin 31 Aug '07

31 Aug '07

In order to use the gather module outside of pungi itself, you need to pass it a valid config object. To make that easier, this patch moves the config object definition from pungi into pypungi. Signed-off-by: Mark McLoughlin <markmc(a)redhat.com> Index: pungi/pungi =================================================================== --- pungi.orig/pungi +++ pungi/pungi @@ -13,14 +13,13 @@ # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. import os +import pypungi.config import pypungi.gather import pypungi.pungi import yum import pykickstart.parser import pykickstart.version -from ConfigParser import SafeConfigParser - def main(): # You must be this high to ride. @@ -28,54 +27,14 @@ def main(): print >> sys.stderr, "You must run pungi as root" return 1 - # Set some default variables, can be overrided in config file + config = pypungi.config.Config() - # Turn this into a dict someday, to iterate over when setting defaults - osdir = "os" - sourcedir = "source" - debugdir = "debug" - isodir = "iso" - cdsize = "4608.0" - relnotefilere = "eula.txt fedora.css GPL README-BURNING-ISOS-en_US.txt RELEASE-NOTES-en_US.html ^RPM-GPG" - relnotedirre = "images stylesheet-images" - relnotepkgs = "fedora-release fedora-release-notes" - - (opts, args) = get_arguments() + (opts, args) = get_arguments(config) # Set up the kickstart parser and pass in the kickstart file we were handed ksparser = pykickstart.parser.KickstartParser(pykickstart.version.makeVersion()) ksparser.readKickstart(opts.config) - config = SafeConfigParser() - config.add_section('default') - - # add some sections here before setting stuff. - - # hard coded non-options - config.set('default', 'osdir', osdir) - config.set('default', 'sourcedir', sourcedir) - config.set('default', 'debugdir', debugdir) - config.set('default', 'isodir', isodir) - config.set('default', 'relnotefilere', relnotefilere) - config.set('default', 'relnotedirre', relnotedirre) - config.set('default', 'relnotepkgs', relnotepkgs) - config.set('default', 'product_path', 'Packages') - config.set('default', 'cachedir', '/var/cache/pungi') - config.set('default', 'arch', yum.rpmUtils.arch.getBaseArch()) - - # set configs from cli options - config.set('default', 'name', opts.name) - config.set('default', 'version', opts.ver) - config.set('default', 'flavor', opts.flavor) - config.set('default', 'destdir', opts.destdir) - config.set('default', 'cachedir', opts.cachedir) - config.set('default', 'bugurl', opts.bugurl) - config.set('default', 'discs', opts.discs) - - # set some other defaults - config.set('default', 'iso_basename', config.get('default', 'name')) - - config.set('default', 'cdsize', cdsize) for part in ksparser.handler.partition.partitions: if part.mountpoint == 'iso': config.set('default', 'cdsize', part.size) @@ -148,27 +107,34 @@ def main(): if __name__ == '__main__': from optparse import OptionParser import sys - import time - - today = time.strftime('%Y%m%d', time.localtime()) - def get_arguments(): + def get_arguments(config): parser = OptionParser(version="%prog 1.0.1") + def set_config(option, opt_str, value, parser, config): + config.set('default', option.dest, value) + # Pulled in from config file to be cli options as part of pykickstart conversion - parser.add_option("--name", default="Fedora", dest="name", + parser.add_option("--name", dest="name", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='the name for your distribution (defaults to "Fedora")') - parser.add_option("--ver", default=today, dest="ver", + parser.add_option("--ver", dest="version", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='the version of your distribution (defaults to datestamp)') - parser.add_option("--flavor", default="", dest="flavor", + parser.add_option("--flavor", dest="flavor", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='the flavor of your distribution spin (optional)') - parser.add_option("--destdir", default=".", dest="destdir", + parser.add_option("--destdir", dest="destdir", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='destination directory (defaults to current directory)') - parser.add_option("--cachedir", default="/var/cache/pungi", dest="cachedir", + parser.add_option("--cachedir", dest="cachedir", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='package cache directory (defaults to /var/cache/pungi)') - parser.add_option("--bugurl", default="http://bugzilla.redhat.com", dest="bugurl", + parser.add_option("--bugurl", dest="bugurl", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='the url for your bug system (defaults to http://bugzilla.redhat.com)') - parser.add_option("--discs", default='1', dest="discs", + parser.add_option("--discs", dest="discs", type="string", + action="callback", callback=set_config, callback_args=(config, ), help='the number of discs you want to create (defaults to 1)') parser.add_option("--nosource", action="store_true", dest="nosource", help='disable gathering of source packages (optional)') Index: pungi/pypungi/config.py =================================================================== --- /dev/null +++ pungi/pypungi/config.py @@ -0,0 +1,45 @@ +#!/usr/bin/python -tt +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; version 2 of the License. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Library General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +import time +import yum + +from ConfigParser import SafeConfigParser + +class Config(SafeConfigParser): + def __init__(self): + SafeConfigParser.__init__(self) + + self.add_section('default') + + self.set('default', 'osdir', 'os') + self.set('default', 'sourcedir', 'source') + self.set('default', 'debugdir', 'debug') + self.set('default', 'isodir', 'iso') + self.set('default', 'relnotefilere', + 'eula.txt fedora.css GPL README-BURNING-ISOS-en_US.txt RELEASE-NOTES-en_US.html ^RPM-GPG') + self.set('default', 'relnotedirre', 'images stylesheet-images') + self.set('default', 'relnotepkgs', 'fedora-release fedora-release-notes') + self.set('default', 'product_path', 'Packages') + self.set('default', 'cachedir', '/var/cache/pungi') + self.set('default', 'arch', yum.rpmUtils.arch.getBaseArch()) + self.set('default', 'name', 'Fedora') + self.set('default', 'iso_basename', 'Fedora') + self.set('default', 'version', time.strftime('%Y%m%d', time.localtime())) + self.set('default', 'flavor', '') + self.set('default', 'destdir', '.') + self.set('default', 'bugurl', 'http://bugzilla.redhat.com') + self.set('default', 'discs', '1') + self.set('default', 'cdsize', '4608.0') + --

1 0

[patch 3/4] Re-factor package download code
by Mark McLoughlin 31 Aug '07

31 Aug '07

There are two copies of the package downloading code. Trivially merge them to ease maintenance. Signed-off-by: Mark McLoughlin <markmc(a)redhat.com> Index: pungi/pypungi/gather.py =================================================================== --- pungi.orig/pypungi/gather.py +++ pungi/pypungi/gather.py @@ -292,69 +292,69 @@ class Gather(pypungi.PungiBase): if not srpm in self.srpmlist: self.srpmlist.append(srpm) + def _link(self, local, target): + try: + os.link(local, target) + except OSError, e: + if e.errno != 18: # EXDEV + self.logger.error('Got an error linking from cache: %s' % e) + raise OSError, e - def downloadPackages(self): + # Can't hardlink cross file systems + shutil.copy2(local, target) + + def _downloadPackageList(self, polist, relpkgdir): """Cycle through the list of package objects and download them from their respective repos.""" - - downloads = [] - for pkg in self.polist: - downloads.append('%s.%s' % (pkg.name, pkg.arch)) - downloads.sort() - self.logger.info("Download list: %s" % downloads) - - # Package location within destdir, name subject to change/config - pkgdir = os.path.join(self.config.get('default', 'destdir'), self.config.get('default', 'version'), - self.config.get('default', 'flavor'), - self.config.get('default', 'arch'), - self.config.get('default', 'osdir'), - self.config.get('default', 'product_path')) + pkgdir = os.path.join(self.config.get('default', 'destdir'), + self.config.get('default', 'version'), + self.config.get('default', 'flavor'), + relpkgdir) if not os.path.exists(pkgdir): os.makedirs(pkgdir) - for pkg in self.polist: - repo = self.ayum.repos.getRepo(pkg.repoid) - remote = pkg.relativepath - local = os.path.basename(remote) - local = os.path.join(self.config.get('default', 'cachedir'), local) - if os.path.exists(local) and self.verifyCachePkg(pkg, local): + for po in polist: + repo = self.ayum.repos.getRepo(po.repoid) + + basename = os.path.basename(po.relativepath) + + local = os.path.join(self.config.get('default', 'cachedir'), basename) + target = os.path.join(pkgdir, basename) + + if os.path.exists(local) and self.verifyCachePkg(po, local): self.logger.debug("%s already exists and appears to be complete" % local) - target = os.path.join(pkgdir, os.path.basename(remote)) if os.path.exists(target): os.remove(target) # avoid traceback after interrupted download - try: - os.link(local, target) - except OSError, e: - if e.errno == 18: - # Can't hardlink cross file systems - shutil.copy2(local, target) - else: - self.logger.error('Got an error linking from cache: %s' % e) - raise OSError, e + self._link(local, target) continue # Disable cache otherwise things won't download repo.cache = 0 - self.logger.info('Downloading %s' % os.path.basename(remote)) - pkg.localpath = local # Hack: to set the localpath to what we want. + self.logger.info('Downloading %s' % basename) + po.localpath = local # Hack: to set the localpath to what we want. # do a little dance for file:// repos... - path = repo.getPackage(pkg) + path = repo.getPackage(po) if not os.path.exists(local) or not os.path.samefile(path, local): shutil.copy2(path, local) - try: - os.link(local, os.path.join(pkgdir, os.path.basename(remote))) - except OSError, e: - if e.errno == 18: - # Can't hardlink cross file systems - shutil.copy2(local, os.path.join(pkgdir, os.path.basename(remote))) - else: - self.logger.error('Got an error linking from cache: %s' % e) - raise OSError, e + self._link(local, target) + def downloadPackages(self): + """Download the package objects obtained in getPackageObjects().""" + + downloads = [] + for pkg in self.polist: + downloads.append('%s.%s' % (pkg.name, pkg.arch)) + downloads.sort() + self.logger.info("Download list: %s" % downloads) + + self._downloadPackageList(self.polist, + os.path.join(self.config.get('default', 'arch'), + self.config.get('default', 'osdir'), + self.config.get('default', 'product_path'))) self.logger.info('Finished downloading packages.') @@ -425,51 +425,4 @@ class Gather(pypungi.PungiBase): sys.exit(1) # do the downloads - pkgdir = os.path.join(self.config.get('default', 'destdir'), self.config.get('default', 'version'), - self.config.get('default', 'flavor'), 'source', 'SRPMS') - - if not os.path.exists(pkgdir): - os.makedirs(pkgdir) - - for pkg in srpmpolist: - repo = self.ayum.repos.getRepo(pkg.repoid) - remote = pkg.relativepath - local = os.path.basename(remote) - local = os.path.join(self.config.get('default', 'cachedir'), local) - if os.path.exists(local) and self.verifyCachePkg(pkg, local): - self.logger.debug("%s already exists and appears to be complete" % local) - if os.path.exists(os.path.join(pkgdir, os.path.basename(remote))) and self.verifyCachePkg(pkg, os.path.join(pkgdir, os.path.basename(remote))): - self.logger.debug("%s already exists in tree and appears to be complete" % local) - else: - try: - os.link(local, os.path.join(pkgdir, os.path.basename(remote))) - except OSError, e: - if e.errno == 18: - # Can't hardlink cross file systems - shutil.copy2(local, os.path.join(pkgdir, os.path.basename(remote))) - else: - self.logger.error('Got an error linking from cache: %s' % e) - raise OSError, e - - continue - - # Disable cache otherwise things won't download - repo.cache = 0 - self.logger.info('Downloading %s' % os.path.basename(remote)) - pkg.localpath = local # Hack: to set the localpath to what we want. - - # do a little dance for file:// repos... - path = repo.getPackage(pkg) - if not os.path.exists(local) or not os.path.samefile(path, local): - shutil.copy2(path, local) - - try: - os.link(local, os.path.join(pkgdir, os.path.basename(remote))) - except OSError, e: - if e.errno == 18: - # Can't hardlink cross file systems - shutil.copy2(local, target) - else: - self.logger.error('Got an error linking from cache: %s' % e) - raise OSError, e - + self._downloadPackageList(srpmpolist, os.path.join('source', 'SRPMS')) --

1 0

[patch 2/4] Fix local repository handling
by Mark McLoughlin 31 Aug '07

31 Aug '07

Fix a clear typo in the way local repositories are handled. The path to the package returned by getPackage() is "repo" and it is this that should be copied to /var/cache/pungi Signed-off-by: Mark McLoughlin <markmc(a)redhat.com> Index: pungi/pypungi/gather.py =================================================================== --- pungi.orig/pypungi/gather.py +++ pungi/pypungi/gather.py @@ -343,7 +343,7 @@ class Gather(pypungi.PungiBase): # do a little dance for file:// repos... path = repo.getPackage(pkg) if not os.path.exists(local) or not os.path.samefile(path, local): - shutil.copy2(local, path) + shutil.copy2(path, local) try: os.link(local, os.path.join(pkgdir, os.path.basename(remote))) @@ -461,7 +461,7 @@ class Gather(pypungi.PungiBase): # do a little dance for file:// repos... path = repo.getPackage(pkg) if not os.path.exists(local) or not os.path.samefile(path, local): - shutil.copy2(local, path) + shutil.copy2(path, local) try: os.link(local, os.path.join(pkgdir, os.path.basename(remote))) --

1 0

[patch 1/4] Remove Python 2.5 usage
by Mark McLoughlin 31 Aug '07

31 Aug '07

An empty list of base classes wasn't allowed before Python 2.5 so remove it since it seems to be the only Python 2.5 feature being used. Signed-off-by: Mark McLoughlin <markmc(a)redhat.com> Index: pungi/pypungi/__init__.py =================================================================== --- pungi.orig/pypungi/__init__.py +++ pungi/pypungi/__init__.py @@ -16,7 +16,7 @@ import logging import os import subprocess -class PungiBase(): +class PungiBase: """The base Pungi class. Set up config items and logging here""" def __init__(self, config): --

1 0

[patch 0/4] Some miscellanous pungi patches
by Mark McLoughlin 31 Aug '07

31 Aug '07

Hi, What follows are a few miscellanous patches for pungi. There's nothing major here. The one I'm most interested in is the third patch which adds new API to make it easier for other programs to use pypungi.gather. Cheers, Mark. --

1 0

remaking an initrd.img for diskboot.img and pxeboot
by Phil Meyer 17 Aug '07

17 Aug '07

For odd reasons, I am attempting to add a driver to an initrd.img. Using the exact same kernel as on the original media, I have built a driver module. I have extracted pxeboot/initrd.img into: /tmp/img/original I have extracted the /tmp/img/original/modules/modules.cgz file into: /tmp/img/moddir I have added the properly built modules into: /tmp/img/moddir/2.6.18-8.el5/x86_64 I have manually edited: /tmp/img/original/modules/modules.dep I rebuilt the initrd.img using these steps: # cd /tmp/img/moddir # find 2.6.18-8.el5 -depth -print | cpio --quiet -H crc -o | gzip -9 > /tmp/modules.cgz # cd /tmp/img/original # cp /tmp/modules.cgz modules/ # find . | cpio --quiet -c -o | gzip -9 > ../initrd.img I then copied the initrd.img to the proper PXE/TFTP directory, and tested a boot to it. It starts an install with no obvious errors, but the modules won't load. What did I miss? I am doing it remote with no access to <ALT>F3 or <ALT>F4 I will attempt to get over to the system and try from there to provide additional info, but I was hoping someone could spot an idiot mistake in my steps. Thanks for any help.

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys August 2007