buildsys June 2007

buildsys@lists.fedoraproject.org

30 participants
33 discussions

Known mock/yum problem?
by Paul Howarth 06 Jul '07

06 Jul '07

Came across an oddity today and I'm not sure if it's mock or yum that's the problem. I'm running with these updates from updates-testing (on an FC7 host): mock-0.7.2-1.fc7 yum-3.2.1-1.fc7 My rawhide mock config points to a single baseurl for the fedora repo, and goes through a local squid proxy. I'm also using autocache. From time to time packages cannot be retrieved over the network, perhaps due to the mirror being in mid-sync, or maybe transient network issues. Sometimes this results in one or more (but not all) packages not being available to populate a buildroot when running mock. When this happens, mock is not terminating the build during the setup phase, and allows it to continue to the build phase. I noticed this problem because it often results in a failed build due to to some important missing buildreq. More worrying though is the possibility that a missing buildreq package may be needed only to enhance the functionality of the package being built. In such cases the overall build may succeed but produce a package with reduced functionality, and be different from the same package rebuilt by someone else on their own system. So is this yum not returning a proper exit code, or mock ignoring it? I'm not sure. Is this a known issue? I couldn't see anything that looked like this in bugzilla. Paul.

2 3

Koji and SVN
by Oscar Bacho 02 Jul '07

02 Jul '07

Hi List I have two questions Koji can support SVN at the moment? why not? Oscar Palma Linux Mexico

4 7

mock 0.7.2 push to F7 updates
by Michael E Brown 26 Jun '07

26 Jun '07

After a week in updates-testing with no comments (for 0.7.2, add another week for 0.7.1), I have pushed 0.7.2 into updates/ for F7. This completes this round of upgrades, as 0.7.2 is now pushed into EPEL-4, EPEL-5, FC6, F7, and Fedora-Devel. -- Michael

2 1

Adding file / directory to pungi scripts.
by Joel Andres Granados 26 Jun '07

26 Jun '07

Hi list: Been working on a script that adds files and directories to the pungi tree. It does this by using an rpm package (the current way of doing it in pungi). I just think this might make somebodies life much easier :) Source is attached This is how it is used: Options: -h, --help show this help message and exit -s SOURCES, --source=SOURCES This script takes local files and directories. --source=/path/to/dir. --pungi-conf=PUNGICONF Pungi configuration file that will be modified. --yum-conf=YUMCONF Yum configuration file that will be modified. --name=PKGNAME It specifies the name of the package. --version=PKGVER Package Version. --release=PKGREL Package Release. -a ARCH, --arch=ARCH Package Architecture. noarch is recomended. -r REPONAME, --repo=REPONAME The repository name. It will be located in the pungi workdir. --revert Is used to undo the changes that this scripts does to the yum config file and the pungi config file. Any comment is greatly appreciated. Regards -- Joel Andres Granados #!/usr/bin/python import sys import optparse import os import os.path import shutil import subprocess from ConfigParser import SafeConfigParser ######## #Stages# ######## def recollectData(): """ Copy all files and directories to temp file """ #os.removedirs(options.workPath) os.makedirs(options.workPath) for nmS,src in options.bareSources.iteritems(): dst = os.path.join(options.workPath,nmS)#complete destination path if os.path.isdir(src): shutil.copytree(src, dst) continue elif os.path.isfile(src): shutil.copy(src, dst) continue else: #If not file and not dir ERROR print >> sys.stderr , "Error, Cannot find %s in system"% source sys.exit(1) def createRPM(): """ Create an rpm with the rpm -ba specfile.spec command """ #Create and put file in the working directory. specFile = """ Name: %s Version: %s Release: %s Summary: Sumary Group: System Environment/Base License: GPL Source: %%{name}-%%{version}.tar.gz Buildroot: %%{_tmppath}/%%{name}-%%{version}-%%{release}-root-%%(%%{__id_u} -n) BuildArch: %s provides: %%{name} %%description Description %%prep %%setup -q %%build # Nothing to see here. %%install # Nothing to see here. """ % (options.pkgName, options.pkgVer, options.pkgRel, options.arch) #Create the files section fileSec = "%%files\n %%doc %s.spec " % options.pkgName for name in options.bareSources.iterkeys(): fileSec = fileSec + "%s " % name # Add the files section the the spec file. specFile = specFile + fileSec + "\n" # Put the spec file in the directory. specfd = open( os.path.join(options.workPath,options.pkgName+".spec"), 'w' ) specfd.write(specFile) specfd.close() #make a tar file and put it in SOURCES options.sourceDir = "/usr/src/redhat/SOURCES" tarCommand = [ "tar", "zcvvf", "%s/%s-%s.tar.gz"%(options.sourceDir, options.pkgName, options.pkgVer), "%s"%options.workDir] p = subprocess.Popen(tarCommand, stdout=subprocess.PIPE, stdin=subprocess.PIPE, cwd=options.tempDir) p.wait() #execute rpmbuild rpmCommand = [ "rpmbuild", "-ba", "%s/%s.spec"%(options.workPath, options.pkgName)] p = subprocess.Popen(rpmCommand, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE) p.wait() def createRepo(): """ Create repo and put in destdir/repodir """ destdir = options.pconf.get('default', 'destdir') options.repodir = os.path.join(destdir, "repository") if not os.path.exists(options.repodir):os.makedirs(options.repodir) #retrieve the rpm from the /usr/src/redhat/RPMS shutil.copy("/usr/src/redhat/RPMS/%s/%s-%s-%s.%s.rpm"%(options.arch, options.pkgName, options.pkgVer, options.pkgRel, options.arch), options.repodir) #run createrepo createrepoCommand = ["createrepo", "-q", "%s"%options.repodir] p = subprocess.Popen(createrepoCommand, stdin=subprocess.PIPE, stdout=subprocess.PIPE, cwd=options.repodir) p.wait() def addRepo(): """ Add created repo to the yum conf file specified in the pungi conf file. """ #lets handle this with a config parser. options.yconf.add_section(options.repoName) options.yconf.set(options.repoName, 'name', options.repoName) options.yconf.set(options.repoName, 'baseurl', 'file://%s'% options.repodir) options.yconf.set(options.repoName, 'enabled', '1') options.yconf.set(options.repoName, 'gpgcheck', "0") options.yconf.write(open(options.yumConf, 'w')) def pungiConfFiles(): """ Modify the relnotepkgs, relnotefilere, relnotedirre accordingly """ #create the re variables. options.pconf.set('default','relnotepkgs', options.pkgName) relnotedirre = "" relnotefilere = "" for nmS, src in options.bareSources.iteritems(): if os.path.isdir(src): relnotedirre = "%s %s"%(relnotedirre, nmS) if os.path.isfile(src): relnotefilere = "%s %s"%(relnotefilere, nmS) #write to file if not relnotedirre == "": options.pconf.set('default','relnotedirre', relnotedirre) if not relnotefilere == "": options.pconf.set('default','relnotefilere', relnotefilere) options.pconf.write(open(options.pungiConf, 'w')) def manifestPkgs(): """ Add the created package to the manifest. """ manifest = options.pconf.get('default', 'manifest') fd = open(manifest, 'r+') lines = fd.readlines() try: lines.index(options.pkgName+'\n') except: lines.append(options.pkgName+'\n') fd.writelines(lines) fd.close() def clean(): """ Clean temp files. """ #Delete temporary file shutil.rmtree(options.tempDir, True) def revert(): """ Revert the changes from the pungi config files """ # Take out the relnotedirre and relnotefilere stuff relnotedirre = options.pconf.get('default', 'relnotedirre').split() relnotefilere = options.pconf.get('default', 'relnotefilere').split() for key in options.bareSources.iterkeys(): try: relnotedirre.remove(key) except: try: relnotefilere.remove(key) except: pass options.pconf.remove_option('default', 'relnotedirre') if len(relnotedirre) > 0: options.pconf.set('default', 'relnotedirre', relnotedirre) options.pconf.remove_option('default', 'relnotedirre') if len(relnotefilere) > 0: options.pconf.set('default', 'relnotefilere', relnotefilere) # Take out the relnotepkgs stuff relnotepkgs = options.pconf.get('default', 'relnotepkgs').split() for elem in relnotepkgs: try: relnotepkgs.remove(options.pkgName) except: pass options.pconf.remove_option('default', 'relnotepkgs') if len(relnotepkgs) > 0: options.pconf.set('default', 'relnotepkgs', relnotepkgs) # Take out the yum config stuff options.yconf.remove_section(options.repoName) # Write the configurations options.pconf.write(open(options.pungiConf, 'w')) options.yconf.write(open(options.yumConf, 'w')) def main(): #If revert just do it and exit. if options.revert: revert() sys.exit(0) #Recollect all the files recollectData() #Make the rpm package createRPM() #Make the very small repo createRepo() #Add the repo to the yum config addRepo() #Add the files and directories to the pungi config. pungiConfFiles() #Add packages to manifes manifestPkgs() #remember to clean up clean() if __name__ == "__main__": # Get user options. parser = optparse.OptionParser() parser.add_option( "-s","--source", action="append", dest="sources", type="string", help="This script takes local files and directories. \ --source=/path/to/dir. ") parser.add_option( "--pungi-conf", dest="pungiConf", type="string", default="/etc/pungi/pungi.conf", help="Pungi configuration file that will be modified.") parser.add_option( "--yum-conf", dest="yumConf", type="string", default="/etc/pungi/yum.conf.f7.x86_64", help="Yum configuration file that will be modified.") parser.add_option( "--name", dest="pkgName", type="string", default="name", help="It specifies the name of the package.") parser.add_option( "--version", dest="pkgVer", type="string", default="0.0.0", help="Package Version.") parser.add_option( "--release", dest="pkgRel", type="string", default="01", help="Package Release.") parser.add_option( "--arch", "-a", dest="arch", type="string", default="noarch", help="Package Architecture. noarch is recomended.") parser.add_option( "--repo", "-r", dest="repoName", type="string", default="additionals", help="The repository name. It will be located in the pungi workdir.") parser.add_option( "--revert", action="store_true", dest="revert", default=False, help="Is used to undo the changes that this scripts does to\ the yum config file and the pungi config file.") (options, args) = parser.parse_args() if options.sources == None: parser.print_help() sys.exit(1) # Create some needed variables. options.workDir = "%s-%s"%(options.pkgName,options.pkgVer) options.tempDir = os.path.join("/tmp", "temp-rpm%d"%os.getpid()) options.workPath = os.path.join(options.tempDir, options.workDir) # Set the handlers for the configuration files options.pconf = SafeConfigParser() options.pconf.read(options.pungiConf) options.yconf = SafeConfigParser() options.yconf.read(options.yumConf) # We are eventually going to need the bare sources. options.bareSources = {} for source in options.sources: src = os.path.normpath(source)# without leading '/' nmS = os.path.basename(src)# Name of file or dir without path dst = os.path.join(options.workPath,nmS)#complete destination path options.bareSources[nmS] = src main()

1 0

Integrating deltarpm creation
by Jeremy Katz 22 Jun '07

22 Jun '07

One of the things that we're looking at for Fedora 8 is how to integrate deltarpms and the presto yum plugin so that we can reduce the amount that our users have to download. To make this happen, though, is going to require some work on the side of the build system and compose tools so that we can generate these without any real problems. Based on the discussion Jesse and I were having driving home yesterday, we came to the following as a proposal. (Note: any errors here are probably mine since I transcribed it a while after the conversation). The first thing is that it makes the most sense for the deltas to be created and stored by koji rather than as a secondary process. This adds the advantage that they're stored consistently with the packages and also can be cached rather than recreated every time. It feels somewhat analagous to me to the current situation with signing. To handle the case of deltas for updates, we can have a step either happen pre-mash or early in mash to call out to koji to create deltas. We'll want to be creating the delta for dist-fX-GA -> the update as well as the latest package in dist-FX-updates and including them. We then just need to generate the delta metadata and do modifyrepo much like we'll be doing for the updateinfo. Updates-testing can basically work identically to updates. Generating deltas for rawhide is a little trickier. The idea we got to was that we'd change dist-rawhide to not inherit and instead be tagging packages with it. Then, for the packages we tag, we can also generate a delta from the last dist-rawhide version to the new version. Then mash can pull off of the dist-rawhide tag (like now) and use deltas much like the update case What do people think? Entirely crazy? Just a little bit crazy? Has holes big enough to drive a truck through? Jeremy

6 6

bodhi and security updates/updates-testing
by Axel Thimm 22 Jun '07

22 Jun '07

Hi, I created a new update which was a security relevant update and was marked as such. bodhi offered me the possibility to push into testing or stable. I think the push into stable was offered only because it was maerked a security sensitive, so updates-testing could be shortcutted. I also know that the default policy was for security updates to bypass updates-testing entirely. Still, the system offered my to go testing or stable, and I chose testing. The update later was simply saying that a request was made to move it, but not whereto. A while later the package made it into stable. I'm not saying to change the policy, just noting a discrepancy in the application's workflow. Thanks! -- Axel.Thimm at ATrpms.net

3 10

mock update in testing doesn't fix glibc-devel.i386 problem
by Jesse Keating 21 Jun '07

21 Jun '07

Seems the fix I made to config files either never got committed to my branch or got lost in the shuffle, but we wound up not fixing the bug and I didn't notice until now (since I was using locally modified config files). I've fixed it again in HEAD, so we should probably respin the update or just leave it in there to be picked up in the next update. Not sure, we're already messing with config files a lot, may be worth doing it now when everybody will have to re-review the config files and re-do any local modification. -- Jesse Keating Release Engineer: Fedora

3 3

Re: pungi used to create CD that includes a kickstart file
by Steve Magoun 21 Jun '07

21 Jun '07

Hi, Given the discussion about adding kickstart files to a DVD, I thought I'd add my experience with Fedora 7. We're using pungi 0.3.5 to create a custom F7 install DVD that includes multiple kickstarts on the DVD. I've hacked pungi to add our kickstart files to the installer's boot menu. This way non-technical users can select a kickstart at install time by choosing a menu item rather than typing. We've found it quite useful, perhaps others will too. Here's what we do: 1) Package all kickstart files in a kickstart-config RPM and deploy the RPM to our repository. The RPM is setup so that kickstart files go into /kickstart: %install rm -rf $RPM_BUILD_ROOT install -d -m 0755 $RPM_BUILD_ROOT/kickstart for file in kickstart/* ; do install -p -m 0644 $file $RPM_BUILD_ROOT/kickstart done [...] %files %defattr(-,root,root,-) %dir /kickstart /kickstart/* 2) Add the kickstart-config RPM to the pungi manifest 3) Add our kickstart RPM + files to the relnote entries of pungi's config file: relnotepkgs = fedora-release fedora-release-notes kickstart-config relnotedirre = images stylesheet-images kickstart 4) Hack pungi to add our kickstart files to isolinux.cfg. Not pretty, but it gets the job done for us: 5) Run pungi Thanks to Jesse + others for an excellent set of tools! Steve

1 0

RE: Fedora 7 Test Update: pungi-0.3.7-2.fc7
by ぽちけん 21 Jun '07

21 Jun '07

> From: jkeating(a)redhat.com> To: fedora-buildsys-list(a)redhat.com> Date: Wed, 20 Jun 2007 17:05:46 -0400> Subject: Fwd: Fedora 7 Test Update: pungi-0.3.7-2.fc7> > Per earlier thread, this should fix the duplicate packages issues. If you > guys could test this and let me know I'd appreciate it.> > ---------- Forwarded Message ----------> > Subject: Fedora 7 Test Update: pungi-0.3.7-2.fc7> Date: Wednesday 20 June 2007> From: updates(a)fedoraproject.org> To: fedora-test-list(a)redhat.com> > --------------------------------------------------------------------------------> Fedora Test Update Notification> FEDORA-2007-0579> 2007-06-20 13:00:30.949566> --------------------------------------------------------------------------------> > Name : pungi> Product : Fedora 7> Version : 0.3.7> Release : 2.fc7> Summary : Distribution compose tool> Description :> A tool to create anaconda based installation trees/isos of a set of rpms.> > -----------------------------------------------------------! ---------------------> Update Information:> > When using pungi with both Fedora 7 and Fedora 7 updates repos to respin > Fedora 7 with updates pungi would pull in both the original Fedora 7 package > and the Fedora 7 update package as well. This was due to how dep resolution > was being done. This update fixes this bug so that only the newest version > of a dep resolver will be pulled in.> --------------------------------------------------------------------------------> ChangeLog:> > * Wed Jun 20 2007 Jesse Keating <jkeating(a)redhat.com> - 0.3.7-2> - Fix a bug where new and old copies of a package were being included> * Wed May 30 2007 Jesse Keating <jkeating(a)redhat.com> - 0.3.7-1> - Handle the cdsize variable correctly> - More fixes for cached download stuff> - Fix default CD size storing> - Update comps file with what shipped for F7> * Fri May 25 2007 Jesse Keating <jkeating(a)redhat.coM> - 0.3.6-1> - Handle the cdsize variable correctly> ------------------------------------! --------------------------------------------> Updated packages! :> Tested it with a Everything manifest. I checked, ISO image included only the newest version. _________________________________________________________________ Windows Vista ついに発売！ http://search.msn.co.jp/results.aspx?q=windows+vista&FORM=MSNH&cp=65001

1 0

Fwd: Fedora 7 Test Update: pungi-0.3.7-2.fc7
by Jesse Keating 21 Jun '07

21 Jun '07

Per earlier thread, this should fix the duplicate packages issues. If you guys could test this and let me know I'd appreciate it. ---------- Forwarded Message ---------- Subject: Fedora 7 Test Update: pungi-0.3.7-2.fc7 Date: Wednesday 20 June 2007 From: updates(a)fedoraproject.org To: fedora-test-list(a)redhat.com -------------------------------------------------------------------------------- Fedora Test Update Notification FEDORA-2007-0579 2007-06-20 13:00:30.949566 -------------------------------------------------------------------------------- Name : pungi Product : Fedora 7 Version : 0.3.7 Release : 2.fc7 Summary : Distribution compose tool Description : A tool to create anaconda based installation trees/isos of a set of rpms. -------------------------------------------------------------------------------- Update Information: When using pungi with both Fedora 7 and Fedora 7 updates repos to respin Fedora 7 with updates pungi would pull in both the original Fedora 7 package and the Fedora 7 update package as well. This was due to how dep resolution was being done. This update fixes this bug so that only the newest version of a dep resolver will be pulled in. -------------------------------------------------------------------------------- ChangeLog: * Wed Jun 20 2007 Jesse Keating <jkeating(a)redhat.com> - 0.3.7-2 - Fix a bug where new and old copies of a package were being included * Wed May 30 2007 Jesse Keating <jkeating(a)redhat.com> - 0.3.7-1 - Handle the cdsize variable correctly - More fixes for cached download stuff - Fix default CD size storing - Update comps file with what shipped for F7 * Fri May 25 2007 Jesse Keating <jkeating(a)redhat.coM> - 0.3.6-1 - Handle the cdsize variable correctly -------------------------------------------------------------------------------- Updated packages: 1ffb4d5a9bd955fb91fe039b245a8f9818aff159 pungi-0.3.7-2.fc7.noarch.rpm 01542aff5238389e27b0775aa9e02ad0bf26fcf4 pungi-0.3.7-2.fc7.src.rpm This update can be installed with the 'yum' update program. Use 'yum update package-name' at the command line. For more information, refer to 'Managing Software with yum,' available at http://docs.fedoraproject.org/yum/. -------------------------------------------------------------------------------- -- fedora-test-list mailing list fedora-test-list(a)redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list ------------------------------------------------------- -- Jesse Keating Release Engineer: Fedora

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys June 2007