buildsys September 2008

buildsys@lists.fedoraproject.org

18 participants
20 discussions

Revisor / yum oddity with package conflicts...
by Martin Langhoff 11 Sep '08

11 Sep '08

With the exact same commandline, roughly 50% of the time I get a conflict betweem generic-logos and fedora-logos. This happens with stock F9 revisor from rpm and the tip of the F-9 branch in git. Nothing whatsoever requires fedora-logos. Sometimes it works, sometimes it doesn't. !? $ sudo revisor --cli --config revisor/revisor.conf \ --kickstart=revisor/anaconda-f9.ks \ --kickstart-include --kickstart-default \ --install-unified --model 'xs-f9-i386' --yes \ --rebrand-name=xs SELinux on this host is disabled. Composed media will not have SELinux, and as a result the system you install from the composed media will not have SELinux either. Loading Repositories: ######################################## 100.0% Select kickstart packages: ######################################## 100.0% Resolving Dependencies: ######################################## 100.0% generic-logos-9.0.0-1.fc9.noarch from fedora has depsolving problems --> generic-logos conflicts with fedora-logos Resolving Dependencies: ######################################## 100.0% Unable to resolve dependencies for some packages selected: generic-logos conflicts with fedora-logos *nothing* requires fedora-logos. Calling revisor with debug=5 shows... Adding required package kernel-0:2.6.25-14.fc9.i686 Adding required package kernel-xen-0:2.6.25-2.fc9.i686 Checking dependencies - not allowing any conflicts within the package set Resolving Dependencies: 0.0% Removing package fedora-release-0:9-2.noarch for rebranding Removing package fedora-release-notes-0:9.0.0-1.noarch for rebranding Removing package fedora-logos-0:9.0.0-2.fc9.noarch for rebranding Resolving Dependencies: ######################################## 100.0% generic-logos-9.0.0-1.fc9.noarch from fedora has depsolving problems --> generic-logos conflicts with fedora-logos Resolving Dependencies: ######################################## 100.0% Unable to resolve dependencies for some packages selected: cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

3 3

How stable is revisor 'master' as of now? I am keen on using --isolinux-cfg on F9
by Martin Langhoff 11 Sep '08

11 Sep '08

Hoping to provide a custom isolinux menu for the XS spin - and looking at the src in git, I see the --isolinux-cfg option. If it works, it'll be just the ticket. Is it reasonable to expect it to run with F-9 and with F-9 anaconda? cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

2 1

pungi my humble configurations
by Sergio Monteiro Basto 10 Sep '08

10 Sep '08

Hi , I put my humble configurations to build f8 and f9 updated repos with livna repos, on my homepage http://sergiomb.no-ip.org/pungi/confs/ feed back is welcome, hope that can help someone Thanks, -- Sérgio M. B.

1 0

Assembling packages for ISO's
by Bryce 10 Sep '08

10 Sep '08

Can anyone think of a more succinct means to drag out all the latest rpms based on a tag? I could only came up with this horror. mkdir pool; cd pool TAG=f9-bryced koji list-pkgs --tag=${TAG} --quiet | cut -f 1 -d " " | xargs koji latest-pkg ${TAG} --quiet | cut -f1 -d " " | xargs -n1 koji download-build --arch=i386 --arch=i686 --arch=noarch Phil =--=

2 2

Patch fixing a problem with --kickstart-include
by Martin Langhoff 09 Sep '08

09 Sep '08

By naming the kickstart file as ks.cfg, anaconda would _always_ take it, regardless of kernel boot options. This is not what was expected - it is safer to give it a different name, and then use the boot menu item to select it. The patch is on top if F-9 . cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

2 4

[PATCH] Perforce support for koji
by Paul B Schroeder 08 Sep '08

08 Sep '08

The powers that be require us to use Perforce. Thus, the patch. A few things to note about it: * Our P4 server is running on a non-standard port and thus needs to be specified. scm_tuple is now created via rsplit instead. Allows SCM port to be specified. i.e. This now works: allowed_scms=perforcetx.bluecoat.com:1999:/koji/sandbox/paul.schroeder/skeletor * P4 *requires* a stinkin' password. So I added password support to _parse_url. The "user" part of the SCM url can now be "user:password". Still works minus the ":password" too. Let me know if there's a better/preferred way to support passwords. * P4 can only checkout into the static client root specified in the P4 client spec (PITA). Sooo.. After the module_checkout_cmd is executed, if sourcedir does not exist, we do a 'os.rename' of the checked out code into the proper task sourcedir. * A P4 SCM URL looks like: p4://user:pass@p4port/p4client/p4depot?path/to/module#changelist i.e. p4port==host And p4client is your P4 client spec that you've created. Let me know if there are any Qs, comments, or any changes I should make.. Cheers...Paul... diff --git a/builder/kojid b/builder/kojid index 1a7afbf..92f8143 100755 --- a/builder/kojid +++ b/builder/kojid @@ -1830,7 +1830,8 @@ class BuildSRPMFromSCMTask(BaseTaskHandler): use_common = True for allowed_scm in options.allowed_scms.split(): - scm_tuple = allowed_scm.split(':') + # Use rsplit with 1 max split to allow for port numbers w/ scm host + scm_tuple = allowed_scm.rsplit(':', 1) if len(scm_tuple) in (2, 3): if fnmatch(scm.host, scm_tuple[0]) and fnmatch(scm.repository, scm_tuple[1]): # SCM host:repository is in the allowed list @@ -2325,7 +2326,8 @@ class SCM(object): 'GIT': ('git://', 'git+http://', 'git+https://', 'git +rsync://'), 'GIT+SSH': ('git+ssh://',), 'SVN': ('svn://', 'svn+http://', 'svn+https://'), - 'SVN+SSH': ('svn+ssh://',) } + 'SVN+SSH': ('svn+ssh://',), + 'P4': ('p4://',) } def is_scm_url(url): """ @@ -2362,10 +2364,11 @@ class SCM(object): raise koji.GenericError, 'Invalid SCM URL: %s' % url self.url = url - scheme, user, host, path, query, fragment = self._parse_url() + scheme, user, password, host, path, query, fragment = self._parse_url() self.scheme = scheme self.user = user + self.password = password self.host = host self.repository = path self.module = query @@ -2384,9 +2387,9 @@ class SCM(object): Parse the SCM url into usable components. Return the following tuple: - (scheme, user, host, path, query, fragment) + (scheme, user, password, host, path, query, fragment) - user may be None, everything else will have a value + user and password may be None, everything else will have a value """ # get the url's scheme scheme = self.url.split('://')[0] + '://' @@ -2396,14 +2399,21 @@ class SCM(object): dummyscheme, netloc, path, params, query, fragment = urlparse.urlparse(dummyurl) user = None + password = None userhost = netloc.split('@') if len(userhost) == 2: - user = userhost[0] + userpass = userhost[0].split(':') + user = userpass[0] + if len(userpass) == 2: + password = userpass[1] + elif len(userpass) > 2: + raise koji.GenericError, 'Invalid username:password specified: %s' % netloc if not user: # Don't return an empty string user = None - elif ':' in user: - raise koji.GenericError, 'username:password format not supported: %s' % user + if not password: + # Don't return an empty string + password = None netloc = userhost[1] elif len(userhost) > 2: raise koji.GenericError, 'Invalid username@hostname specified: %s' % netloc @@ -2419,7 +2429,7 @@ class SCM(object): raise koji.GenericError, 'Unable to parse SCM URL: %s' % self.url # return parsed values - return (scheme, user, netloc, path, query, fragment) + return (scheme, user, password, netloc, path, query, fragment) def checkout(self, scmdir, uploadpath, logfile, use_common=False): """ @@ -2514,6 +2524,39 @@ class SCM(object): module_checkout_cmd = ['svn', 'checkout', '-r', self.revision, '%s/%s' % (svnserver, self.module), self.module] common_checkout_cmd = ['svn', 'checkout', '%s/common' % svnserver] + elif self.scmtype == 'P4': + if not self.user: + raise koji.BuildError, 'No user specified for repository access scheme: %s' % self.scheme + # P4 URL: + # p4://user:pass@p4port/p4client/p4depot?path/to/module#changelist + + # Perforce uses a "client workspace" + # Before first slash is the p4 client, after is the p4 depot + (p4client, p4depot) = self.repository[1:].split('/', 1) + + p4client_root = None + p4info = os.popen('p4 -p ' + self.host + ' -c ' + p4client + ' info') + for line in p4info: + if line.startswith('Client root:'): + p4client_root = line.split()[-1] + break + p4info.close() + if not p4client_root: + raise koji.BuildError, 'Could not find perforce client root' + + # Perforce cannot checkout to a random dir (in this case scmdir). + # It will only checkout to the p4client_root, so we override scmdir + # with the p4client_root + scmdir = '%s/%s' % (p4client_root, p4depot) + + # When HEAD is given as the revision, we assume we want the latest + p4revision = '' + if self.revision.upper() != 'HEAD': + p4revision = '@%s' % self.revision + + module_checkout_cmd = ['p4', '-p', self.host, '-u', self.user, '-P', self.password, '-c', p4client, 'sync', '-f', '//%s/% s/...%s' % (p4depot, self.module, p4revision)] + common_checkout_cmd = ['p4', '-p', self.host, '-u', self.user, '-P', self.password, '-c', p4client, 'sync', '-f', '//% s/common/...' % p4depot] + else: raise koji.BuildError, 'Unknown SCM type: %s' % self.scmtype @@ -2521,6 +2564,10 @@ class SCM(object): if log_output(module_checkout_cmd[0], module_checkout_cmd, logfile, uploadpath, cwd=scmdir, logerror=1, env=env): raise koji.BuildError, 'Error running %s checkout command "%s", see %s for details' % \ (self.scmtype, ' '.join(module_checkout_cmd), os.path.basename(logfile)) + # Perforce checks out to the client root listed for the perforce client. + # Need to move checked out source into sourcedir. + if not os.path.exists(sourcedir): + os.rename('%s/%s' % (scmdir, self.module), sourcedir) if update_checkout_cmd: # Currently only required for GIT checkouts

2 3

Revisor - package not found and kickstart options change dep error tolerance?
by Martin Langhoff 05 Sep '08

05 Sep '08

Hi Jeroen, I am trying to build a revisor-based School Server installer, and I am finding some oddities - As I mentioned before, it complains about a missing dependency for a package that is right there, in the same repo as other packages it is picking up. - If I pass --kickstart-include --kickstart-default the missing dependency is fatal. If I don't, it shows the warning and builds the image... both quite strange problems... I'm jumping into the src right now to see if I can find out more, thought I'd mention it... btw, http://lists.fedoraunity.org/mailman/listinfo/revisor-devel is still unreachable... cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

3 4

Pungi kickstart packages - stage2 deps vs instaled pkgs
by Martin Langhoff 04 Sep '08

04 Sep '08

Perhaps this is by design: the "build tools" dependencies (anaconda-runtime an friends) which end up in stage2 AFAICS need to be listed (and their dependencies met) in the main packages listing in the kickstart file. Is there a way to control those package listings separately? As a (trivial) example, we end up with an anaconda-runtime rpm in the iso, which we are not interested in... cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

3 4

Pungi / anaconda: seeding the pkg selection
by Martin Langhoff 03 Sep '08

03 Sep '08

The installer cd I'm trying to build (for the School Server) is driven by 2 custom packages - a metapackage that pulls in all the dependenciess, and a configuration package that whacks /etc . Feeding those to Pungi via a kickstart file gets them on the installer CD, but does not get them installed by default. My overall goals (and stumbling blocks) are: 1 - Support interactive installs (that defaults to installing the XS packages!) for developers & pilot sites. This should be the default on the grub menu... - How do get anaconda to pick those 2 custom packages by default? Using a kickstart file is not an option - it switches off the interactive install mode. Is anaconda auto-installing the 'base' group and should I add them there? Should I create a new group in comps.xml and get anaconda to have it selected by default? - How can I feed Pungi a custom comps.xml? 1.1 - The interactive install should have a custom 'default' layout for the autopartitioning. Can I seed that somehow? 2 - Support a fully automated kickstart install (non-default option in grub menu). This is also an example for local teams to edit to their needs... - Does Pungi support a %post that can "copy in" a new grub menu to the isolinux/syslinux directory? - ... and add a ks.cfg file to the initrd? - Will anaconda probe for ks.cfg in any directories that are easier to edit than the initrd? lots of questions :-/ cheers, m -- martin.langhoff(a)gmail.com martin(a)laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff

3 4

koji-1.2.6
by Mike McLean 03 Sep '08

03 Sep '08

It's been a long time since we posted a fresh tarball, so I just tagged and posted 1.2.6. Following is a summary of changes since 1.2.5. A couple of them /may/ cause some slight upgrade issues. - kojid no longer sleeps after taking a task. This shouldn't be too much of a problem, but it is a behaviour change and may affect your load distribution. - pathinfo.taskrelpath. The upload path for task workfiles has changed to avoid creating over-large directories. This may cause older clients to report broken urls. Updated clients, of course, will report the correct ones. - unified boolean option parsing in kojihub. This is a bugfix, but may expose configuration errors. - ServerOffline fault. This is a new fault code, and older clients will not be able to interpret it. Howevever, in the case where the exception is intended to be used this should be no worse than previous behavior. (mostly) full change summary: • don't sleep after taking a task • pathinfo.taskrelpath • unified waitrepo logic • scm user_common flag • shorter waitrepo timeout • support building from git subdirs • robust sessions for kojid ∘ new config options: retry_interval, max_retries, offline_retry, offline_retry_interval • authtype option for koji cli • top level quiet option for koji cli • select config file section based on invocation name • larger blocksize for uploads • prune-signed-copies improvements • saner duplicate filtering in list-pkgs • expand group deps in generated comps • several new options for download-build • wait-repo cli command • advertise help --admin • support db password auth in kojihub • listTagged/listTaggedRPMS by owner • handle importing an srpm for an existing build • unified boolean option parsing in kojihub • alias system.multiCall as system.multicall • ServerOffline exception ∘ raised when db connect fails ∘ also ServerOffline hub config option • fix Source: line in spec • package koji-gc (in -utils) • changed default config location for koji-gc • gc: fix age calculation for untagged builds • option to provide a ts for get_rpm_header • set rpmbuild_timeout of 24hours in mock configs • more robust ClientSession connections ∘ better retries ∘ retries for anonymous sessions ∘ special treatement for ServerOffline case ∘ don't mask real exceptions • use _callMethod in ClientSession.multiCall • larger default blocksize in uploadWrapper • automatically refresh stale db connections • kojira: don't throw an exception if a tag is renamed or deleted • add new session options for kojira: ∘ retry_interval, max_retries, offline_retry, offline_retry_interval • kojiweb: ∘ handle exceptions gracefully ∘ allow rss feed to filter by package name (specified with package=) ∘ allow listTagged* and readTagged* to filter by package owner name (specified with owner=) ∘ allow the builds page in the web UI to filter by package ID (specified with packageID=) ∘ only send the session cookie over a secure connection (patch from Toshio Kuratomi, with modifications) ∘ remove the KojiWebURL config option (no longer necessary) ∘ sort rpm listing on the buildinfo page by arch and -debuginfo ∘ add links to the headers on the info pages ∘ put identification information into the title on the *info pages ∘ change the title format to put the important bits first (format suggested by Will Woods) ∘ put the task label into the taskinfo header ∘ move the task ID into the main table

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys September 2008