koji-devel June 2015

koji-devel@lists.stg.fedorahosted.org

10 participants
16 discussions

Koji client tools python3 compatible
by Adam Miller 18 Jun '15

18 Jun '15

It was recently brought up to me that with the impending "Python3 as Default"[0] change coming to Fedora it would be nice if we had Python3 client-side compatibility. The main motivation is from a developer (Fedora Workstation User) perspective, it would be nice if the didn't need to pull in all of the python2 stack just to be a Fedora Packager. I don't know how big of an overhaul this would be or what it would potentially break on the back end for the ability to still run server side bits on EL5 machines, but I wanted to at least initiate the discussion. Thank you, -AdamM [0] - https://fedoraproject.org/wiki/Changes/Python_3_as_Default

3 2

Extra options for tags and dnf support
by Mike McLean 18 Jun '15

18 Jun '15

With mock switching to dnf, we needed to conditionally adjust mock's package_manager config on a per-tag basis. I didn't want to add yet another specialized field in the tag_config table, so I extended the schema to support more flexible extra options for tags. This set of patches adds the extra options functionality and uses it to control set mock's package_manager option. Each tag can now be assigned arbitrary extra options. These are stored as json in the database, so they can be of any type that json supports. The getBuildConfig call follows inheritance when determining these. To tell mock to use dnf in a tag, you would run a command like: # koji edit-tag TAG -x mock.package_manager=dnf This setting is inherited.

3 9

[PATCH] download-logs: print a message when there are no logs to download
by Mike McLean 17 Jun '15

17 Jun '15

see: https://bugzilla.redhat.com/show_bug.cgi?id=1232916 --- cli/koji | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/cli/koji b/cli/koji index 908d829..9ec098d 100755 --- a/cli/koji +++ b/cli/koji @@ -6229,15 +6229,21 @@ def anon_handle_download_logs(options, session, args): task_log_dir = os.path.join(parent_dir, "%s-%s" % (task_info["arch"], task_id)) + count = 0 state = koji.TASK_STATES[task_info['state']] if state == 'FAILED': if not match or koji.util.multi_fnmatch(FAIL_LOG, match): write_fail_log(task_log_dir, task_id) + count += 1 elif state not in ['CLOSED', 'CANCELED']: sys.stderr.write(_("Warning: task %s is %s\n") % (task_id, state)) for log_filename in logs: download_log(task_log_dir, task_id, log_filename) + count += 1 + + if count == 0 and not recurse: + sys.stderr.write(_("No logs found for task %i. Perhaps try --recurse?\n") % task_id) if recurse: child_tasks = session.getTaskChildren(task_id) -- 1.9.3

1 0

[PATCH] include lib64 dir in default pluginpath
by Mike McLean 17 Jun '15

17 Jun '15

--- builder/kojid | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builder/kojid b/builder/kojid index b6b79c1..8243434 100755 --- a/builder/kojid +++ b/builder/kojid @@ -4460,7 +4460,7 @@ def get_options(): 'topdir': '/mnt/koji', 'topurl': None, 'workdir': '/var/tmp/koji', - 'pluginpath': '/usr/lib/koji-builder-plugins', + 'pluginpath': '/usr/lib/koji-builder-plugins:/usr/lib64/koji-builder-plugins', 'mockdir': '/var/lib/mock', 'mockuser': 'kojibuilder', 'packager': 'Koji', -- 1.9.3

2 2

[PATCH] capture mock output even if it exits immediately
by Mike McLean 17 Jun '15

17 Jun '15

--- builder/kojid | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builder/kojid b/builder/kojid index 8243434..2e1add3 100755 --- a/builder/kojid +++ b/builder/kojid @@ -374,7 +374,7 @@ class BuildRoot(object): results = os.listdir(resultdir) except OSError: # will happen when mock hasn't created the resultdir yet - continue + results = [] for fname in results: if fname.endswith('.log') and not logs.has_key(fname): -- 1.9.3

1 0

[PATCH] inject the url of the install tree into the kickstart used for generating images
by Brian Stinson 06 Jun '15

06 Jun '15

From: Brian Stinson <bstinson(a)redhat.com> Date: Sat, 6 Jun 2015 16:43:40 -0500 --- builder/kojid | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/builder/kojid b/builder/kojid index ad2cf1f..b6b79c1 100755 --- a/builder/kojid +++ b/builder/kojid @@ -2841,7 +2841,7 @@ class OzImageTask(BaseTaskHandler): "'%s' : %s" % (kspath, e)) return ks - def prepareKickstart(self, kspath): + def prepareKickstart(self, kspath, install_tree): """ Process the ks file to be used for controlled image generation. This method also uploads the modified kickstart file to the task output @@ -2878,6 +2878,8 @@ class OzImageTask(BaseTaskHandler): self.logger.debug('BASEURL: %s' % baseurl) ks.handler.repo.repoList.append(repo_class( baseurl=baseurl, name='koji-override-0')) + # inject the URL of the install tree into the kickstart + ks.handler.url.url = install_tree return ks def writeKickstart(self, ksobj, ksname): @@ -3327,7 +3329,7 @@ class BaseImageTask(OzImageTask): # First, prepare the kickstart to use the repos we tell it kspath = self.fetchKickstart() - ks = self.prepareKickstart(kspath) + ks = self.prepareKickstart(kspath, inst_tree) kskoji = self.writeKickstart(ks, os.path.join(self.workdir, 'koji-%s-%i-base.ks' % (self.target_info['build_tag_name'], self.id))) -- 1.8.3.1

2 1

[PATCH] only use PlgHTTPSConnection for ssl_login
by Mike McLean 06 Jun '15

06 Jun '15

--- koji/__init__.py | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/koji/__init__.py b/koji/__init__.py index 58971da..36772bb 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -1599,12 +1599,7 @@ class ClientSession(object): self.explicit_port = bool(_port) self._path = uri[2] default_port = 80 - if self.opts.get('certs'): - ctx = ssl.SSLCommon.CreateSSLContext(self.opts['certs']) - cnxOpts = {'ssl_context' : ctx} - cnxClass = ssl.SSLCommon.PlgHTTPSConnection - default_port = 443 - elif scheme == 'https': + if scheme == 'https': cnxOpts = {} cnxClass = httplib.HTTPSConnection default_port = 443 @@ -1737,21 +1732,21 @@ class ClientSession(object): certs['ca_cert'] = ca certs['peer_ca_cert'] = serverca - ctx = ssl.SSLCommon.CreateSSLContext(certs) - self._cnxOpts = {'ssl_context' : ctx} - # 60 second timeout during login - old_timeout = self._cnxOpts.get('timeout') - self._cnxOpts['timeout'] = 60 + # for ssl authentication we use custom a connection class so + # that we can pass our certs try: + old_cnx = (self._cnxClass, self._cnxOpts) + ctx = ssl.SSLCommon.CreateSSLContext(certs) + self._cnxOpts = {'ssl_context' : ctx} + # 60 second timeout during login + self._cnxOpts['timeout'] = 60 self._cnxClass = ssl.SSLCommon.PlgHTTPSConnection if self._port == 80 and not self.explicit_port: self._port = 443 sinfo = self.callMethod('sslLogin', proxyuser) finally: - if old_timeout is None: - del self._cnxOpts['timeout'] - else: - self._cnxOpts['timeout'] = old_timeout + # restore original connection type + self._cnxClass, self._cnxOpts = old_cnx if not sinfo: raise AuthError, 'unable to obtain a session' -- 1.9.3

1 1

[PATCH 1/2] logfile config option in kojid/kojivmd/kojira
by Mike McLean 06 Jun '15

06 Jun '15

--- builder/kojid | 8 +++++++- util/kojira | 2 +- vm/kojivmd | 8 +++++++- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/builder/kojid b/builder/kojid index dcd0840..49c1510 100755 --- a/builder/kojid +++ b/builder/kojid @@ -4411,6 +4411,7 @@ def get_options(): parser.add_option("--debug-mock", action="store_true", default=False, #obsolete option help=SUPPRESS_HELP) + parser.add_option("--logfile", help="specify log file") parser.add_option("--skip-main", action="store_true", default=False, help="don't actually run main") parser.add_option("--maxjobs", type='int', help="Specify maxjobs") @@ -4445,6 +4446,7 @@ def get_options(): 'minspace': 8192, 'admin_emails': None, 'log_level': None, + 'logfile': '/var/log/kojid.log', 'topdir': '/mnt/koji', 'topurl': None, 'workdir': '/var/tmp/koji', @@ -4509,6 +4511,9 @@ def get_options(): if getattr(options, name, None) is None: setattr(options, name, value) + if options.logfile in ('','None','none'): + options.logfile = None + #honor topdir if options.topdir: koji.BASEDIR = options.topdir @@ -4549,7 +4554,8 @@ def quit(msg=None, code=1): sys.exit(code) if __name__ == "__main__": - koji.add_file_logger("koji", "/var/log/kojid.log") + if options.logfile: + koji.add_file_logger("koji", options.logfile) #note we're setting logging params for all of koji* options = get_options() if options.log_level: diff --git a/util/kojira b/util/kojira index c18f63b..86e6b80 100755 --- a/util/kojira +++ b/util/kojira @@ -779,7 +779,7 @@ if __name__ == "__main__": if not os.access(options.logfile,os.W_OK): sys.stderr.write("Cannot write to logfile: %s\n" % options.logfile) sys.exit(1) - koji.add_file_logger("koji", options.logfile) + koji.add_file_logger("koji", options.logfile) #note we're setting logging for koji.* logger = logging.getLogger("koji") if options.debug: diff --git a/vm/kojivmd b/vm/kojivmd index cf44f0c..fa79e58 100755 --- a/vm/kojivmd +++ b/vm/kojivmd @@ -82,6 +82,7 @@ def get_options(): help="show xmlrpc debug output") parser.add_option("--skip-main", action="store_true", default=False, help="don't actually run main") + parser.add_option("--logfile", help="specify log file") parser.add_option("--maxjobs", type='int', help="Specify maxjobs") parser.add_option("--sleeptime", type='int', help="Specify the polling interval") parser.add_option("--admin-emails", help="Address(es) to send error notices to") @@ -108,6 +109,7 @@ def get_options(): 'minmem': 4096, 'vmuser': 'qemu', 'admin_emails': None, + 'logfile': '/var/log/kojivmd.log', 'workdir': '/tmp/koji', 'topurl': '', 'imagedir': '/var/lib/libvirt/images', @@ -153,6 +155,9 @@ def get_options(): if getattr(options, name, None) is None: setattr(options, name, value) + if options.logfile in ('','None','none'): + options.logfile = None + #make sure workdir exists if not os.path.exists(options.workdir): koji.ensuredir(options.workdir) @@ -1040,7 +1045,8 @@ class VMTaskManager(TaskManager): #################### if __name__ == "__main__": - koji.add_file_logger("koji", "/var/log/kojivmd.log") + if options.logfile: + koji.add_file_logger("koji", options.logfile) #note we're setting logging params for all of koji* options = get_options() if options.debug: -- 1.9.3

1 1

[PATCH 1/4] schema: add missing image channel from the default schema
by Dennis Gilmore 06 Jun '15

06 Jun '15

Signed-off-by: Dennis Gilmore <dennis(a)ausil.us> --- docs/schema.sql | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/schema.sql b/docs/schema.sql index 3582b87..aad24a9 100644 --- a/docs/schema.sql +++ b/docs/schema.sql @@ -183,6 +183,7 @@ INSERT INTO channels (name) VALUES ('maven'); INSERT INTO channels (name) VALUES ('livecd'); INSERT INTO channels (name) VALUES ('appliance'); INSERT INTO channels (name) VALUES ('vm'); +INSERT INTO channels (name) VALUES ('image'); -- Here we track the build machines -- each host has an entry in the users table also -- 2.4.2

2 4

Koji content generators
by Ken Dreyer 05 Jun '15

05 Jun '15

Hi Mike, For the Ceph project, we're very interested in building Debian packages on Red Hat's build system, so I've been kicking around the idea of a Koji buildDSCFromSCM task. When I mentioned this to one of the rel-eng guys in Red Hat, they mentioned that "content generators" might be able to handle our use-case in Ceph. This was the first time I'd heard of such a thing. In your Koji 2.0 planning thread you mentioned "content generators" as well. Can you provide more details about this? - Ken

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

koji-devel June 2015