rel-eng February 2015

rel-eng@lists.fedoraproject.org

16 participants
127 discussions

[PATCH] srpm-excluded-arch.py enhancement
by Jakub Cajka 25 May '15

25 May '15

Hello, I have prepared enhancement of srpm-excluded-arch.py based on sharkcz's ideas(if I'm not mistaken :) ). I have moved this script from optparse to argparse, added possibility to run this script over multiple paths/directories(for example updates and release, etc.) and prepare list of excluded packages based on most recent ENVR. Also is possible to issue warning(to stderr) if package have multiple versions present and state of exclusion have changed(to warn about possible unintentional changes). Command syntax should be the same as original version, except switch -w, which adds up mentioned warnings(outputted to stderr), and --path accepts more than one path now, expansion is done up to 3 levels using python glob. I hope it brings desired improvements( and no bugs :)), looking forward for any feedback,... Patch follows. Best regards, Jakub >From b91af25ac1a89b4892cc1ffb4af60346cdb40eca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20=C4=8Cajka?= <jcajka(a)redhat.com> Date: Mon, 23 Feb 2015 15:57:48 +0100 Subject: [PATCH] srpm-excluded-arch.py can check multiple paths --- scripts/srpm-excluded-arch.py | 97 ++++++++++++++++++++++++++++++------------- 1 file changed, 68 insertions(+), 29 deletions(-) diff --git a/scripts/srpm-excluded-arch.py b/scripts/srpm-excluded-arch.py index 2125517..718f7c2 100755 --- a/scripts/srpm-excluded-arch.py +++ b/scripts/srpm-excluded-arch.py @@ -3,39 +3,54 @@ # srpm-exlcuded-arch: is a tool to give you a list of packge names that # are excluded on the given arches. access to a srpm tree is needed. # -# Copyright (C) 2008-2013 Red Hat, Inc. +# Copyright (C) 2008-2015 Red Hat, Inc. # SPDX-License-Identifier: GPL-2.0+ # # Authors: # Dennis Gilmore <dennis(a)ausil.us> +# Jakub Cajka <jcajka(a)redhat.com> import rpm import os import sys -import optparse +import argparse import glob +import types -OptionParser = optparse.OptionParser -usage = "%prog [options]" -parser = OptionParser(usage=usage) -parser.add_option("-a", "--arches", - help="space or command separated list of arches to check for") -parser.add_option("--path", default='./', - help="path to dir with srpms, default current directory") -(options, args) = parser.parse_args() -arches = options.arches -if arches == None: - print "You must pass arches to check for in." - sys.exit() -else: - if arches.find(',') == -1: - arches = arches.split(' ') - else: - arches = arches.split(',') - -srpm_path = options.path -srpms = glob.glob('%s/*.rpm' % srpm_path) -pkglist = [] +def rpmvercmp((e1, v1, r1), (e2, v2, r2)): + """find out which build is newer""" + rc = rpm.labelCompare((e1, v1, r1), (e2, v2, r2)) + if rc == 1: + #first evr wins + return 1 + elif rc == 0: + #same evr + return 0 + else: + #second evr wins + return -1 + +parser = argparse.ArgumentParser() +parser.add_argument('-a', '--arches', nargs='+', required=True) +parser.add_argument('-w', '--withcheck', action='store_true') +parser.add_argument('--path', nargs='+', default='./') +args = parser.parse_args() + +# be previous version compatible i.e. -a "arch arch ..." +if len(args.arches) == 1 and type(args.arches[0]) is types.StringType: + if args.arches[0].find(',') == -1: + args.arches = args.arches[0].split(' ') + else: + args.arches = args.arches[0].split(',') + +arches = args.arches +srpms = [] + +for srpm_path in args.path: + srpms += glob.glob('%s/*/*/*.rpm' % srpm_path) + srpms += glob.glob('%s/*/*.rpm' % srpm_path) + srpms += glob.glob('%s/*.rpm' % srpm_path) +pkglist = {} for srpm in srpms: """Return the rpm header.""" @@ -52,14 +67,38 @@ for srpm in srpms: if arch not in hdr[rpm.RPMTAG_EXCLUSIVEARCH]: if arch not in ExcludeArch: ExcludeArch.append(arch) - if ExcludeArch == arches: - pkgname = hdr[rpm.RPMTAG_NAME] - if pkgname not in pkglist: - pkglist.append(pkgname) - #print "Excluding: %s" % pkgname + + pkgname = hdr[rpm.RPMTAG_NAME] + pkgevr = ('0'if not hdr[rpm.RPMTAG_EPOCH] else str(hdr[rpm.RPMTAG_EPOCH]), + hdr[rpm.RPMTAG_VERSION], + hdr[rpm.RPMTAG_RELEASE]) + if not pkgname in pkglist: + pkglist[pkgname] = [] + excluded = set(ExcludeArch) == set(arches) + + pkglist[pkgname].append({'evr':pkgevr, 'excluded':excluded}) output = "" +warning = "" + for pkg in pkglist: - output += pkg + " " + pkglist[pkg].sort(cmp=rpmvercmp, key=lambda x: x['evr'], reverse=True) + if pkglist[pkg][0]['excluded']: + output += pkg + " " + + if args.withcheck: + last = None + change = "" + for srpm in pkglist[pkg]: + if last and last['excluded'] != srpm['excluded']: + change += str(srpm)+'->'+str(last) + last = srpm + if change: + warning += pkg + " " + change + "\n" + +if args.withcheck and warning: + warning = "WARNING: Fellowing packages have changed their exclude state:\n"+warning + sys.stderr.write(warning) print output + -- 1.9.3

3 3

FAD proposal
by Paul W. Frields 14 May '15

14 May '15

One of Matthew Miller's goals for Fedora is to diversify our release deliverables. This is one reason the Fedora Engineering team is bringing on a full-time person soon to work on release tools. We want to contribute to the Fedora release engineering effort too (as opposed to simply asking existing folks to do more). :-) I would expect over the first month or two in that role, our release tools person will likely be: * looking at the field of deliverables needed, across not just the products but also at containers and Project Atomic needs * getting familiar with any current release processes as needed * discussion with stakeholders in the project, including rel-eng, mattdm, infrastructure, and others, on this list and elsewhere about requirements During this time, I'd expect there will be some decisions on process, architecture, etc. To kickstart development, I'd like to set up a FAD where key community members, including developers and release engineers, can start (or continue) working on actual code and supporting infrastructure. I started a wiki page here for interested people to sign up: https://fedoraproject.org/wiki/FAD_Release_Tools_and_Infrastructure_2015 There are some proposed dates listed on the wiki page. None have been fully decided yet. I have preliminary approval from the FPL, and I'm awaiting OSAS input on budget availability, and any time restrictions from their POV. There is a good chance we will do this at the Red Hat Tower (HQ) in Raleigh, NC USA. The event will not be restricted to USA people, although we'll have to select a roster of people to meet goals. There's a specific section in the page for you to write your name and any restrictions on dates. This will help us make the best date/location plan. https://fedoraproject.org/wiki/FAD_Release_Tools_and_Infrastructure_2015#Po… -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ The open source story continues to grow: http://opensource.com

1 4

#6017: please give kushal permissions to do scratch cloud image builds in koji
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6017: please give kushal permissions to do scratch cloud image builds in koji ----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ If I remember right (and I admit I remember only vaguely) there are special permissions required to build images in koji. Please give these to kushal so he can create scratch images to test things out. Thank you! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6017> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

2 11

#6040: system to run regular orphaned packages status reports from
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6040: system to run regular orphaned packages status reports from -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ Currently I am creating status reports for orphaned packages from autosign01, but this should move to its own system. Requirements: - run cron jobs - send e-mails - fast access to koji and pkgdb - access to mash repos - possible in the future: access to a directory served by a web server for HTML based status reports -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6040> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 7

#5665: Providing a master CHECKSUMS file for boot.fpo
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5665: Providing a master CHECKSUMS file for boot.fpo -----------------------------+------------------------ Reporter: shaiton | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Alpha | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, as requested to webmasters[1], could we get a CHECKSUMS file (as provided for other Fedora releases) including all checksums? It also needs to be signed of course. You might decide that it is not needed, just forwarded the query. [1] https://lists.fedoraproject.org/pipermail/websites/2013-July/011535.html -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5665> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5939: f21-kde koji tag/target
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5939: f21-kde koji tag/target ---------------------+------------------------ Reporter: rdieter | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: koji Keywords: | Blocked By: Blocking: | ---------------------+------------------------ Please create a f21-kde koji tag/target. kde-sig will continue to have monthly kde4 and kde frameworks 5 updates to work on, so having this till be greatly beneficial. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5939> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#1107: auto-cleanup rawhide trees
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#1107: auto-cleanup rawhide trees -------------------------+-------------------------------------------------- Reporter: notting | Owner: rel-eng(a)lists.fedoraproject.org Type: enhancement | Status: new Milestone: | Component: koji Keywords: | -------------------------+-------------------------------------------------- Rawhide tree expiry is manual at the moment, unless I missed something. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/1107> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 8

#6100: create bsd-style checksum files for CHECKSUM files
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6100: create bsd-style checksum files for CHECKSUM files -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Create the files like https://getfedora.org/en/static/checksums/Fedora- Workstation-21-x86_64-CHECKSUM using "--tag", this adds an tag that shows which digest algorithm was used: {{{ $ sha256sum --tag /dev/null SHA256 (/dev/null) = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 }}} This still allows to verify the files as usual. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6100> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#6112: Migrate koji to well-known CA certificate
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6112: Migrate koji to well-known CA certificate -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Final | Component: koji Keywords: | Blocked By: Blocking: 6111 | -----------------------------+------------------------ Depends on: https://fedorahosted.org/koji/ticket/312 -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6112> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#6118: Please create a f22-gnome side tag
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6118: Please create a f22-gnome side tag -----------------------------+------------------------ Reporter: kalev | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 22 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, Please create a new f22-gnome side tag and build target. We'll be building GNOME 3.15.91 early next week and this would help ensure it doesn't cause instabilities in F22 proper while we're preparing the update. Thanks, Kalev -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6118> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

rel-eng February 2015