infrastructure November 2014

infrastructure@lists.fedoraproject.org

36 participants
52 discussions

Plan for tomorrow's Fedora Infrastructure meeting (2014-11-20)
by Kevin Fenzi 20 Nov '14

20 Nov '14

The infrastructure team will be having it's weekly meeting tomorrow, 2014-11-13 at 18:00 UTC in #fedora-meeting on the freenode network. Suggested topics: #topic New folks introductions and Apprentice tasks. If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so in this part of the meeting. Don't be shy! #topic Freeze reminder We are in freeze, just to remind everyone. #topic Applications status / discussion Check in on status of our applications: pkgdb, fas, bodhi, koji, community, voting, tagger, packager, dpsearch, etc. If there's new releases, bugs we need to work around or things to note. #topic Sysadmin status / discussion Here we talk about sysadmin related happenings from the previous week, or things that are upcoming. #topic nagios/alerts recap Here we go over the last weeks alerts and see if we can find ways to make it so they don't happen again. #topic Upcoming Tasks/Items https://apps.fedoraproject.org/calendar/list/infrastructure/ #topic Open Floor Submit your agenda items, as tickets in the trac instance and send a note replying to this thread. More info here: https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings Thanks kevin

1 1

Meeting Agenda Item: Introduction for alexove
by Alex Irmel Oviedo Solis 20 Nov '14

20 Nov '14

* FAS account nickname : alexove * IRC nickname : alexove * What can to offer I have knowledge of Fedora and CentOS and deployed some small servers (Proxies, web servers, vsftp, etc) * Which I would like to learn I want to help efficiently to sysadmin group and gain experience for the community. -- *"Una alegría compartida se transforma en doble alegría; una pena compartida, en media pena."*--> http://alexove.mallkudev.com --> Celular (Claro) : 984-305721 --> Celular (Movistar): 959-625-001 --> Sigueme en Twitter: http://twitter.com/alleinerwolf --> http://groups.google.com.pe/group/mosoq_kallpa

1 0

Freeze Break Request: point pkgdb2 memcached config to memcached01
by Pierre-Yves Chibon 20 Nov '14

20 Nov '14

Hi all, Looking into the configuration of dogpile for elections 2.4 this morning I found out that dogpile is incorrectly configured for pkgdb2. Below is the change I would like to apply and run to fix this: diff --git a/ roles/pkgdb2/templates/pkgdb2.cfg b/ roles/pkgdb2/templates/pkgdb2.cfg index 828d41d..85885fb 100644 --- a/ roles/pkgdb2/templates/pkgdb2.cfg +++ b/ roles/pkgdb2/templates/pkgdb2.cfg @@ -29,7 +29,7 @@ ADMIN_GROUP = ['sysadmin-main', 'cvsadmin'] PKGDB2_CACHE_BACKEND = 'dogpile.cache.memcached' PKGDB2_CACHE_KWARGS = { 'arguments': { - 'url': "127.0.0.1:11211", + 'url': "memcached01:11211", } } +1s ? Thanks, Pierre

4 4

Freeze Break Request: Better rsync to bapp02::docs
by Pete Travis 20 Nov '14

20 Nov '14

The proxies are currently not purging deleted content when rsyncing docs.fp.o. I'd like to fix that: diff --git a/modules/fedora-docs/manifests/init.pp b/modules/fedora-docs/manifests/init.pp index 1d618b0..6efc559 100644 --- a/modules/fedora-docs/manifests/init.pp +++ b/modules/fedora-docs/manifests/init.pp @@ -33,7 +33,7 @@ define fedora-docs::proxy($website) { cron { "sync-fedora-docs": # TODO: Make add some locking to this. - command => "/usr/bin/rsync -aSHPq --exclude=.git/objects/ bapp02::docs/ /srv/web/docs.fedoraproject.org/", + command => "/usr/bin/rsync --delete -aSHPq --exclude=.git/objects/ bapp02::docs/ /srv/web/docs.fedoraproject.org/", user => "root", minute => 10, ensure => present, -- -- Pete Travis - Fedora Docs Project Leader - 'randomuser' on freenode - immanetize(a)fedoraproject.org

3 3

Freeze Break Request: new depcheck on taskotron clients
by Tim Flink 20 Nov '14

20 Nov '14

I thought that I had updated the depcheck task on production taskotron a while back but it turns out that never happened and depcheck is hitting failures in production that it shouldn't be hitting. Original issue is: https://phab.qadevel.cloud.fedoraproject.org/T366 The fix was reviewed before updating dev/stg and has been running there more than 2 weeks now without issue. I'd like to update the depcheck task in production to fix some recurring issues that some devs have been complaining about. Diff is at the end of this email. Tim diff --git a/depcheck/__init__.py b/depcheck/__init__.py index ccac8a4..609eb13 100755 --- a/depcheck/__init__.py +++ b/depcheck/__init__.py @@ -36,6 +36,7 @@ import sys import argparse import tempfile import platform +import logging from . import depcheck from . import metadata @@ -91,7 +92,7 @@ def run(rpms, arch, repos, report_format="updates", workdir=None): arch = 'i686' # handle data formatting for repo metadata (list of dicts) - log.logger.debug("Prepairing metadata download") + log.logger.debug("Preparing metadata download") handled_repos = {} for repo, path in repos.items(): log.logger.debug(" - %r: %r, %r", repo, arch, path) @@ -121,7 +122,7 @@ def run(rpms, arch, repos, report_format="updates", workdir=None): # Sometimes, depcheck gets run with no rpms if not run_result: - log.logger.info("No results") + log.logger.warn("No results") output = squash_results.format_output(run_result, arch, report_format) log.logger.info(output) @@ -131,6 +132,7 @@ def run(rpms, arch, repos, report_format="updates", workdir=None): def depcheck_cli(): parser = argparse.ArgumentParser() parser.add_argument("rpm", nargs='+', help="RPMs to be tested") + parser.add_argument("-d", "--debug", action="store_true", help="enable debug logging for depcheck") parser.add_argument("-a", "--arch", choices=["i386", "x86_64", "armhfp"], help=""" Architecture to be tested. If omitted, defaults to the current machine's architecture. @@ -143,9 +145,17 @@ def depcheck_cli(): parser.add_argument("-f", "--format", choices=["rpms", "updates"], help=""" Specify output format to override configuration. """) - + parser.add_argument("-w", "--workdir", help="workdir to use for depcheck") args = parser.parse_args() + + # initialize logging when entering depcheck via cli - this won't be executed + # otherwise but in that case, taskotron is responsible for initialization + logging.basicConfig() + + if args.debug: + log.logger.setLevel(logging.DEBUG) + # autodetect arch if none is passed in arch = args.arch if arch is None: @@ -155,8 +165,8 @@ def depcheck_cli(): repos = dict([(repo, repo) for repo in args.repos]) try: - print run(args.rpm, arch, repos, args.format) - except KeyboardInterrupt, e: + run(args.rpm, arch, repos, args.format, args.workdir) + except KeyboardInterrupt: print >> sys.stderr, ("\n\nExiting on user cancel.") sys.exit(1) diff --git a/depcheck/depcheck.py b/depcheck/depcheck.py index bb0b37a..e5ba15c 100755 --- a/depcheck/depcheck.py +++ b/depcheck/depcheck.py @@ -30,7 +30,6 @@ import solv import tempfile from . import Pass, Fail -from . import exc from . import log @@ -77,6 +76,8 @@ class Depcheck(object): checking dependencies for the supplied repositories """ + log.logger.debug('preparing libsolv repos for depcheck run') + for repo in self.repos: primary = open(self.repos[repo]['primary'], 'rb') filelist = open(self.repos[repo]['filelists'], 'rb') @@ -85,37 +86,36 @@ class Depcheck(object): self.solvrepo.add_rpmmd(filelist, None, solv.Repo.REPO_EXTEND_SOLVABLES) - def _check_rpm(self, rpmfile): + def _check_rpm(self, rpmfile, solvable): """Check a single rpm file against prepared repodata :param rpmfile: filename of rpm file to check :type rpmfile: str + :param solvable: libsolv solvable from solvpool for the rpm to check + :type solvable: libsolv.Solver.Solvable """ - solvable = self.solvrepo.add_rpm(rpmfile) - self.solvpool.addfileprovides() - self.solvpool.createwhatprovides() + # the job setup here is mostly taken from libsolv's installcheck tool + # create a job using the predetermined solvable in the repo job = self.solvpool.Job(solv.Job.SOLVER_INSTALL | solv.Job.SOLVER_SOLVABLE, - solvable.id) + solvable.id) + self.solvpool.set_flag(solv.Solver.SOLVER_FLAG_IGNORE_RECOMMENDED, 1) - # important note - make sure to keep the solver in the same scope as - # the problem analysis code or else python-solv segfults will ensue - solvpool_solver = self.solvpool.Solver() - solv_problems = solvpool_solver.solve([job]) + # solve the deps for the input solvable's rpm and look for problems + solver = self.solvpool.Solver() + solv_problems = solver.solve([job]) if len(solv_problems) == 0: - return Pass, [] - + return {'result': Pass,'details':[]} else: problem_strs = [] for problem in solv_problems: probrules = problem.findallproblemrules() - for rule in probrules: ruleinfos = rule.allinfos() for info in ruleinfos: problem_strs.append(info.problemstr()) + return {'result':Fail, 'details':problem_strs} - return Fail, problem_strs def check_rpm_deps(self, rpmfiles): """Check dependencies of the given rpms against supplied repodata @@ -125,26 +125,33 @@ class Depcheck(object): :returns: dictionary of results {rpm: {'results': Pass/Fail, 'details': []}} """ - self._prepare_libsolv() + log.logger.debug('adding all rpms to libsolv repos and preparing solvpool') - run_result = {} + # add all rpmfiles that we're checking to the libsolv repo and save the + # solvable that is assigned to them. + # this makes libsolv aware of all the incoming rpms and allows it to + # check for any dep breakage between the rpms + self._prepare_libsolv() + solvables = {} for rpmfile in rpmfiles: - log.logger.debug('running depcheck for %s' % rpmfile) - result_state, result_details = self._check_rpm(rpmfile) + rpmname = rpmfile.split('/')[-1] + solvable = self.solvrepo.add_rpm(rpmfile) + solvables[rpmname] = solvable - if rpmfile in run_result: - if result_state != run_result[rpmfile]['result']: - #TODO maybe do something saner, ideas welcomed - raise exc.DepcheckException("RPM %r has multiple conflicting results" % rpmfile) - else: - run_result[rpmfile]['details'].append(result_details) + # now that the rpms have been added, prepare the solvpool + self.solvpool.addfileprovides() + self.solvpool.createwhatprovides() - else: - run_result[rpmfile] = {'result': result_state, - 'details': result_details} - return run_result + # check each rpm using its solvable that was assigned while adding to + # the repo + results = {} + for rpmname in solvables: + log.logger.debug('running depcheck for %s' % rpmname) + run_result = self._check_rpm(rpmname, solvables[rpmname]) + results[rpmname] = run_result + return results diff --git a/depcheck/squash_results.py b/depcheck/squash_results.py index 7376f0d..30330ca 100644 --- a/depcheck/squash_results.py +++ b/depcheck/squash_results.py @@ -1,4 +1,3 @@ -import os import json from . import Pass, Fail @@ -7,7 +6,6 @@ from . import log from libtaskotron.koji_utils import KojiClient from libtaskotron.bodhi_utils import BodhiUtils from libtaskotron import check -from libtaskotron.rpm_utils import rpmformat def _squash_rpms_to_builds(run_result, koji_client = None): """ diff --git a/tests/depcheck_scenarios b/tests/depcheck_scenarios index f1e5d57..3ad160a 160000 --- a/tests/depcheck_scenarios +++ b/tests/depcheck_scenarios @@ -1 +1 @@ -Subproject commit f1e5d57b0e68c53bd05c1b4d6eab3078706d269e +Subproject commit 3ad160ad3145dc4595286291e5e43e2d91737ea9 diff --git a/tests/functest_depcheck_scenarios.py b/tests/functest_depcheck_scenarios.py index dc69e07..3f5733d 100644 --- a/tests/functest_depcheck_scenarios.py +++ b/tests/functest_depcheck_scenarios.py @@ -36,9 +36,12 @@ def get_depcheck_scenarios(): for dircontent in dircontents: if len(dircontent[2]) == 0: continue + if dircontent[1] == 'known_broken': + continue for infile in dircontent[2]: filename = '%s/%s' % (dircontent[0], infile) - yamlfiles.append((scenario_type == 'pass', filename)) + if infile.split('.')[-1] in ['yml', 'yaml']: + yamlfiles.append((scenario_type == 'pass', filename)) return yamlfiles diff --git a/tests/test_depcheck.py b/tests/test_depcheck.py index d27d011..909b2ad 100644 --- a/tests/test_depcheck.py +++ b/tests/test_depcheck.py @@ -36,10 +36,10 @@ class TestDepcheck(object): monkeypatch.setattr(test_depcheck, '_prepare_libsolv', Dingus()) rpmfiles = ['rpm_1', 'rpm_2'] - def fake_check_rpm(rpmfile): + def fake_check_rpm(rpmfile, solvable): results = { - rpmfiles[0]: (Pass, []), - rpmfiles[1]: (Fail, ['message']), + rpmfiles[0]: {'result':Pass, 'details':[]}, + rpmfiles[1]: {'result':Fail, 'details':['message']}, } return results.get(rpmfile, (Pass, [])) @@ -54,25 +54,3 @@ class TestDepcheck(object): assert outcome['rpm_2']['result'] == Fail assert outcome['rpm_2']['details'] == ['message'] - def test_check_rpm_deps_conflicting_results(self, monkeypatch): - """ - Checks whether rpm_deps raises Exceptions when conflicting - results are provided for one RPM. - """ - - test_depcheck = depcheck.Depcheck(None, None, "fakedir") - monkeypatch.setattr(test_depcheck, '_prepare_libsolv', Dingus()) - - self.tmp_result = Pass - def fake_check_rpm(rpmfile): - if self.tmp_result == Pass: - self.tmp_result = Fail - else: - self.tmp_result = Pass - return (self.tmp_result, []) - - monkeypatch.setattr(test_depcheck, '_check_rpm', fake_check_rpm) - - with pytest.raises(exc.DepcheckException): - outcome = test_depcheck.check_rpm_deps(['rpm1', 'rpm1']) -

3 3

Freeze Break Request: Add collectd monitoring for memcached
by Ralph Bean 19 Nov '14

19 Nov '14

Hi all- Memcached has been locking up on us lately. If I remember correctly it was Saturday morning, then again Monday morning, and it did it again today (which manifested in a badges.fedoraproject.org outage for users). We currently don't have any collectd monitoring directly on memcached. Here's an ansible patch that should add it. Since we're in freeze, this would need two +1s in order to be able to be applied. If, for some reason it was broken or hosed up, I could revert it by removing the /etc/collectd.d/memcached.conf file on both memcached servers and subsequently restarting the memcached daemon. Patch follows: From 6535e0344f4db5b0eb00aeb37007c92f471d7224 Mon Sep 17 00:00:00 2001 From: Ralph Bean <rbean(a)redhat.com> Date: Wed, 19 Nov 2014 14:37:50 +0000 Subject: [PATCH] Add collectd monitoring for the memcached daemon. --- playbooks/groups/memcached.yml | 1 + roles/collectd/memcached/files/memcached.conf | 6 ++++++ roles/collectd/memcached/tasks/main.yml | 8 ++++++++ 3 files changed, 15 insertions(+), 0 deletions(-) create mode 100644 roles/collectd/memcached/files/memcached.conf create mode 100644 roles/collectd/memcached/tasks/main.yml diff --git a/playbooks/groups/memcached.yml b/playbooks/groups/memcached.yml index 5fc955b..90d52b0 100644 --- a/playbooks/groups/memcached.yml +++ b/playbooks/groups/memcached.yml @@ -31,6 +31,7 @@ - hosts - fas_client - collectd/base + - collectd/memcached - sudo - memcached diff --git a/roles/collectd/memcached/files/memcached.conf b/roles/collectd/memcached/files/memcached.conf new file mode 100644 index 0000000..d59815b --- /dev/null +++ b/roles/collectd/memcached/files/memcached.conf @@ -0,0 +1,6 @@ +LoadPlugin memcached + +<Plugin memcached> + Host "localhost" + Port "11211" +</Plugin> diff --git a/roles/collectd/memcached/tasks/main.yml b/roles/collectd/memcached/tasks/main.yml new file mode 100644 index 0000000..3023af7 --- /dev/null +++ b/roles/collectd/memcached/tasks/main.yml @@ -0,0 +1,8 @@ +--- + +- name: Copy in the memcached collectd config + copy: src=memcached.conf dest=/etc/collectd.d/memcached.conf + tags: + - collectd + - memcached + notify: restart collectd -- 1.7.2.1

5 8

Retroactive freeze break: pykickstart downgrade on builders
by Kevin Fenzi 19 Nov '14

19 Nov '14

Greetings. When I applied one last round of updates to builders yesterday, I also applied a newer pykickstart update that was in our builder repo. Dennis had built it to solve an issue, but it turned out to not be the solution and caused problems, so he downgraded back to the f20 version. My re-upgrading it caused all the nightly livecd's composes to fail. ;( I've downgraded it again already, but wanted to note it here and get some retroactive +1s for doing so. :) kevin

3 2

Fixing livecd ISO labels (bz 1145264)
by Cole Robinson 19 Nov '14

19 Nov '14

Hi all, The ISO volume ID for the Fedora 21 workstation livecd is 'Fedora-Live-Workstation-x86_64-2', which doesn't actually say 21 in it, and libosinfo can't use it to detect the fedora version for boxes/virt-manager. Koji is responsible for generating the volume ID, and there's a bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=1145264 I also sent a (untested) koji patch to buildsys list: https://lists.fedoraproject.org/pipermail/buildsys/2014-October/004411.html However neither have received a response, so I'm reaching out here. If this isn't fixed before F21 GA, we are stuck with the crappy volume ID forever and tools that are depending on it are in a tough spot. Is there any process to escalate this and get the fix deployed to infrastructure? (FWIW I've just proposed the bug as a freeze exception, but presumably it will take more coordination that that) Thanks, Cole

5 8

Freeze break: abrt on memcached02
by Kevin Fenzi 18 Nov '14

18 Nov '14

Greetings. Several times recently, memcached02 has had memcached segfault on it. Before the freeze I fixed things so systemd will just restart it if this happens again, which should result in almost no outage. I also filed a bug on the rhel7 memcached about the crashes. They would like us to install/run abrt to catch the backtrace when it happens again. Therefore, I would like to do the following on memcached02: - 'yum install abrt-cli libreport-plugin-mailx' - 'systemctl enable abrtd' - 'systemctl start abrtd' This should catch the crash and let us report the stacktrace. +1s? kevin

4 4

Elections: how to organize?
by Honza Horak 17 Nov '14

17 Nov '14

Hi guys, my google-fu is not great today, since I fail to search any steps we need to do if we want to organize elections within Fedora community. Env & Stacks WG agreed in the Charter to have elections aligned with fesco elections, which seems like beginning of the next year. What do we need to do in order to prepare the voting system? Is a trac ticket on infra good enough? Honza

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure November 2014