infrastructure July 2019

infrastructure@lists.fedoraproject.org

19 participants
24 discussions

BYO Jenkins for Pagure CI?
by Justin W. Flory 16 Jul '19

16 Jul '19

Hi all, I was looking into Pagure CI with Jenkins, but it wasn't clear if there is a Fedora-provided Jenkins server or if someone needs to bring their own Jenkins build infrastructure for Pagure CI. If there is a Fedora-provided Jenkins, could someone point me towards documentation on requesting an account / getting started? Thanks! -- Cheers, Justin W. Flory justinwflory.com TZ=America/Chicago Pronouns: he/him/his

3 2

Anitya 0.15.0 (stg.release-monitoring.org) deployed on staging
by Michal Konecny 15 Jul '19

15 Jul '19

Hi everybody, I want to announce that the Anitya 0.16.0 was deployed on staging (https://stg.release-monitoring.org/) Feel free to test it. This version solves the issue with error 500 when accessing packages within specific distribution. Full changelog could be found here https://github.com/release-monitoring/anitya/releases/tag/0.16.0 Regards, mkonecny

1 0

OSBS buildroot image + flatpak-module-tools-0.10.1
by Owen Taylor 12 Jul '19

12 Jul '19

[ Summary: if you are going to rebuild the OSBS buildroot image in the near future, talk to me to make sure things don't break ] I just submitted a new version of flatpak-module-tools to Bodhi. This updates flatpak-module-tools to use the libmodulemd v2 API - this was forced because module-build-service hsa switched to use the v2 API. *However* this update is incompatible with the current version of atomic-reactor - which has references to the libmodulemd v1 API. So, if the OSBS buildroot image is rebuilt after flatpak-module-tools-0.10.1 hits stable, but before atomic-reactor is fixed, Flatpak builds are going to break. Submitting the necessary atomic-reactor fixes upstream blocks on getting a libmodulemd v2 package into EPEL-7: https://bugzilla.redhat.com/show_bug.cgi?id=1724271 - but we can land them as a local patch to the Fedora atomic-reactor package if necessary. Owen

1 0

OpenShift Resources
by Shraddha Agrawal 11 Jul '19

11 Jul '19

Hello! I was working on deploying Fedora Happiness Packets on OpenShift. I was looking for resources for the same. I followed Clement's article to get an initial idea. https://fedoramagazine.org/deploy-a-django-rest-service-on-openshift/ If you could help me find OpenShift resources on how to deploy a Django App that'll be great. Thanks Shraddha

2 1

proposal: move pagure01 and pagure-stg01 to rdu2
by Kevin Fenzi 10 Jul '19

10 Jul '19

Greetings. I'd like to propose we migrate pagure01 (which is pagure.io) and pagure-stg01 (pagure-stg.io) from osuosl to the community cage in rdu2. We have had odd networking problems between .cz and osuosl and at times between phx2 and osuosl. Moving things would allow us to: drop the pagure-proxy in ibiblio (which currently proxies all traffic to the osuosl instances to avoid the networking issues from cz). move forward with adding repospanner to ansible and have it appear in pagure.io so people could do PR's and CI and other fun things against our ansible repo. Why not move it to openshift? Well, we could, but thats going to take cycles we don't have right now, so a simple migration seems much easier and gets us a lot of good. Why not move it to phx2? Well, we would need to get a bunch of ports opened for it's use and we may well be moving resources out of phx2 at some point, so we really don't want to add another thing to move out. Why not make a new instance in rdu2 and migrate just data to it? Again, we could, but at this point a brute force migration seems like a better use of our cycles than figuring exactly what needs synced over. If we really don't want the downtime we can look more into this, I just worry that it will take a lot of time to make sure we have everything synced and setup right. If folks are ok with this, I'd like to consider taking it down friday late afternoon and migrating it over this weekend. It would have downtime for however long it takes to sync the disk over. I can do pagure-stg01 as a test run later this week if we like. Thoughts? kevin

4 4

Proposed Meeting Agenda for 2019-07-11
by Stephen John Smoogen 10 Jul '19

10 Jul '19

= Preamble = The infrastructure team will be having its weekly meeting tomorrow, 2019-07-11 at 15:00 UTC in #fedora-meeting-1 on the freenode network. We have a gobby document at https://infinote.fedoraproject.org/cgit/infinote/tree/fedora-infrastructure… which can be edited for the agenda (see: https://fedoraproject.org/wiki/Gobby ) Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. = Introduction = We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2019-07-11) #meetingname infrastructure #topic aloha #chair nirik pingou puiterwijk relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs = Let new people say hello = #topic New folks introductions #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted = Status / Information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info bowlofeggs is on extended leave #info cverna will be going on extended leave #info abombard will be going on extended leave #info other vacations/leave/etc? #info Flock2Fedora 2019-08-08 -> 2019-08-11 #info No site trip to PHX2 #info Red Hat is now owned by IBM. = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic Oncall #info https://fedoraproject.org/wiki/Infrastructure/Oncall #info smooge is on call from 2019-07-04 -> 2019-07-11 #info ?????? is on call from 2019-07-11 -> 2019-07-18 #info Summary of last week: (from smooge ) #topic Monitoring discussion #info https://nagios.fedoraproject.org/nagios #info Go over existing out items and fix #topic Tickets discussion #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket Go thru each ticket one by one ===== Put all topics for discussion under here ===== Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #info = Meeting end stuff = #topic Open Floor #endmeeting -- Stephen J Smoogen.

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 10 Jul '19

10 Jul '19

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2019-07-11 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before or view it at https://infinote.fedoraproject.org/cgit/infinote/tree/fedora-infrastructure… Source: https://apps.fedoraproject.org/calendar/meeting/22/

1 0

compose connection issues
by Kevin Fenzi 09 Jul '19

09 Jul '19

Greetings everyone. So, we have seen some very sporadic connection issues that have been breaking composes. I'm at a loss how to debug it or figure out whats going on, so I thought I would collect information here and see if anyone had any ideas. Of the last 11 doomed rawhide composes: Fedora-Rawhide-20190622.n.0 - loop issue [0] Fedora-Rawhide-20190622.n.1 - loop issue [0] Fedora-Rawhide-20190623.n.0 - armv7 http2 framing error [1] Fedora-Rawhide-20190624.n.0 - loop issue [0] Fedora-Rawhide-20190626.n.0 - kde broken deps Fedora-Rawhide-20190626.n.1 - loop issue [0] Fedora-Rawhide-20190627.n.0 - loop issue [0] Fedora-Rawhide-20190627.n.1 - pkg download issue [2] Fedora-Rawhide-20190630.n.0 - armv7 http2 framing error[1] Fedora-Rawhide-20190630.n.1 - a x86_64 live downloading issue [3] Fedora-Rawhide-20190702.n.0 - pkg download issue [2] [0] mock in old chroot has 5 loop devices available, if you try and use more than that they just don't appear in the chroot. For some reason loop devices aren't getting cleaned up as easily or we hit multiple composes per builder and it goes over 5. I patched our mock to have 11 of them. :) See: [1] This looks like: DEBUG util.py:585: BUILDSTDERR: [MIRROR] libavc1394-0.5.4-10.fc30.armv7hl.rpm: Curl error (16): Error in the HTTP2 framing layer for https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20190623.… [] DEBUG util.py:585: BUILDSTDERR: [FAILED] libavc1394-0.5.4-10.fc30.armv7hl.rpm: No more mirrors to try - All mirrors were already tried without success DEBUG util.py:585: BUILDSTDERR: Unable to create appliance : Unable to download from repo : Cannot download Packages/l/libavc1394-0.5.4-10.fc30.armv7hl.rpm: All mirrors were tried [2] This looks like: (it's a screenshot, see: https://kojipkgs.fedoraproject.org//work/tasks/3061/35873061/screenshot.ppm ) but basically: Failed to download the following packages: Cannot download Packages/s/systemd-bootchart-233-4.fc30.x86_64.rpm: All mirrors were tried. [3] This looks like: DEBUG util.py:585: BUILDSTDERR: 2019-07-01 03:02:48,081: Non interactive installation failed: Failed to download the following packages: Cannot download Packages/i/iwl3945-firmware-15.32.2.9-97.fc31.noarch.rpm: All mirrors were tried. A short summary of our setup: A builder, which might be running a mock chroot or a vm requests something from kojipkgs. kojipkgs resolves for them in dns to 2 ip's: proxy101 or proxy110. Those run apache which proxies incoming requests for that host to haproxy also running on those hosts. haproxy checks for liveness of the two backend kojipkgs servers: kojipkgs01 and kojipkgs02. The request will go to one of the two or whichever is up. kojipkgs01/02 have varnish listening on port 80, so if they have the thing in cache is just replies with that. Otherwise it makes a query against a locally running apache to fetch the item. That apache reads the package from a nfs mount of all the koji data those machines have on them. So, lots of layers, but erverything should be pretty reliable. And most of the time it is. The builders download tons and tons of things just fine via this setup without problems usually. At the point all these are failing, it would be running the rawhide curl/dnf stack, not the native f29 one on the builders, so if it's a curl problem I would expect it to happen more often. So, how can we debug or mitigate this? any ideas? kevin

2 2

Retire keys.fedoraproject.org?
by Kevin Fenzi 06 Jul '19

06 Jul '19

Hey everyone, As some of you may have read: https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f and https://dkg.fifthhorseman.net/blog/openpgp-certificate-flooding.html or other media reports about vulnerabilities of the current gpg keyserver software/network/policy. TLDR: Someone can (and has been) flooding sks keyservers with poisoned certs. Users that download from sks keyservers may well find gpg just stops working, hangs, or breaks in terrible ways. The SKS software is no longer maintained and because the policy is 'never delete anything' there's likely no way to mitigate the attacks. I've cc'ed nb here for his take on things, but as I read it, it might be best to just retire the keys.fedoraproject.org service at least for now to avoid breaking users or telling them we have a service they should trust when they really... should not. Thoughts? kevin

5 4

Re: Retire keys.fedoraproject.org?
by Julen Landa Alustiza 05 Jul '19

05 Jul '19

Clicking my gpg key on admin.fp.o ends on keys.fp.o . we should get rid of that link Julen Landa Alustiza <jlanda(a)fedoraproject.org>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure July 2019