infrastructure April 2020

infrastructure@lists.fedoraproject.org

37 participants
54 discussions

Back again to Fedora Infra team
by Amr Zaki 29 Apr '20

29 Apr '20

Hello All, The past time, I was busy due to my work so I need to back again to playing role at team. Please, @kavin could you add me again to apprentice admin group? Best Regards, Amr Zaki

2 2

Meeting Agenda for 2020-04-30 1500 UTC
by Stephen John Smoogen 29 Apr '20

29 Apr '20

## Preamble The infrastructure team will be having its weekly meeting tomorrow, 2020-04-30 at 15:00 UTC in #fedora-meeting-1 on the freenode network. We have a document at https://board.net/p/fedora-infra Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. ## Introduction We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. ### Meeting start stuff ``` #startmeeting Infrastructure (2020-04-30) #meetingname infrastructure #chair nirik pingou smooge cverna mizdebsk mkonecny abompard #info Agenda is at: https://board.net/p/fedora-infra #info About our team: https://docs.fedoraproject.org/en-US/cpe/ #topic aloha ``` ### Let new people say hello ``` #topic New folks introductions #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted ``` ### Determine who the next chair is #topic Next chair #info magic eight ball says: #info 2020-04-30 - smooge #info 2020-05-07 - cverna #info 2020-05-14 - siddharthvipul #info 2020-05-21 - ??? ### Status / Information / Trivia / Announcements (We put things here we want others on the team to know, but don't need to discuss) (Please use ```#info (the thing - your name)``` ``` #topic announcements and information #info CPE Sustaining EU-hours team has standups on Tuesday and Thursday at 1400 UTC in #fedora-admin - please join #info CPE Sustaining NA-hours team has a Monday through Friday 30 minute meeting going through tickets at 1800 UTC in #fedora-admin #info Fedora Infrastructure will be moving in 2020-06 from its Phoenix Az datacenter to one near Herndon Va. A lot of planning will be involved on this. Please watch out for announcements on changes. #info Fedora Communishift move has started but will take longer than expected. Current estimate for bringing back into production is TBD #info F32 final freeze is in effect! #info Taskotron will EOL in 2020-05 ``` ### Things we should discuss We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use ```#topic your discussion topic - your username)``` ``` #topic Oncall #info https://fedoraproject.org/wiki/Infrastructure/Oncall #info siddharthvipul is oncall 2020-04-16 -> 2020-04-23 #info cverna is oncall 2020-04-23-> 2020-04-30 #info siddharthvipul is oncall 2020-04-30 -> 2020-05-07 #info ??? is oncall 2020-05-07 -> 2020-05-14 #info ??? is oncall 2020-05-14 -> 2020-05-21 ## .oncalltakeeu .oncalltakeus #info Summary of last week: (from current oncall ) #info #topic Monitoring discussion [nirik] #info https://nagios.fedoraproject.org/nagios #info Go over existing out items and fix ### Put all topics for discussion under here Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. ### Learn about some application or setup in infrastructure (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) ``` #info ``` ### Meeting end stuff ``` #topic Open Floor #endmeeting ``` -- Stephen J Smoogen.

1 0

Fedora 32 Final Freeze now in effect!
by Kevin Fenzi 29 Apr '20

29 Apr '20

Greetings. we are now in the infrastructure freeze leading up to the Fedora 32 Final release. This is a final release freeze. We do this to ensure that our infrastructure is stable and ready to release Fedora 32 when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git ansible/scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2020-04-21 (or later if release slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, kevin

1 1

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 29 Apr '20

29 Apr '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-04-30 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

Meeting Agenda Item: Introduction Mark O'Brien
by Mark O'Brien 28 Apr '20

28 Apr '20

Hello, May name is mark, I have just started as a software engineer at redhat, I have worked as a DevOps for 3 years previously and am familiar with tooling such as ansible and scripting languages such as python and bash. My irc handle: mjobrien

4 3

FBR: Pre GA tasks
by Mohan Boddu 27 Apr '20

27 Apr '20

Hi, The following changes are needed for pre ga tasks, please review them. Thanks.

4 4

FBR: reboot ppc9-02
by Kevin Fenzi 27 Apr '20

27 Apr '20

ppc9-02 is in a odd state. Guests are running ok, but the host itself is hanging any ansible runs against it. :( Usually this is a sign of some stuck hardware. I've disabled the builders on it and once builds finish on them I would like to update and reboot ppc9-02. Due to existing builds this may be later this weekend... +1s? kevin

3 2

FBR: Fix DNS to match closer zones
by Stephen John Smoogen 27 Apr '20

27 Apr '20

Currently several countries are getting routed to proxies which are timing out for them. This should 'fix' some though I expect we will need to break this out further per continental region. diff --git a/roles/dns/files/named.conf b/roles/dns/files/named.conf index c8b95b9..135ec69 100644 --- a/roles/dns/files/named.conf +++ b/roles/dns/files/named.conf @@ -610,7 +610,7 @@ view "RDU2" { // The zones view "NA" { - match-clients { US; CA; MX; }; + match-clients { US; CA; MX; BM; GL; }; recursion no; // no rate-limit on internal requests rate-limit { @@ -636,9 +636,10 @@ view "NA" { }; -// This is not "EU" countries, I just wanted a short way to represent Europe. +// This should have been EME and during the next freeze break should be made as such. view "EU" { - match-clients { AT; BE; BG; CY; CZ; DE; DK; EE; ES; FI; FR; GR; HU; IT; LT; LU; LV; MT; NL; PL; PT; RO; RU; SE; UA; GB; IE; IS; NO; }; + match-clients { AD; AL; AT; AX; BA; BE; BG; CH; CZ; DE; DK; EE; ES; FI; FO; FR; GB; GG; GI; GR; HR; HU; IE; IM; IS; IT; JE; LI; LT; LU; LV; MC; ME; MK; MT; NL; NO; PL; PT; RO; RS; RU; SE; SI; SJ; SK; SM; UA; VA; AE; AM; AZ; BH; CY; GE; IL; IQ; JO; KW; LB; OM; QA; SA; TR; YE; }; + recursion no; zone "fedoraproject.org" { type master; @@ -660,7 +661,7 @@ view "EU" { }; view "APAC" { - match-clients { AE; AF; AM; AU; AZ; BD; BH; BN; BT; CC; CN; CX; CY; GE; HK; ID; IL; IN; IO; IQ; IR; JO; JP; KG; KH; KP; KR; KW; KZ; LA; LB; LK; MM; MN; MO; MV; MY; NP; NZ; OM; PH; PK; PS; QA; RU; SA; SG; SY; TH; TJ; TL; TM; TR; TW; UZ; VN; YE; }; + match-clients { AF; BD; BN; BT; CN; HK; ID; IN; JP; KG; KH; KZ; LA; LK; MM; MN; MO; MV; MY; NP; PH; PK; SG; TH; TJ; TL; TM; UZ; VN; AS; AU; CC; CK; CX; FJ; FM; GU; HM; KI; MH; MP; NC; NF; NR; NU; NZ; PF; PG; PN; PW; SB; TK; TO; TV; UM; VU; WF; WS; } recursion no; zone "fedoraproject.org" { type master; -- Stephen J Smoogen.

3 3

CPE Weekly: 2020-04-26
by Aoife Moloney 26 Apr '20

26 Apr '20

# CPE Weekly: 2020-04-26 --- title: CPE Weekly status email tags: CPE Weekly, email --- Background: The Community Platform Engineering group is the Red Hat team combining IT and release engineering from Fedora and CentOS.Check out our teams info here https://docs.fedoraproject.org/en-US/cpe/ ## GitForge Updates * We will be tracking our progress here https://fedoraproject.org/wiki/Git_forge_update * Please be aware this does not contain any new information, as we don't have any to share yet. * However once we have more information for you, we will be posting it to the above link, and with the URL in the weekly emails for quick access. * We are currently doing a technical deep-dive with our own team on what we need from GitLab * We are also still engaging with Fedora Council who are tracking the Fedora Communities needs here https://pagure.io/Fedora-Council/tickets/issue/292 * And we are looking at ways to engage closer with the CentOS community too to share information as and when we have it * Public Q&A sessions are still being discussed to allow for open conversations in public forums too as we move through this journey, so thank you for your patience with us. ## Fedora Updates * F32 release is a Go! * Estimated release date is April 28th - well done to everyone involved! ### Data Centre Move * Due to Covid-19 restrictions at each datacentre, we have unfortunately experienced a delay during the initial move. * The need to push out our dates by 2-3 weeks was made necessary on Thursday 23rd April, following meetings with Red Hat IT on network connectivity status and estimate connection times for our team to access the new datacentre. * We will need to adjust our outage schedule and move schedule to properly reflect the new dates of services being turned off and reduced, and the full amended schedule will be published to hackmd, with emails sent to the devel-announce & infra lists next week (week ending 1st May) * We will also include links to the updated move schedule and service impact schedule in next weeks CPE Weekly * Please be aware that CommuniShift is now down indefinitely until earliest May 8th. ### AAA Replacement * The team have begun phase two of the project and you can view their board for more technical information on their work here https://github.com/orgs/fedora-infra/projects/6 * They are doing amazing work :) ### CI/CD * Monitor-gating is now running in production and has already caught a couple of issues with bodhi (both in stg and in prod)! * Rpmautospec * In review as a Fedora package: https://bugzilla.redhat.com/show_bug.cgi?id=1816124 * Work progressing on Koji tagging plugin (post-build), full use case support for bumping releases * Plan is still to get it deployed in staging ### Sustaining Team * Mbbox Upgrade * Some progress on CRD for koji-builder and koji-hub components * Bodhi 5.2.2 released * Some issues with celery tasks, however rawhide monitoring super useful. * Compose tracker enhancement * Focusing on tagging issues, and having the ability to ping maintainers Odcs-backend-releng01 provision to enable testing in Fedora Minimal Compose * Infra * The daily standup the team has has helped a lot with managing infra tickets - they are down to 99 tickets! * Mass update of stg and prod * Please note you may experience some Kojira slowness * New review-stats application deployed - (https://fedoraproject.org/PackageReviewStatus/) ## CentOS Updates ### CentOS * ppc64le and aarch64, 8 and 8-stream nodes now available in cico for tenants to checkout. -- Email sent to ci-user list * New signing for SIGs (through https://cbs.centos.org) live on 25th April 2020 ### CentOS Stream * Summit Videos took our attention this week * We have a booth, so don't forget to register your attendance! https://www.redhat.com/en/summit * Qt5.12 pushed in response to an internal request * NetworkManager re-imported As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great week ahead! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ -- Aoife Moloney Product Owner Community Platform Engineering Team Red Hat EMEA Communications House Cork Road Waterford

1 0

FBR: Reboot of ppc09-02
by Stephen John Smoogen 24 Apr '20

24 Apr '20

Server has some problems with services and guests. We would like to reboot the system to clear some issues being seen. -- Stephen J Smoogen.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure April 2020