infrastructure October 2021

infrastructure@lists.fedoraproject.org

16 participants
26 discussions

About JS framework
by Pierre-Yves Chibon 14 Sep '22

14 Sep '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

15 19

Infrastructure and release engineering Documentation
by Mark O'Brien 09 Nov '21

09 Nov '21

Hi All, As some of you may be aware there has been chatter around having the documentation for infrastructure and release engineering centralised in one place. A possible solution to this is to move all of these under a new section on docs.fedoraproject.org called something like Infrastructure and Release Engineering (very original I know). We could then go about moving docs which are suitable from currrent locations to the new central point. Each doc should be updated before moving and the old document should be updated to only contain a link to the new doc to avoid a case of multiple versions of a doc (https://xkcd.com/927/ ). The following links contain the bulk of the documentation: https://fedora-infra-docs.readthedocs.io/en/latest/ https://docs.pagure.org/releng/ https://fedoraproject.org/wiki/Infrastructure These would be the suggested first steps with other docs to follow if/when these are completed. As I say this is just a possible solution so as always all feedback and suggestions are encouraged and welcomed. Thanks, Mark

4 8

aws instance provisioning
by Kevin Fenzi 04 Nov '21

04 Nov '21

Greetings. Mark and I have talked in the past about moving some of the aws provisioning we are doing into ansible, and it came up again this morning in #fedora-admin. We have many (but not all) maintainer-test instances, a bunch of proxies and all the copr machines in aws that are currently in ansible. I think we could do something similar to tasks/virt_instance_create.yml (which we have for using virt-install and installing libvirt vms). We do have a tasks/aws_cloud.yml that copr instances use, but it doesn't provision the instance, just sets up things for ansible. Copr folks: if we expand aws_cloud.yml to provision also (if the host is not up/reachable on it's dns name/ip) would you be ok using that as well? Or should we seperate out copr from maintainer-test/proxies? Also, there's the question of auth. batcave01 will need creds for those things. Sadly, I think this means making a user and getting a token, but if there's some better way to handle auth there that might be nice. Finally, we are using ansible-2.9.x currently. We would need any solution to be able to work with that and newer ansible, which we likely will be switching to before long. Any other ideas or thoughts around this? Thanks, kevin

4 5

Meeting Agenda Item: Introduction nedia (Aiden)
by Aiden Langley 28 Oct '21

28 Oct '21

Kia ora koutou! This is the 3rd intro that I'm writing, so there is a little copy pasta here... I'm nedia, I live in NZ & I've been knocking about the matrix/irc for a few days now. I'll be 28 on the 30th, I've been using Fedora for about a year, Linux for about 3-4 all up, & have been doing software dev for about 5-6 years. Most of my software experience is in the web space - lots of SQL, PHP & JavaScript (no vanilla JS really, just the front end frameworks), but only a little bit of Python, although my short time w/ Django was pretty good. I am keen on giving Go some love, and have dabbled in some Rust too. I'm a tinkerer and often find myself trying to automate workflows. Most recently I've been building Todoist into the terminal so I'm greeted with my latest todo items when I open my terminal up. I wrote a CLI that interfaced w/ AWS to spawn EC2 instances at a previous job. I went to the devel team firstly since it made the most sense, but infrastructure might be a better fit, I'm not sure, so I'm saying hello anyway. :) I've penciled myself in to updating `pop-shell` for F35, and afterwards I might look to get involved in something more in the infrastructure space. I'm managing IT for a not for profit in the far north of New Zealand, it's voluntary, but some of it is on contract to pay the bills. I also teach general computing once a week to some older folk. Nothing hardcore. I have done IT & network support for some local schools too. Unfortunately I don't have much hardware on hand for testing, but that could change in the future as I could have access to some old hardware if the company I work with get some newer kit. It'll be hard for me to make the weekly meetings since they're on at 3-4am my time, but I'll have a go. I might be able to comfortably settle into a semi-night owl routine. Cheers, Aiden

1 0

CPE Weekly Update – Week of October 25th – 29th
by Akashdeep Dhar 28 Oct '21

28 Oct '21

Hello, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/). If you wish to read in a well-formatted blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-october-2… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Updates ------- ### Fedora Infra * Freeze breaks: added regions to aws fedimg uploads and fixed a caching issue with upgrade json * Rebooted: proxy34 and bvmhost-x86-07 * Tried to fix move of wiki talk pages, ended up creating PR to disable all talk pages. * At 66 tickets, but many should be closable after freeze ### CentOS Infra including CentOS CI * Kicked some migration for tenants on legacy cluster (nfs-ganesha) * Rebased cico-workspace container to 8-stream (staging) (Ref https://quay.io/repository/centosci/cico-workspace/build/b197513b-0105-4774… ) * Pushed some new ciphers in prod through ansible role to get A+ cert on Qualys (Ref https://www.ssllabs.com/ssltest/analyze.html?d=git.centos.org&hideResults=on ) * Mirrormanager tuning with Adrian for 9-stream inside CI infra (Ref https://admin.fedoraproject.org/mirrormanager/host/2751) * Added/announced aarch64 as covered architecture for CI infra tenants ### Release Engineering * F35 RC-1.2 is out and can be found at https://dl.fedoraproject.org/pub/alt/stage/35_RC-1.2/ * Business as usual ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Basic Stream/RHEL Buildroot reporting is in place, thanks James! * Open discussion on pruning older packages from what we publish to the mirrors * Open discussion on the impact of consolidating the Stream 8 and Stream 9 workflows for maintainers * Business as usual ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Set up a boilerplate with a skeleton application * Set up the CI in the repository with tests and coverage * Created a CLI for configuring parameters * Discussed workflows and methods for implementing models ## FCOS OpenShift migration Goal of this Initiative ----------------------- Move current Fedora CoreOS pipeline from the centos-ci OCP4 cluster to the newly deployed fedora infra OCP4 cluster. Updates ------- * Obtaining access on the cluster * Creating playbook to create OpenShift resources * Got the cluster updated and ready Thanks and regards, Akashdeep Dhar t0xic0der(a)fedoraproject.org akashdeep(a)redhat.com

1 0

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 25 Oct '21

25 Oct '21

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2021-10-26 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

https://docs.pagure.org/infra-docs/ Sphinx is not most up to date
by Reon Beon 24 Oct '21

24 Oct '21

http://sphinx-doc.org/ Sphinx 3.5.4 at https://docs.pagure.org/infra-docs/ 4.2.0 is the latest. See: https://github.com/sphinx-doc/sphinx/releases/tag/v4.2.0

4 7

CPE Weekly Update – Week of October 18th – 22nd
by Michal Konecny 21 Oct '21

21 Oct '21

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/). If you wish to read in form of blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-october-1… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ ### Fedora Infra * Fixed fasjson in stg (was stuck on a image pull) * Added 200GB to ostree netapp volume * Retired iddev and simple-koji-ci cloud instances * Declared the sssd bug fixed (hadn’t happened in 2 weeks) ### CentOS Infra including CentOS CI * Stream 9 available in CI (x86_64 only though) * Newer python-cicolient * Duffy2 hotfix for paramiko issue with el9 * Exploring options for secureboot for SIGs * Deploying Duffy Dev Lab for initiative * Business as usual (new tags created for SIGs, ….) ### Release Engineering * Updated critpath packages for all releases * F35 final RC request landed * Compose is finished with incomplete state armhfp container aarch64 KDE and failed ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Work continues on Content Resolver buildroot support * We're now running repoclosure before we sync to the mirrors ## Datanommer/Datagrepper V.2 Goal of this Initiative ----------------------- The datanommer and datagrepper stacks are currently relying on fedmsg which we want to deprecate. These two applications need to be ported off fedmsg to fedora-messaging. As these applications are 'old-timers' in the fedora infrastructure, we would also like to look at optimizing the database or potentially redesigning it to better suit the current infrastructure needs. For a phase two, we would like to focus on a DB overhaul. Updates ------- * Import crashed because of a database schema design decision that didn’t account for the size of our messages, it's fixed and the import started again because the schema update on existing data would take a lot of time too. ETA is back to 70 days. ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Took stock and decided to rewrite * Cleaned up branches * Boilerplate work Kindest regards, CPE Team

1 0

openshift 3 to openshift 4 migration
by Kevin Fenzi 20 Oct '21

20 Oct '21

Greetings everyone. So, we now have openshift 4 clusters for both prod and stg and I think (although correct me if I am wrong) they are all checked out and ready to handle apps. :) To move an app over: * stop all the apps pods in openshift3 cluster * Define any NFS storage that the app has in the ocp4 cluster. * change the playbook to use 'os-control01' and 'os-control01.stg' for copying the templates for the app to /etc/openshift_apps/ and running oc create to create the apps namespace, objects, etc. * change the proxies reverseproxy to use the ocp4 backend instead of os3 * add some metrics? * Profit Apps that don't have any NFS storage and don't have a DB could probibly be deployed in the new cluster and left running in the old one, then switched in proxies, but that won't work with things that have DB/NFS or other persistent state where we don't want the app running in multiple places at the same time. We need to determine how we want to migrate things however. Do we want a flag day where we schedule a big outage and try and move everything? Do we want a flag day for stg, but schedule outages for prod? Or do we want to schedule outages for prod, but do all the stg ones as our time permits? Are there any folks in sysadmin-openshift wanting to drive this? (Although we could likely use help from anyone to test apps as they move, help with announcements, etc). Do note that we have a lot more apps in stg (thing that haven't yet been deployed in prod). kevin

3 2

Phishing emails from fedora project
by Mark O'Brien 20 Oct '21

20 Oct '21

Hi All, There have been some reports of phishing emails claiming to be from the fedora project. An example of which can be seen here: https://pagure.io/fedora-infrastructure/issue/10286 As always please be careful and follow basic guidelines when reading your emails. - Do not click on links or attachments from senders that you do not recognize. Be especially wary of .zip or other compressed or executable file types. - Do not provide sensitive personal information (like passwords) over email. - Watch for email senders that use suspicious or misleading domain names. - If you can’t tell if an email is legitimate or not, please contact admin(a)fedoraproject.org If in any doubt please reach out to us to confirm before sharing any information or clicking any links you are unsure of in an email. Mark

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure October 2021