infrastructure November 2021

infrastructure@lists.fedoraproject.org

24 participants
35 discussions

About JS framework
by Pierre-Yves Chibon 14 Sep '22

14 Sep '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

15 19

Sysadmin - main group
by Mark O'Brien 29 Nov '21

29 Nov '21

I'm happy to announce that We have added a new member to the sysadmin-main group: Saffronique - David Kirwan This is the core group of trusted folks that have high level access to almost everything in fedora infrastructure. David has been working on Fedora infra for about a year now and has been involved in some fairly big pieces of work including adding an aarch64 cluster to osbs as well as bringing up the new ocp4 cluster in Fedora. He has proved trustworthiness, and ability and I'd like to be the first to wish him congratulations Mark

6 5

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 29 Nov '21

29 Nov '21

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2021-11-30 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

CPE Weekly Update – Week of November 22nd – 26th
by Lenka Segura 26 Nov '21

26 Nov '21

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/). If you wish to read this in form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-november-… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ * Discourse2fedmsg mini initiative nearing completion ### Fedora Infra * Some issues post mainframe migration on s390x builders. Investigation ongoing * Issues with some users password expiring, abompard fixed with a script * Business as usual ### CentOS Infra including CentOS CI * Announced switch to 8-stream based container for cico-workspace (ci infra) (https://lists.centos.org/pipermail/ci-users/2021-November/002187.html) * Work completed to allow SIG to switch buildroot from centos 8 to RHEL8 ( https://pagure.io/centos-infra/issue/400) * New SIGs tags: * Automotive experimental (Stream 9) * Storage TDX-devel (Stream 8) * Fixed issue with debuginfod pkgs filtered out ( https://pagure.io/centos-infra/issue/532) after process change for Stream 9/SIGs * Tuned mirror.stream.centos.org pool due to hard-disk space constraint on sponsored/donated nodes (https://pagure.io/centos-infra/issue/529) ### Release Engineering * F33 distgit branches had the wrong EOL set to 16-11-2021 but the real f33 EOL is 30-11-2021 * FCOS prunned OStree compose repo, saved about 2TB of disk space * business as usual ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Investigating AWS cleanups * Finishing Content Resolver buildroot changes * Wrote ELN docs about package differences (in FAQ) ( https://docs.fedoraproject.org/en-US/eln/faq/) and adding packages to ELN Extras (https://docs.fedoraproject.org/en-US/eln/extras/) * Many PTOs this week ## Datanommer/Datagrepper V.2 Goal of this Initiative ----------------------- The datanommer and datagrepper stacks are currently relying on fedmsg which we want to deprecate. These two applications need to be ported off fedmsg to fedora-messaging. As these applications are 'old-timers' in the fedora infrastructure, we would also like to look at optimizing the database or potentially redesigning it to better suit the current infrastructure needs. For a phase two, we would like to focus on a DB overhaul. Updates ------- * The DB had been rebooted a few times and the script restarted. 40 days to go. ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Duffy has sub-commands * Serving the web app * Initial setup of DB schema * Interactive shell for debugging and development * Initialize DB connection and related code when starting the app * Ongoing: API endpoints * Wednesday’s review + planning call: Conversation with Fedora QA about how Duffy can serve them * CI: Let dependabot check for transient updates (and apply them) ## FCOS OpenShift migration Goal of this Initiative ----------------------- Move current Fedora CoreOS pipeline from the centos-ci OCP4 cluster to the newly deployed fedora infra OCP4 cluster. Updates ------- * Work complete, handed over to the Fedora CoreOS team. ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * EPEL 9 Next is ready, but not announced yet * EPEL Steering Committee is evaluating some alternative plans that may let us launch EPEL 9 and EPEL 9 Next together * Hoping to announce on the same day as the CentOS Stream 9 launch promotion (1-2 weeks) Kindest regards, CPE Team

1 0

sourcegraph and src.fedoraproject.org
by Matthew Miller 25 Nov '21

25 Nov '21

Sourcegraph is planning to, very shortly, start cloning all non fork repos from src.fedoraproject.org as well as asking the Pagure API for the metadata of those repos. Their user agent for API requests is `Sourcegraph-Bot` and for git protocol stuff `git/Sourcegraph-Bot`. If there's a problem, let me know and I'll let them know. :) -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

2 2

Sourcegraph and exploded sources
by Matthew Miller 23 Nov '21

23 Nov '21

I mentioned on devel list that Sourcegraph is going to be indexing source.fedoraproject.org. So I guess, first — heads-up! if you see their traffic, it's not malicious. (I asked them to provide us with the user agent they use, and I'll pass that on.) But then, I'd actually like to go a step further, and have them index _the actual source for every build_. They're open to doing that, but what they need is a git repo. So...... how hard / ridiculous / bad would it be to have a step in the build process in koji which, between the %prep and %build phases, push the unpacked-and-prepped source tree to Gitlab, under, say, https://gitlab.com/fedora/exploded-build-sources/package-name/? I'm imaginging this would work something like this: 1. If remote repo doesn't exist, create it with the gitlab api 2. Do a shallow, no-checkout clone of that remote repo, using --git-dir and --work-tree so that the .git directory isn't created inside the working directory 3. copy the unpacked source tree from %prep into the work-tree dir (are we building on btrfs? cp -l if not, to save io?) 4. git add --all 5. git commit -m "Build ID: ${buildID} https://koji.fedoraproject.org/koji/buildinfo?buildID=${buildID}" 6. git push With some details to be worked out. :) Like, repo tags as branches, maybe? And, do this on all arches, or just one of them? Or maybe run up through %prep as part of the src build rather than any of the binary builds? Make the commit as the Fedora username of the person who did the build? But those kinds of implementation details aside -- does this seem like something we might be able to do? Or, any ideas for an alternate approach? I mean, obviously, source-git would be one such alternative, but getting that for _everything_ would require a big rework of how everyone works. (I think we should get to that eventually, but that's a long way off.) -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

4 7

[Fedocal] Reminder meeting : Websites & Apps Team Meeting
by t0xic0der＠fedoraproject.org 22 Nov '21

22 Nov '21

Dear all, You are kindly invited to the meeting: Websites & Apps Team Meeting on 2021-11-23 from 15:00:00 to 16:00:00 UTC At https://meet.jit.si/fedora-websites-apps-meeting The meeting will be about: Weekly team meeting for the Websites & Apps Team. This is part of the [Websites & Apps Community Revamp Objective](https://fedoraproject.org/wiki/Objectives/Websites_%26_Apps_Comm…. See [past discussion]( https://discussion.fedoraproject.org/t/planning-meeting-for-websites-apps-t…) to learn more about the Websites & Apps Team and how it came together. More information available at: [hackmd.io/Mxm2We3yTqKybLsdohadOA](https://hackmd.io/Mxm2We3yTqKybLsdohadOA?view) Source: https://calendar.fedoraproject.org//meeting/9990/

1 0

Re: Sourcegraph and exploded sources
by Frank Ch. Eigler 22 Nov '21

22 Nov '21

Hi - > [...] So...... how hard / ridiculous / bad would it be to have a step > in the build process in koji which, between the %prep and %build > phases, push the unpacked-and-prepped source tree to Gitlab, under, > say, https://gitlab.com/fedora/exploded-build-sources/package-name/? > [...] They could download our -debugsource RPMs, commit them to a git branch for each n or n-v, and bob's their uncle. - FChE

1 0

CPE Weekly Update - Week of November 15th - 19th
by Patrik Polakovic 22 Nov '21

22 Nov '21

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/). If you wish to read this in the form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-november-… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Updates ------- ### Fedora Infra * Mass updates & reboots carried out * Increased volumes on db01 and torrent02 * Moved s390x from z13 to z15 mainframe successfully * F34 & F35 maintainer test instances now available, F32 retired ### CentOS Infra including CentOS CI * CentOS Linux 8.5.2111 released! * Preparing to sunset its infra needs and switch SIG content for RHEL8 buildroot (opt-in) * Handed over the dedicated Duffy Dev LAB infra for duffy initiative ### Release Engineering * Business as usual ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Thanks to a contribution from the folks in the Automotive SIG we are now publishing Vagrant boxes for CentOS Stream 9 to https://cloud.centos.org/centos/9-stream/x86_64/images/ * Business as usual ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * All things database because it practically blocked everything (ongoing) -- Enable having various sub-commands in the first place to be able to do other things than just run the app -- Add one that sets up an empty database with tables from the model -- Prepare database schema migrations -- Test synchronous and asynchronous DB operations * Implement not-yet-functional (see above) API endpoints for users, physical nodes and virtual nodes (ongoing) * Write an Ansible role to deploy Duffy, set up database and fetch playbooks from their repos (ongoing) ## FCOS OpenShift migration Goal of this Initiative ----------------------- Move current Fedora CoreOS pipeline from the centos-ci OCP4 cluster to the newly deployed fedora infra OCP4 cluster. Updates ------- * Work complete, handed over to the FCOS team ## EPEL Goal of this Initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * Fedora s390x builder migration was successful, unblocking epel9 * Productive meetings with Terry Bowling and Eric Hendricks refining the EPEL “feature” and planning future collaboration Kindest regards, CPE Team _______________________________________________ infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapr… Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

1 0

19 Nov '21

Good idea?

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure November 2021