infrastructure July 2022

infrastructure@lists.fedoraproject.org

18 participants
17 discussions

About JS framework
by Pierre-Yves Chibon 14 Sep '22

14 Sep '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

15 19

planet
by Kevin Fenzi 24 Aug '22

24 Aug '22

Hey folks. I thought I would open a discussion about fedoraplanet and possibly some plans for it. Right now: fedoraplanet.org runs on people02.fedoraproject.org (aka fedorapeople). To add a blog/rss feed you have to login there and edit your .planet file, then scripting pulls all those .planet files and tries to fetch all the feeds and then serves them up at http://fedoraplanet.org. It uses a app called 'venus' to do this. venus is written in very old python2 and very very dead upstream. We run into the following problems with it: * Sometimes it gets stuck and just stops processing until it's killed. * It's serving on a http site, which causes people to ask us to make it https, but that would just change the errors because many feeds it pulls are still http since they were added back before letsencrypt existed. * We have a handy 'website' field in our new account system, but aren't using it at all. * The .planet parsing is poor, any number of things can cause it to break. We have two open tickets on it: - https://pagure.io/fedora-infrastructure/issue/10383 (upgrade to pluto, a ruby based, but maintained thing) - https://pagure.io/fedora-infrastructure/issue/10490 ( planet not served via ssl) Which I am just going to close now. So, I can think of a number of options and would love everyone who has thoughts on it to chime in: 1. Do nothing. Venus "works" and .planet files are cool and retro. 2. Switch to pluto and use account system 'website' fields of contributors. We could likely shove it in openshift and serve it directly from there to avoid fedorapeople entirely. (This would likely break anyone who has multiple feeds in there) 3. Switch to something better/bigger. I would think (although I don't know) that there might be something that would not only aggregate rss feeds for contributors, but perhaps mastodon/twitter/whatever also. 4. Planets are old and tired, just drop the entire thing. People can maintain their own rss lists. 5. Planets are old and tired, just drop the entire thing. But also, get our social media people to maintain contributor / interesting lists. ie, the fedoraproject twitter account could maintain a list of 'fedora contributors' and 'fedora packagers' or whatever. 6. Switch to pluto as in 2, but also setup some curators. Have a 'firehose' of all feeds, but the main fedora planet would be just curated things that are known to be related to fedora and not off topic or unrelated. 6. Get someones (not it!) to take in all the twitter/facebook/mastodon/blog posts/rss feeds and post some kind of curated round up every week or something. 7. Your brilliant idea here! So, thoughts? this is not at all urgent, but we should end up doing something with it sometime. :) kevin

10 21

nest infra&releng mini barcamp
by Kevin Fenzi 01 Aug '22

01 Aug '22

I put in for a 1/2 day mini hackfest for infra and releng at nest. See: https://pagure.io/flock/issue/393 My thought was to collect ideas for cool things we could work on or do, vote on them before the event and do them in most popular order. (bar camp style). Please do chime in here or in the flock issue if you have ideas or would rather do something else. :) kevin

3 4

Where should mirroring docs live?
by Ben Cotton 29 Jul '22

29 Jul '22

Hi folks, adamw noticed that we have two pages that claim to give information about setting up a public mirror: a wiki page[1] and a Quick Docs page[2] which is actually blank. This is one more than we need, so the question is how to fix it. We can either leave the wiki page as-is and remove the Quick Docs page. Or we can port the wiki to Docs and set up a redirect. If we do that, I think it's probably better suited for the infra docs than the Quick Docs. But I leave this up to the infra team because it's your content. I'm happy to help make whichever answer you pick happen. [1] https://fedoraproject.org/wiki/Infrastructure/Mirroring [2] https://docs.fedoraproject.org/en-US/quick-docs/mirroring/ Thanks, BC -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

2 1

CPE Weekly Update - Week 30 2022
by Michal Konecny 29 Jul '22

29 Jul '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/). Week: 25th July - 29th July 2022 If you wish to read this in form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-30-2022/ # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-07-27.pdf Link to docs: https://docs.fedoraproject.org/en-US/infra/ Update ------ ### Fedora Infra * New Fedora Media Writer release (mostly translation fixes) 5.0.2 * Rabbitmq cluster instability last week due to both ocp3 and ocp4 clusters running monitor-gating and them obsoleting each other over and over. ;( * Worked around redhat.com SPF issue by accepting email from redhat.com, expanding fedoraproject.org alias and sending it right back out through redhat.com mx to deliver. ### CentOS Infra including CentOS CI * Preparing duffy.ci migration ([now announced for next week](https://lists.centos.org/pipermail/ci-users/2022-July/004582.html)) ### Release Engineering * F37 Mass rebuild finished ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * Imported c8s modules into the new mbs. * CentOS Linux 7.9 ISOs are respun ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * EPEL9 is up to 6986 (+378) packages from 3137 (+54) source packages * Backported fix for CVE-2021-21419 to EPEL8's [python-eventlet](https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-0… * Retired several rawhide branches of EPEL-only packages * Updated KDE (5.24.6) is available in epel9-next and epel8-next testing. * Will not go out to regular epel until RHEL 8.7 and 9.1 * Working on an EPEL survey that will go out in August ## FMN replacement Goal of this initiative ----------------------- FMN (Fedora-Messaging-Notification) is a web application allowing users to create filters on messages sent to (currently) fedmsg and forward these as notifications on to email or IRC. The goal of the initiative is mainly to add fedora-messaging schemas, create a new UI for a better user experience and create a new service to triage incoming messages to reduce the current message delivery lag problem. Community will profit from speedier notifications based on own preferences (IRC, Matrix, Email), unified fedora project to one message service and human-readable results in Datagrepper. Also, CPE tech debt will be significantly reduced by dropping the maintenance of fedmsg altogether. Updates ------- * Basic repo structure in place (Python + JS side), more boilerplate work ongoing * Fedora-messaging schemas checkup commenced Kindest regards, CPE Team

1 0

erolkeskin.dev@gmail.com
by Erol Keskin 23 Jul '22

23 Jul '22

1 1

CPE Weekly Update – Week 29 2022
by Lenka Segura 22 Jul '22

22 Jul '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/). Week: 18th - 22nd of July 2022 If you wish to read this in form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update--week-29-2022/ # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-07-20.pdf Link to docs: https://docs.fedoraproject.org/en-US/infra/ Update ------ ### Fedora Infra * New version of FMN released - removal of python-fedora requirement and all FAS2 calls * (almost) all builders reinstalled with f36 and bvmhosts moved to RHEL9 or Fedora36. There’s a few arm buildhw’s to be done after the mass rebuild. * FMN (py2) caught up finally, still needs fas2. Looking to switch to FMN (py3) soon. * Some recent rabbitmq / fedora-messaging cluster problems. Gave it more memory and hopefully it’s happy again. * Finally got *.fedoraproject.org ssl cert working with “FUTURE” encryption. * 5 new aarch64 machines and ½ new shelf of netapp storage arriving soon. ### CentOS Infra including CentOS CI * Mirror requests * Duffy migration tasks * Business as usual ### Release Engineering * Mass rebuild starts later this morning (Wed 20th) unless any last minute delays. ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * A Stream 8 compose has been pushed to correct some repoclosure issues with the kernel * Lots of PTO ## Package Automation (Packit Service) Goal of this initiative ----------------------- Automate RPM packaging of infra apps/packages. Updates ------- * Datagrepper is with packit * Mirrormanager2 in progress ## Flask-oidc: oauth2client replacement Goal of this initiative ----------------------- Flask-oidc is a library used across the Fedora infrastructure and is the client for ipsilon for its authentication. flask-oidc uses oauth2client. This library is now deprecated and no longer maintained. This will need to be replaced with authlib. Updates: -------- * Pull request is in https://github.com/fedora-infra/flask-oidc/pull/8 * Worked through the comments/suggestions from Aurelien * NEED EXPERIENCED TESTERS (to help write tests, need to update existing tests, scenario is complex, mocking out Ipsilon responses etc feel free to join) ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * EPEL9 is up to 6608 (+266) packages from 3083 (+127) source packages * Sponsoring of new packagers with interest in EPEL * Add python-tenacity to resolve ansible-lint fail to install bug * Notable package additions: * keepassxc * asciinema ## FMN replacement Goal of this initiative ----------------------- FMN (Fedora-Messaging-Notification) is a web application allowing users to create filters on messages sent to (currently) fedmsg and forward these as notifications on to email or IRC. The goal of the initiative is mainly to add fedora-messaging schemas, create a new UI for a better user experience and create a new service to triage incoming messages to reduce the current message delivery lag problem. Community will profit from speedier notifications based on own preferences (IRC, Matrix, Email), unified fedora project to one message service and human-readable results in Datagrepper. Also, CPE tech debt will be significantly reduced by dropping the maintenance of fedmsg altogether. Updates ------- * Early days - pre-planning talks done today * Scoping/research/fact finding from the arc investigation https://fedora-arc.readthedocs.io/en/latest/fmn/april2022/index.html * Possibility of using Vue instead of React on frontend, typescript vs javascript also a possibility, FastAPI for the backend API Kindest regards, CPE Team

1 0

Email replies to src.fpo notifications
by Frank R Dana Jr. 18 Jul '22

18 Jul '22

Recently, after receiving an email notification regarding a pull request at src.fedoraproject.org, I attempted[1] to update the discussion via email reply to the notification email. That attempt was not successful. Moreover, I was only made AWARE that the attempt was unsuccessful 24 hours later, when Google's MTA responded with an undeliverable notification saying that it couldn't contact the recipient email, pagure(a)pkgs.fedoraproject.org. If email reply to src.fpo notifications is supposed to be working, it's not. If it's INTENDED not to be working, then perhaps the reply address for notifications could be set to something that bounces immediately (donotreply(a)fedoraproject.org or similar, perhaps?), rather than an address at a non-responsive hostname that's subject to lengthy timeouts before the status of the attempted reply is updated? [1]: https://discussion.fedoraproject.org/t/should-pagure-actually-src-fedorapro…

3 3

Software Bill of Materials for Fedora
by Jason Shepherd 15 Jul '22

15 Jul '22

Hello Fedora Infrastructure team, Red Hat Product Security are building an application called Component Registry to meet the requirements set out in the recent Executive Order 14028 [1], "Improving the Nation's Cybersecurity". The executive order requires that software producers and suppliers should take steps to report and validate a listing of all components included in or used by their software products, aka a Software Bill of Materials. We'd like to build our application in the open by providing the source code to the opensource community. Since all the Red Hat build infrastructure is internal to Red Hat, we'd like also provide this service to Fedora so that our open source project can have a life outside of Red Hat's corporate firewall. I suspect we are close to being able to provide an example of the Software Bill of Materials (SBOM) for Fedora, since it is built in a very similar way to Red Hat Enterprise Linux. The reason for reaching out is to find out if you are interested in hosting an SBOM for Fedora or not. We could build it inside the Red Hat firewall, and provide a static file for each target release of Fedora, undated periodically. Alternatively we could run the application somewhere on your infrastructure in order to make the data available via an API on demand. In which case we'd probably need to help to maintain that infrastructure. Let me know your thoughts. I didn't provide a link to the code repository as it's currently private to Red Hat associates. But we expect to opensource the project in the coming weeks. At that time we'll be able to provide the source code. Regards, Jason Shepherd Red Hat Product Security [1] https://www.cisa.gov/executive-order-improving-nations-cybersecurity

5 5

Re: Once again, more than 8 days delayed notifications
by Kevin Fenzi 12 Jul '22

12 Jul '22

crossposting to infra list to keep folks there in the loop... On Tue, Jul 12, 2022 at 02:53:24PM +0100, Jonathan Wakely wrote: > On Mon, 11 Jul 2022 at 15:06, Stephen Smoogen wrote: > > That said, IRC is actually one of the fastest ones we can push to. > > Is https://apps.fedoraproject.org/notifications/ really still sending > notifications to freenode though? > > Hasn't everybody moved to libera? yes. It's just incorrect/out of date there. Let me sum up what I know and perhaps I can point people to this post later. :) The current state is bad. We know it's bad. We have known for a long time that it's bad. It's bad for all of the following reasons: * It's running a python2 codebase. Upstream development/PR's have long ago moved to python3, but thats not the version we current have deployed. * It's heavily tied to the old account system. Several of us spent many hours last week trying to untangle it. I think we might be getting close now, but it's really hard to tell. * It's pretty heavily tied to fedmsg (not fedora-messaging). fedmsg still works of course, but it's another layer of confusion. * It's rules/interface lets you do all kinds of cool things, but it's complex and confusing to most everyone that tries to use it. * It's running on a end of life OS version. ;( * In order to try to scale it has a number of layers of things which makes it hard to debug. For example it uses redis, it's own rabbitmq instance with a bunch of queues, multiple workers talking to all those things and multiple backends for irc and email. You might think performance shouldn't be a big deal, but when you have thousands of users, each with their own custom rulesets, that means you have to potentially match a incoming message against every single ruleset of every user and you have to do it fast enough to keep up with the incoming flow of messages and the outgoing flow of notifications. :( Short term, I would like to hope that the python2/current version can catch mostly up. (Its currently 4 days behind). Once it does, I would very much like to try switching to the python3 version. It has a lot of the problems the existing one does, but it should also have some advantages, like running on a supported OS, much easier to release and deploy new versions, etc. Longer term, we are just now starting an iniative to re-write FMN from the ground up. It's going to be a while until it's ready, but I really hope it will be much better for everyone involved. Better/easier interface, much better handling of messages, etc. Hope that helps clarify things... kevin

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure July 2022