infrastructure July 2020

infrastructure@lists.fedoraproject.org

28 participants
18 discussions

About JS framework
by Pierre-Yves Chibon 14 Sep '22

14 Sep '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

15 19

Another Rust MirrorManager experiment
by Adrian Reber 03 Nov '20

03 Nov '20

Our MirrorManager setup exports the current state of all mirrors every hour at :30 to a protobuf based file which is then used by the mirrorlist servers to answer the requests from yum and dnf. The Python script requires up to 10GB of memory and takes between 35 and 50 minutes. The script does a lot of SQL queries and also some really big SQL queries joining up to 6 large MirrorManager tables. I have rewritten this Python script in Rust and now it only needs around 1 minute instead of 35 to 50 minutes and only 600MB instead of 10GB. I think the biggest difference is that I am almost not doing any joins in my SQL request. I download all the tables once and then I do a lot of loops over the downloaded tables and this seems to be massively faster. As the mirrorlist-server in Rust has proven to be extremely stable over the last months we have been using it I would also like to replace the mirrorlist protbuf input generation with my new Rust based code. I am planing to try out the new protobuf file in staging in the next days and would then try to get my new protobuf generation program into Fedora. Once it is packaged I would discuss here how and if we want to deploy in Fedora's infrastructure. Having the possibility to generate the mirrorlist input data in about a minute would significantly reduce the load on the database server and enable us to react much faster if broken protobuf data has been synced to the mirrorlist servers on the proxies. Adrian

4 17

What is our technical debt?
by Pierre-Yves Chibon 21 Sep '20

21 Sep '20

Good Morning Everyone, Just like every team we have technical debt in our work. I would like your help to try to define what it is for us. So far, I've come up with the following: - python3 support/migration - fedora-messaging - fedora-messaging schema - documentation - (unit-)tests - OpenID Connect What else would we want in there? Looking forward to your thoughts, Pierre

22 55

Tech debt analysis tool
by James Richardson 08 Aug '20

08 Aug '20

Hi All, Vipul and I are looking into several different tools that will allow us to better analyze our tech debt with any new code that is merged into apps in http://github.com/fedora-infra. Currently, we have looked at the tools below, but we would love any and all input from the team and community on this. SonarCloud LGTM ShiftLeft Our goal is to find an open source tool that is easy to integrate as well as providing useful and timely feedback. So far, SonarCloud has proved to be the one that looks best, but again, we are very open to any and all suggestions, and at this early stage, a good conversation to arrive at the best solution. Regards, James -- James Richardson Engineering Intern He | Him | His Red Hat Waterford <https://www.redhat.com/> Communications House Cork Road, Waterford City jamricha(a)redhat.com <lgriffin(a)redhat.com> M: +353851970521 <+353877545162> IM: jamricha @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://red.ht/sig>

8 9

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 29 Jul '20

29 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-30 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

s390x for EPEL
by Miroslav Suchý 29 Jul '20

29 Jul '20

What is using Koji to build EPEL builds for s390x. We want to enable s390x in Copr. It is without problem for Fedora, but we cannot do that for EPEL, because there is no CentOS for s390x. Do we have entitlements for RHEL I can use? Or we have some local repo? Something completely different? -- Miroslav Suchy, RHCA Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

4 7

packages app
by Kevin Fenzi 28 Jul '20

28 Jul '20

So, I think we have two groups approaching a replacement app for packages? Can we perhaps decide which way we want to go and look at deploying something soon? This has been more important since we didn't move the old broken packages app from the old datacenter and people keep asking about it. We could wait until we have staging back, but I don't think we need to wait for that to decide what we want to do here? Thoughts? kevin

3 8

CPE Weekly: 2020-07-25
by Aoife Moloney 26 Jul '20

26 Jul '20

--- title: CPE Weekly status email tags: CPE Weekly, email --- # CPE Weekly: 2020-07-25 Background: The Community Platform Engineering group is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give. See our wiki page here for more information:https://docs.fedoraproject.org/en-US/cpe/ ## General Project Updates The CPE team are working on the following projects for Quarter 3, which is the months of July, August & September: * Data Centre Move - Final Works * CentOS Stream Phase 3 * Noggin Phase 3 * Packager Workflow Healthcare * Fedora Messaging Schemas Details of the above projects, and of projects currently in progress, done and what projects are in our backlog, can be found on our taiga board per project card: https://tree.taiga.io/project/amoloney1-cpe-team-projects/kanban?epic=null We also have an updated initiative timetable for briefing in new projects to our team & key dates here:https://docs.fedoraproject.org/en-US/cpe/time_tables/ *Note: Initiatives are large pieces of work that require a team of people and weeks/months to complete. Please continue to open tickets in the normal way for bugs, issues, etc. ### CPE Product Owner Office Hours #### #fedora-meeting-1 * Weekly on Thursdays @ 1300 UTC on #fedora-meeting-1 * Next Meeting: 2020-07-30 #### #centos-meeting * Every second Tuesday @ 1500 UTC on #centos-meeting * Next Meeting: 2020-08-04 ### Misc * CFP for Nest with Fedora submissions are closing on Monday 27th July - dont forget to submit your ideas/talks here https://pagure.io/flock * CPE Engagement email was sent a few weeks ago, please read if you have not already and feel free to send me your feedback on or off list before 31st July. A follow up email will be sent with feedback received and actions taken where it makes sense then afterwards. Thanks for your engagement!https://lists.fedoraproject.org/archives/list/devel-announce@lis… ## Fedora Updates ### Data Centre Move * The team have been working incredibly hard over the last few weeks and are now working on reinstalling the staging environment. * We estimate this work to be completed by July 31st, however things happen so we thank you in advance for your patience * The project card on taiga for builder - staging - remaining app bringup dates here https://tree.taiga.io/project/amoloney1-cpe-team-projects/us/42?kanban-stat… * A list of affected services is available here https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view * Details on what this move may mean for you can be found here https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr… * If an application is not working correctly at all, please check this list https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA?view before opening a ticket to make sure its not listed as being moved. If it is being moved, please wait a day or two, then try again. * And finally a reminder to please be patient when opening tickets for service issues in general as we are in the critical point in this move and all of our sys-admins and wider teams are assisting in the successful bringup of the reduced Fedora service and facilitation of the final hardware shipment and move. ### AAA Replacement * The team are focusing work on this ticket and collaborating with the Sustaining team for the next sprint, with the goal to have noggin installed & deployed to staging https://pagure.io/fedora-infrastructure/issue/9152 * The have also been addressing how to curtail spam email and more information on that effort can be found here https://github.com/fedora-infra/noggin/issues/27 * And have added the feature to allow groups with a prefix for groups to import to avoid overlaps between CentOS and Fedora https://github.com/fedora-infra/fas2ipa/issues/5 * Please feel free to check out the team kanban board for more information on the features the team are working on and have already completed here https://github.com/orgs/fedora-infra/projects/6 ### Fedora Messaging Schemas * The team are using a kanban approach for this project as Noggin takes priority until fully complete * The team have already built a list of applications that require messaging schemas, list can be found here https://hackmd.io/@nilsph/H1i8CAbkP/edit * They also have completed a readme which contains documentation on messaging schemas, a cookie-cutter template to create the schema and a definition of Done for writing a schemas https://github.com/fedora-infra/fedora-messaging-schemas-issues * The board they are working from can be viewed here https://github.com/orgs/fedora-infra/projects/7 ### Packager Workflow Healthcare * Project information: This is an investigative project that aims to look at the entire packager workflow as a single piece of tooling to identify where failures happen, and try to identify when and why packages fail at different points within the workflow. We hope to have two possible outcomes from this project at the end of the quarter (September): * The workflow breaks at X point and we will work on a solution to fix * OR * The workflow works fine, but we will need better monitoring on the pipeline so we will work on a solution for this * The team are using this Monitor-Script https://pagure.io/fedora-ci/monitor-gating/ * Current status: failing consistently because of CI test timeouts * The jenkins instance being used is under-resourced, jobs waiting up to 40 minutes for a node * They are using a workable dump of historical datanommer/datagrepper data for queries to pull metrics on the historical/aggregate Packager Workflow from as a baseline * And are working on an outline of the workflow steps (from packager PoV) and systems involved (CPE team PoV), identifying metrics to be measured * The teams work is being tracked here https://teams.fedoraproject.org/project/cpe-cicd/kanban ## CentOS Updates ### CentOS * CentOS-infra project created https://pagure.io/centos-infra/ * Day to Day working with CentOS-infra being documented at: https://pagure.io/cpe/docs/pull-request/10 * New website is live https://www.centos.org ### CentOS Stream * Business as usual, the team are churning through building packages in Stream * They are also working on tooling to bring back some rpkg features As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great week! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ?view -- Aoife Moloney Product Owner Community Platform Engineering Team Red Hat EMEA Communications House Cork Road Waterford

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 22 Jul '20

22 Jul '20

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2020-07-23 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

The state of the planet (the app)
by Pierre-Yves Chibon 20 Jul '20

20 Jul '20

Good Morning Everyone, Running a personal planet and having upgraded the box running it to a recent OS (Fedora in this case), I have found out that the underlying application does not support python3. To be precise, the main script says: Requires Python 2.1, recommends 2.3. Needless to say, this doesn't work anymore with python3. So I went looking a bit around to see what is available. There is Venus which we run, which has this problem ^ and hasn't been touched in 9 years: https://github.com/rubys/venus There are a few java-based, php-based, or ruby-based, with different level of activity (though to be fair, RSS and planets have been there for a while now, so a working application likely does not need much maintenance anymore). However, nothing jumped at me as a clear successor for our setup. Do people have some experience with planet? Would you recommend us something? At this stage, it is very unclear to me if the best approach is to switch application (new stack, new UI to do, new configuration file to do/adjust...) or port venus to python3 (potentially a 1 time effort but what about the long term maintenance?). Looking forward to hear your thoughts, Pierre

4 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure July 2020