infrastructure December 2017

infrastructure@lists.fedoraproject.org

22 participants
27 discussions

[Release] pagure-dist-git: 0.11.1
by Pierre-Yves Chibon 20 Dec '17

20 Dec '17

Good Morning Everyone, I just cut a release of pagure-dist-git: 0.11.1. Here is its changelog: * Wed Dec 20 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.11.1-1 - Update to 0.11.1 - Ensure releng has always commit ACL - Do not consider forks when generating the extras JSON files for POC and bugzilla Since both fix were already applied in prod, the package has been updated to both stg and prod. Happy packaging! Pierre

1 0

[Release] pagure: 3.13.1
by Pierre-Yves Chibon 20 Dec '17

20 Dec '17

Good Morning Everyone, I just cut a new release of pagure: 3.13.1. Here is its changelog: * Tue Dec 19 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 3.13.1-1 - Update to 3.13.1 - Add an alembic migration removing a constraint on the DB that not only no longer needed but even blocking regular use now The migration worked fine in stg.pagure.io and src.stg.fp.o and the symptoms I was seeing no longer appear. If all goes fine I'll likely update production tomorrow morning. Happy hacking and packaging! Pierre

1 1

[Release] pagure: 3.13
by Pierre-Yves Chibon 18 Dec '17

18 Dec '17

Good Morning Everyone, I just cut a new release of pagure: 3.13. I meant to do a 3.12.1 but a couple of feature slipped in the release and made it a 3.13 instead. Here is the changelog: * Mon Dec 18 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 3.13-1 - Update to 3.13 - Fix the alembic migration adjusting the pull_requests table - Fix how is created the db in the docker development environment (Clement Verna) - Ensure optional dependencies remain optional - Ensure groups cannot be created when it is not allowed - When listing issues, include the project as well in the user's issue API endpoint - Sort forks by date of creation (descending) (Neha Kandpal) - Ensure the pagination arguments are returned when a page is specified - Make the milestone clickable on the issue page - Make the celery tasks update their status so we know when they are running (vs pending) It's running in staging (both stg.pagure.io and src.stg.fp.o) but while testing it I found out that there is a database constraint that did not get lifted in 3.12 and prevents deleting a project from working as expected. The corresponding fix is under review, so expect a 3.13.1 soon :) Happy hacking! Pierre

1 0

MBS disk usage
by Ralph Bean 15 Dec '17

15 Dec '17

Way back in May some of us were in Raleigh for a Hackathon: https://fedoraproject.org/wiki/CI_and_Infrastructure_Hackathon_2017 There, we talked about the MBS and wondered what kind of new storage requirements it would impose on koji. We all worried about it, but we didn't have numbers to figure out how worried we should be. --- Here, below, find a query that Mike McLean shared for the koji database. It very, very roughly shows rpm storage usage per-quarter in koji, with a couple caveats. - It only counts rpm sizes. It doesn't count logs, or signed copied, or isos, or anything like that. - It is also biased towards newer quarters. More recent quarters haven't had time yet for garbage collection to kick in, so while they are larger in magnitude, it needs to be taken in context, with salt, etc. Here's a run:: koji=# select date_trunc('quarter', completion_time) as bin, round(sum(size1)/1073741824, 3) as rpm_gb, round(sum(size2)/1073741824,3) as archive_gb, round(sum(size1+size2)/1073741824, 3) as total_gb from (select build.id, build.completion_time, coalesce((select sum(size) from rpminfo where build_id=build.id),0) as size1, coalesce((select sum(size) from archiveinfo where build_id=build.id),0) as size2 from build where build.state=1 and build.volume_id=0) as data group by date_trunc('quarter', completion_time) order by bin; bin | rpm_gb | archive_gb | total_gb --------------------+----------------------------- 2007-04-01 00:00:00 | 254.416 | 0.000 | 254.416 2007-07-01 00:00:00 | 314.764 | 0.000 | 314.764 2007-10-01 00:00:00 | 317.851 | 0.000 | 317.851 2008-01-01 00:00:00 | 427.718 | 0.000 | 427.718 2008-04-01 00:00:00 | 393.187 | 0.000 | 393.187 2008-07-01 00:00:00 | 404.171 | 0.000 | 404.171 2008-10-01 00:00:00 | 392.340 | 0.000 | 392.340 2009-01-01 00:00:00 | 565.137 | 0.000 | 565.137 2009-04-01 00:00:00 | 349.749 | 0.000 | 349.749 2009-07-01 00:00:00 | 464.736 | 0.000 | 464.736 2009-10-01 00:00:00 | 324.401 | 0.000 | 324.401 2010-01-01 00:00:00 | 331.923 | 0.000 | 331.923 2010-04-01 00:00:00 | 304.823 | 0.000 | 304.823 2010-07-01 00:00:00 | 293.123 | 0.000 | 293.123 2010-10-01 00:00:00 | 262.374 | 0.000 | 262.374 2011-01-01 00:00:00 | 307.806 | 0.000 | 307.806 2011-04-01 00:00:00 | 243.162 | 0.000 | 243.162 2011-07-01 00:00:00 | 239.072 | 0.000 | 239.072 2011-10-01 00:00:00 | 287.522 | 0.000 | 287.522 2012-01-01 00:00:00 | 388.169 | 0.000 | 388.169 2012-04-01 00:00:00 | 347.497 | 0.000 | 347.497 2012-07-01 00:00:00 | 384.210 | 0.000 | 384.210 2012-10-01 00:00:00 | 369.828 | 0.000 | 369.828 2013-01-01 00:00:00 | 522.266 | 0.000 | 522.266 2013-04-01 00:00:00 | 451.709 | 0.000 | 451.709 2013-07-01 00:00:00 | 733.928 | 0.000 | 733.928 2013-10-01 00:00:00 | 551.316 | 0.000 | 551.316 2014-01-01 00:00:00 | 530.781 | 0.000 | 530.781 2014-04-01 00:00:00 | 747.820 | 0.000 | 747.820 2014-07-01 00:00:00 | 1008.080 | 0.000 | 1008.080 2014-10-01 00:00:00 | 665.198 | 0.000 | 665.198 2015-01-01 00:00:00 | 839.060 | 0.000 | 839.060 2015-04-01 00:00:00 | 931.210 | 0.000 | 931.210 2015-07-01 00:00:00 | 887.543 | 0.000 | 887.543 2015-10-01 00:00:00 | 700.465 | 0.000 | 700.465 2016-01-01 00:00:00 | 1189.887 | 152.489 | 1342.376 2016-04-01 00:00:00 | 983.871 | 43.799 | 1027.670 2016-07-01 00:00:00 | 983.195 | 99.737 | 1082.932 2016-10-01 00:00:00 | 1165.745 | 27.005 | 1192.749 2017-01-01 00:00:00 | 1681.824 | 149.185 | 1831.009 2017-04-01 00:00:00 | 1861.706 | 8.289 | 1869.995 2017-07-01 00:00:00 | 2610.738 | 132.464 | 2743.203 2017-10-01 00:00:00 | 2236.426 | 4195.877 | 6432.303 (43 rows) Interesting! Now, let's see the same query, but with the MBS builds *excluded* (the MBS user is userid 3819):: koji=# select date_trunc('quarter', completion_time) as bin, round(sum(size1)/1073741824, 3) as non_mbs_rpm_gb, round(sum(size2)/1073741824,3) as non_mbs_archive_gb, round(sum(size1+size2)/1073741824, 3) as non_mbs_total_gb from (select build.id, build.completion_time, coalesce((select sum(size) from rpminfo where build_id=build.id),0) as size1, coalesce((select sum(size) from archiveinfo where build_id=build.id),0) as size2 from build where build.state=1 and build.volume_id=0 and build.owner!=3819) as data group by date_trunc('quarter', completion_time) order by bin; bin | non_mbs_rpm_gb | non_mbs_archive_gb | non_mbs_total_gb ----------------------------------------------------+----------------- 2007-04-01 00:00:00 | 254.416 | 0.000 | 254.416 2007-07-01 00:00:00 | 314.764 | 0.000 | 314.764 2007-10-01 00:00:00 | 317.851 | 0.000 | 317.851 2008-01-01 00:00:00 | 427.718 | 0.000 | 427.718 2008-04-01 00:00:00 | 393.187 | 0.000 | 393.187 2008-07-01 00:00:00 | 404.171 | 0.000 | 404.171 2008-10-01 00:00:00 | 392.340 | 0.000 | 392.340 2009-01-01 00:00:00 | 565.137 | 0.000 | 565.137 2009-04-01 00:00:00 | 349.749 | 0.000 | 349.749 2009-07-01 00:00:00 | 464.736 | 0.000 | 464.736 2009-10-01 00:00:00 | 324.401 | 0.000 | 324.401 2010-01-01 00:00:00 | 331.923 | 0.000 | 331.923 2010-04-01 00:00:00 | 304.823 | 0.000 | 304.823 2010-07-01 00:00:00 | 293.123 | 0.000 | 293.123 2010-10-01 00:00:00 | 262.374 | 0.000 | 262.374 2011-01-01 00:00:00 | 307.806 | 0.000 | 307.806 2011-04-01 00:00:00 | 243.162 | 0.000 | 243.162 2011-07-01 00:00:00 | 239.072 | 0.000 | 239.072 2011-10-01 00:00:00 | 287.522 | 0.000 | 287.522 2012-01-01 00:00:00 | 388.169 | 0.000 | 388.169 2012-04-01 00:00:00 | 347.497 | 0.000 | 347.497 2012-07-01 00:00:00 | 384.210 | 0.000 | 384.210 2012-10-01 00:00:00 | 369.828 | 0.000 | 369.828 2013-01-01 00:00:00 | 522.266 | 0.000 | 522.266 2013-04-01 00:00:00 | 451.709 | 0.000 | 451.709 2013-07-01 00:00:00 | 733.928 | 0.000 | 733.928 2013-10-01 00:00:00 | 551.316 | 0.000 | 551.316 2014-01-01 00:00:00 | 530.781 | 0.000 | 530.781 2014-04-01 00:00:00 | 747.820 | 0.000 | 747.820 2014-07-01 00:00:00 | 1008.080 | 0.000 | 1008.080 2014-10-01 00:00:00 | 665.198 | 0.000 | 665.198 2015-01-01 00:00:00 | 839.060 | 0.000 | 839.060 2015-04-01 00:00:00 | 931.210 | 0.000 | 931.210 2015-07-01 00:00:00 | 887.543 | 0.000 | 887.543 2015-10-01 00:00:00 | 700.465 | 0.000 | 700.465 2016-01-01 00:00:00 | 1189.887 | 152.489 | 1342.376 2016-04-01 00:00:00 | 983.871 | 43.799 | 1027.670 2016-07-01 00:00:00 | 983.195 | 99.737 | 1082.932 2016-10-01 00:00:00 | 1165.745 | 27.005 | 1192.749 2017-01-01 00:00:00 | 1657.196 | 149.185 | 1806.381 2017-04-01 00:00:00 | 1565.237 | 8.289 | 1573.526 2017-07-01 00:00:00 | 2417.401 | 132.464 | 2549.865 2017-10-01 00:00:00 | 1932.821 | 4195.869 | 6128.691 (43 rows) Finally, let's see the storage usage of only MBS-managed builds:: koji=# select date_trunc('quarter', completion_time) as bin, round(sum(size1)/1073741824, 3) as mbs_rpm_gb, round(sum(size2)/1073741824,3) as mbs_archive_gb, round(sum(size1+size2)/1073741824, 3) as mbs_total_gb from (select build.id, build.completion_time, coalesce((select sum(size) from rpminfo where build_id=build.id),0) as size1, coalesce((select sum(size) from archiveinfo where build_id=build.id),0) as size2 from build where build.state=1 and build.volume_id=0 and build.owner=3819) as data group by date_trunc('quarter', completion_time) order by bin; bin | mbs_rpm_gb | mbs_archive_gb | mbs_total_gb --------------------------------------------+------------- 2017-01-01 00:00:00 | 24.627 | 0.000 | 24.627 2017-04-01 00:00:00 | 296.469 | 0.000 | 296.469 2017-07-01 00:00:00 | 193.337 | 0.000 | 193.337 2017-10-01 00:00:00 | 303.605 | 0.008 | 303.612 (4 rows)

1 0

December status update for Fedora Infrastructure Apprentices
by Kevin Fenzi 14 Dec '17

14 Dec '17

You are getting this email because you are in the 'fi-apprentice' group in the fedora account system (or are reading this on the infrastructure list). I am doing something different this time with this email: You should reply (to me or the list) if you have any suggestions or ideas on how to improve the apprentice program in the coming year. I'm not going to remove anyone for inactivity in December. I plan on thinking on our process over the holidays and sending out a new email on Jan 2nd about any proposed changes in process, etc. Happy Holidays to everyone. kevin

3 6

Plan for Thursdays Fedora Infrastructure Meeting (2017-12-14)
by Stephen John Smoogen 13 Dec '17

13 Dec '17

The infrastructure team will be having it's weekly meeting tomorrow, 2017-12-14 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. The document as of 2017-12-14 22:20 read as: This shared document is for the next fedora infrastructure meeting. = Introduction = We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2017-12-14) #meetingname infrastructure #topic aloha #chair smooge relrod nirik dgilmore threebean pingou puiterwijk pbrobinson maxamillion = Let new people say hello = #topic New folks introductions #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves = Status / Information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info PHX2 Colo Trip, Dec 4th - 9th. Trip was success. All Fedora servers were moved #info Fedora Infrastructure SemiFreeze begins 2017-12-21 #info Many people on vacation. = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus /suor decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic Ticket cleanup #info none this week. #topic PHX2 Move #info Lessons Learned(LL): Having a good plan does work #info LL: Make sure that the people who need to be there are there #info LL: New cages have 30 amp power. #info LL: You can put 1 Dell FX per 2 30 amp power supplies #info LL: Inventory is hard #topic Fedora Infrastructure SemiFreeze #info Red Hat does a year end shutdown every year. #info Most of infrastructure is on vacation during this time #info People will be on IRC every now and then. Do no expect replies to queries #info All changes to infrastructure from 2017-12-21 -> 2018-01-02 need +1 via IRC or email #info Use email or infrastructure tickets so things are captured. = Apprentice office hours = #topic Apprentice Open office hours Here we will discuss any apprentice questions, try and match up people looking for things to do with things to do, progress, testing anything like that. = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the i/nfo in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #topic Learn about: #info none this week = Meeting end stuff = #topic Open Floor #endmeeting -- Stephen J Smoogen.

1 0

OpenStack endpoints in fed-cloud09
by Miroslav Suchy 13 Dec '17

13 Dec '17

Hi, I am curious about current state of fed-cloud09. I wanted to use ansible module os_server to spin up new VM in Fedora Cloud, but I got error that ansible cannot connect to: http://fedorainfracloud.org:8696/v2.0/networks That was strange as 8696 should be an internal port for neutron. HAProxy should listen on default port 9696, handle the ssl stuff, and then forward it to localhost:8696 which should not be propagated to outside networks. (And indeed curl of https://fedorainfracloud.org:9696/ works). When I query endpoints I get: # openstack catalog list .... | neutron | network | RegionOne | | | | publicURL: http://fedorainfracloud.org:8696/ | | | | internalURL: http://fedorainfracloud.org:8696/ | | | | adminURL: http://fedorainfracloud.org:8696/ This is strange, because in our playbooks/hosts/fed-cloud09.cloud.fedoraproject.org.yml playbook we have: - shell: source /root/keystonerc_admin && keystone service-list | grep 'neutron' | awk '{print $2}' check_mode: no changed_when: false register: SERVICE_ID - shell: source /root/keystonerc_admin && keystone endpoint-list | grep {{SERVICE_ID.stdout}} | awk '{print $2}' check_mode: no changed_when: false register: ENDPOINT_ID - shell: source /root/keystonerc_admin && keystone endpoint-list |grep {{SERVICE_ID.stdout}} |grep -v {{ controller_publicname }} && (keystone endpoint-delete {{ENDPOINT_ID.stdout}} && keystone endpoint-create --region 'RegionOne' --service {{SERVICE_ID.stdout}} --publicurl 'https://{{ controller_publicname }}:9696/' --adminurl 'https://{{ controller_publicname }}:9696/' --internalurl 'https://{{ controller_publicname }}:9696/' ) || true Which should set publicURL to 9696. It seems that this is set for some time (but was not for sure in past). I before I run the fed-cloud09 playbook (which itself can break a lot of stuff if some stuff was done manually and not put in playbook) and change the public port to 9696 back (and generaly to ssl version for all services), whether there is some reason I should not touch it? Mirek

3 2

Infrastructure move -- thanks
by Paul W. Frields 11 Dec '17

11 Dec '17

I might be a bit biased from my perspective as the guy who manages the folks on the Infrastructure team. But I wanted to take a moment to say thank you to all the team members and everyone involved for conducting the recent colo move, a complex and critical process, in an outstanding manner. There were some difficulties along the way and the team dealt with them in their usual professional way. Kudos to all, and well done! -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ The open source story continues to grow: http://opensource.com

2 1

Holiday Reminder
by Kevin Fenzi 08 Dec '17

08 Dec '17

Greetings everyone. Just a reminder that holidays are coming up and that means that many people won't be around as much. * If whatever it is can wait, just wait and bring it up next year after people come back. * If you send email don't expect quick answers. * If it's urgent and no one is around, please file a ticket and someone will get to it as soon as we can. Do try not to make any infrastructure changes that might cause alerts or down machines. Red Hat has shutdown period starting Dec 22nd and going until Jan 1st. So, expect all those folks to be back around jan 2nd (tuesday). I personally am off starting monday the 11th. I should be around working on personal projects and the like, but will try not to be working. Hope everyone has a very happy holiday season and gets in some Open Source contributing too! :) kevin

1 0

[Release] pagure: 3.12
by Pierre-Yves Chibon 08 Dec '17

08 Dec '17

Good Morning Everyone, I just cut a new release of pagure: 3.12 Here is its changelog: * Fri Dec 08 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 3.12-1 - Update to 3.12 - Adjust the API endpoint listing project to not return a 404 when not projects are found (Vivek Anand) - Remove --autoreload from the docker dev deployment (Vivek Anand) - Fix ordering issues (Patrick Uiterwijk) - Do not log actions pertaining to private issues, PRs or projects - Fix flagging a PR when no uid is specified - Fix the doc about custom gitolite config - Fix displaying the filename on the side and linking to file for remote PRs - Add irc info in Readme (Vivek Anand) - Make pagure compatible with newer python chardet - Check that the identifier isn't the hash of a git tree in view_file - Fix if the identifier provided is one of a blob instead of a commit in view_commit - Include the status when flagging a PR via jenkins - Enable OpenID Connect authentication (Slavek Kabrda) - Use the updated timestamp in the pull-request list - Add migration to fix the project_from_id foreign key in pull_requests - Let the SSE server to send the notifications so they can be displayed live - Improve the createdb script to support stamping the database in the initial run - Specify a different connection and read timeout in pagure-ci - Small CSS fix making the (un)subscribe show up on the PR page It's running in staging but required two hotfixes, one of which has already been reviewed and merged in git, the other to come shortly. So I'll do a 3.12.1 before pushing things to prod :) Happy hacking! Pierre

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure December 2017