infrastructure July 2015

infrastructure@lists.fedoraproject.org

31 participants
41 discussions

Freeze Break request: playbook run over all hosts with fedmsgdconfig tag
by Kevin Fenzi 29 Jul '15

29 Jul '15

Greetings. The addition of mm-frontend02 yesterday meant that all the dynamically generated fedmsg config needs to add a entry for it. This means the check/diff daily report is pretty large because all these are pending. I would like to run the master playbook with -t fedmsgdconfig and then -t fedmsgmonitor I don't think there will be any issues. +1s? This will make the following changes on fedmsg enabled hosts: --- before: /etc/fedmsg.d/endpoints.py +++ after: /srv/web/infra/ansible/roles/fedmsg/base/templates/endpoints.py.j2 @@ -41,20 +41,24 @@ ], "askbot.ask02": [ "tcp://ask02.%s:30%02i" % (suffix, i) for i in range(6) ], # The mirrormanager2 frontend runs as 2 processes with 1 thread each. "mirrormanager2.mm-frontend01": [ "tcp://mm-frontend01.%s:30%02i" % (suffix, i) + for i in range(2) + ], + "mirrormanager2.mm-frontend02": [ + "tcp://mm-frontend02.%s:30%02i" % (suffix, i) for i in range(2) ], # mizdebsk says he thinks we'll need four sockets. Two services # (koschei-resolver and koschei-polling) are only sending messages, one # service (koschei-watcher) can both send and receive them. The wsgi # webapp doesn't use fedmsg at all and all services are single-threaded # koschei-watcher uses two endpoints "koschei.koschei01": [ "tcp://koschei01.%s:30%02i" % (suffix, i)

3 3

Freeze break request: decrease number of crawlers to 29
by Adrian Reber 29 Jul '15

29 Jul '15

Can I get two +1's for: commit 863cbad2434f5bf612a59cce7cfcb9b242177ad7 Author: Adrian Reber <adrian(a)lisas.de> Date: Wed Jul 29 19:20:19 2015 +0000 Reduce the number of parallel crawled hosts to 29. There has been a report that the MM database was not updated correctly and dmesg on mm-crawler01 shows three OOM killed crawlers. https://fedorahosted.org/fedora-infrastructure/ticket/4845 diff --git a/roles/mirrormanager/crawler/files/crawler.cron b/roles/mirrormanager/crawler/files/crawler.cron index 43f85f7..f77d61d 100644 --- a/roles/mirrormanager/crawler/files/crawler.cron +++ b/roles/mirrormanager/crawler/files/crawler.cron @@ -1,8 +1,8 @@ # run the crawler twice a day # logs sent to /var/log/mirrormanager/crawler.log and crawl/* by default -# 30 threads with 32GB of RAM seems to work so far +# 29 threads with 32GB of RAM seems to work so far # # [ "`hostname -s`" == "mm-crawler02" ] && sleep 2h is used to start the crawl # later on the second crawler to reduce the number of parallel accesses to # the database -0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 2h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 30 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1 +0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 2h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 29 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1

3 3

Freeze break request? Pagure update
by Pierre-Yves Chibon 29 Jul '15

29 Jul '15

Hi everyone, I was planning on making a new release of pagure today, not realizing that we were entering freeze. I would like to hear more opinions about pushing this release, here is the current changelog: - Include the tags in the JSON representation of a project - Add the ability to open a pull-request from a git repo not hosted on pagure - Fix pagination when browsing the list of commits - Fix the fork button when viewing the Settings of a project - Adjust the example apache configuration file - Add a favicon with pagure's logo - Fix asynchronous commentting on pull-requests - Start working on some documentation on how to install pagure - Do no flash messages when a comment is submitted via javascript (ie: async) - Do not blink the tittle of the page if the page is already on focus - Retrieve ssh key from FAS and set it up in pagure if none is currently set-up The remote PR needs a small DB change, everything else is pretty much bug fixes or minor improvements. Thoughts? Thanks, Pierre

4 10

Freeze break request: re-add qadevel.qa.fedoraproject.org to backups
by Kevin Fenzi 29 Jul '15

29 Jul '15

qadevel.qa.fedoraproject.org was out of backups waiting for a firewall change so the backup01 machine could reach it. This has now been completed, so I'd like to readd it to backups. +1s? diff --git a/inventory/backups b/inventory/backups index ed92428..e1693d5 100644 --- a/inventory/backups +++ b/inventory/backups @@ -14,8 +14,7 @@ pagure01.fedoraproject.org people01.fedoraproject.org pkgs02.phx2.fedoraproject.org log01.phx2.fedoraproject.org -# re-add this once we have rhit firewall allowing it -#qadevel.qa.fedoraproject.org:222 +qadevel.qa.fedoraproject.org:222 db-qa01.qa.fedoraproject.org db-koji01.phx2.fedoraproject.org copr-be.cloud.fedoraproject.org kevin

3 3

Freeze break request: Setup G11N fedmsg irc bot
by Patrick Uiterwijk 29 Jul '15

29 Jul '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Could I get +1s for the following change that adds fedmsg ircbot for the G11N team? commit a8ddd164691692775e89f6e10802aec65e6d0075 Author: Patrick Uiterwijk <puiterwijk(a)redhat.com> Date: Wed Jul 29 10:13:27 2015 +0000 Setup G11N with fedmsg irc diff --git a/roles/fedmsg/irc/templates/ircbot.py b/roles/fedmsg/irc/templates/ircbot.py index 8b76651..f26ec01 100644 - --- a/roles/fedmsg/irc/templates/ircbot.py +++ b/roles/fedmsg/irc/templates/ircbot.py @@ -174,6 +174,25 @@ config = dict( ), ), + # And for #fedora-g11n + dict( + network='chat.freenode.net', + port=6667, + make_pretty=True, + make_terse=True, + + {% if env == 'staging' %} + nickname='fedmsg-g11n-stg', + {% else %} + nickname='fedmsg-g11n', + {% endif %} + channel='#fedora-g11n', + # If the word G11N appears in any message, forward it. + filters=dict( + body=['^((?!G11N).)*$'], + ), + ), + # Hook up the design-team with badges messages dict( network='chat.freenode.net', - -- With kind regards, Patrick Uiterwijk Fedora Infra -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVuKd8AAoJEIZXmA2atR5QmcQP/1MOTzKUegP5FQDfX25cce3j aPwBS4UMZqkt4QJqjeSsqf1qzZuDq62nI+SQo33PmZ8D6dOCy7F8vN2uWNm73ygB 8zSLqdz79R1lLZ2ElTDG70CL+z3qV4L4Tkt+MYJzIizrZ4GQCe22v5NfxpiCdZ2j WvWKKvUV3noZu8TpBmNM/L6aGLcEBFfKfHVYJtCnOTbX76ILt5R+tHMzbrj6oX7R REXsK6HkGRxbmdsgj/vQ9TAbYaNKTYs5nl3L8YWzc8Pa8INRz0S9FUbHqGGbg31/ NgWNjT6E1kx4S26f9MnsFhswYnU+so1okfoipqjkto5GBXJpF9LaZpa+OHIT0Ln3 YUEPMawt8PDeGeMDnsXSSh+nNGlh8p/RJBQy5SiksdUPiK24yONWzo2DXY7hJfOI cpIspj6Zo5yG9jsnrHGz+aX5qABfSUboKUnUq8rNCmGUKLlC6uSuIFIKsIgomiw6 fAWeR6bjU+La99ki04xZkySDv1xcyycen7DMrLqeH6c4lSWKUX902XikN5lTiXY8 zwQTqD1o5LpPSfYTEltkKB20GcP/AgZC1etz3yH668B7XTe7SzzQRWpmsVJ82M9d m928akl2gv8KnYysUZWSVnbvgqlh8bRxgCeR41GxuDQQtLRL+01PwY3RUQWqN9/K uqGU2Sg/h+Pk7WVYMQDM =fWbe -----END PGP SIGNATURE-----

3 2

[release] MirrorManager2: 0.4.1
by Pierre-Yves Chibon 29 Jul '15

29 Jul '15

Good morning everyone, I just released a new version of MirrorManager2: 0.4.1 Here is its changelog: * Tue Jul 28 2015 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 0.4.1-1 - Update to 0.4.1 - If the host has no categories do not auto-disable it (Adrian Reber) - Different small umdl and crawler fixe (Adrian Reber) As you can see it is a bug-fix release with few fixes by Adrian. It is currently running in staging and will be pushed to prod once testing is done. Thanks, Pierre

3 4

RE: [release] MirrorManager2: 0.4.1
by Patrick Uiterwijk 29 Jul '15

29 Jul '15

1 0

Freeze break request: [post] Add a pagure.org -> pagure.io redirect
by Patrick Uiterwijk 29 Jul '15

29 Jul '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Could I get retroactive +1s for the following commit, which fixes pagure.org urls (instaad of getting them trapped in the docs.pagure.org vhost). 1 files changed, 22 insertions(+), 0 deletions(-) [puiterwijk@lockbox01 templates]$ git show HEAD commit 0a7efdc4fb9fa7b14d2447294cae56100624f9b7 Author: Patrick Uiterwijk <puiterwijk(a)redhat.com> Date: Wed Jul 29 09:57:44 2015 +0000 Make a pagure.org -> pagure.io redirect diff --git a/roles/pagure/frontend/templates/0_pagure.conf b/roles/pagure/frontend/templates/0_pagure.conf index ea81a92..05bb090 100644 - --- a/roles/pagure/frontend/templates/0_pagure.conf +++ b/roles/pagure/frontend/templates/0_pagure.conf @@ -27,6 +27,28 @@ WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-na {% endif %} </VirtualHost> +<VirtualHost *:443> +{% if env == 'pagure-staging' %} + ServerName stg.pagure.org +{% else %} + ServerName pagure.org +{% endif %} + + SSLEngine on + SSLProtocol all -SSLv2 -SSLv3 + # Use secure TLSv1.1 and TLSv1.2 ciphers + Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" + + SSLCertificateFile /etc/pki/tls/certs/docs.pagure.org.crt + SSLCertificateChainFile /etc/pki/tls/certs/docs.pagure.org.intermediate.crt + SSLCertificateKeyFile /etc/pki/tls/certs/docs.pagure.org.key +{% if env == 'pagure-staging' %} + Redirect permanent / https://stg.pagure.io/ +{% else %} + Redirect permanent / https://pagure.io/ +{% endif %} +</VirtualHost> + <VirtualHost *:443> {% if env == 'pagure-staging' %} - -- With kind regards, Patrick Uiterwijk Fedora Infra -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVuKQ5AAoJEIZXmA2atR5Q/voP/jm5JZtzX1StwAN07b8CxfRK 2dWxTL7nWagRqt+LcmootLNHb4GmH58wyS2XpEa+BlnDt33BeEK8AuoO4QdbISgI tWygWNQwyPaME4gPRdxCNhnJ8Xo+9S/ss5j98dMqQz31MnwclxXdMsz0NYlyE4NX DKgxfiJayelP6EjR1lfjnBTag7S+Hoj3QXBYU74ACky8Um0SKOLc/nRjX2ccHYFL q3qwz4VedunKFvCmlhybGXVs5CANA6Z+W846jXQSiBXx7WaoduvIIYzKVX9Zmdll ZMvjNk6ZmkKDuqKGG51FMyjMe4J0RMu9TW7pWWDYMz6+yOLcrJfUDp17w1JVbZ0g jnuA7XnsEGsvcWoLU4dtClE0SIkyRnrL+A44Ahpa+5Xo1M0eZxUZ+6kBq0LSl5Jo LhOpDhxQ25nVMqjMLjdmfn8+3usUl+8TkxSGnbeKljR6nnOtpA3wRCZ75UXmXf6/ sr6eTCm7RF6+kbkovU7F8jS9aqMDP+Pvzz/GyB4eFuODhUGjPHVsvbh9XKY3YZfH hW3wH85tabaoQJrH3OS0QPpnHln67Ey57tH9ZUroeNVKTx1ulw/o6/Dwdc34iY/D YqWi6ak7QSkTWnuvpEumoM+Uk0BsgQSSa1gE2lObMCfTzC2jyO5lmghlO/ET3RCY rkiGx/E3qRJ3qfp0dnDE =Lbie -----END PGP SIGNATURE-----

3 2

Fedora 23 Alpha Freeze now in effect
by Kevin Fenzi 28 Jul '15

28 Jul '15

Greetings. we are now in the infrastructure freeze leading up to the Fedora 23 Alpha release. This is a pre-release freeze. We do this to ensure that our infrastructure is stable and ready to release the Fedora 23 Alpha when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2015-08-12. (or later if Alpha slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, kevin

1 0

Old Cloud retirement
by Kevin Fenzi 27 Jul '15

27 Jul '15

Greetings. Over the last few weeks we have been working to get all the old instances off our old private cloud and on to the new one (or just terminated if no longer used). I'd like to try and get that all completed by this friday (2015-07-31). There's a few people I have mailed but haven't heard back from, and if they don't say anything by friday their instances may well be terminated. ;) In progress still is jenkins. I will look at moving the builders to the new cloud in the next few days and hopefully we can migrate to the fedora-22 based jenkins in the new cloud soon. I mailed the following people last week and again this morning, but haven't heard anything back yet. If you know any of these folks, please let them know their instances will be terminated this friday: skottler - skottler-rhel7 bkabrda ncoghlan - devpi besser82 - shogun-ca Please let me know if you have any issues or concerns with a friday deadline for the old cloud to go away. Thanks, kevin

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure July 2015