infrastructure April 2016

infrastructure@lists.fedoraproject.org

33 participants
75 discussions

[release] pagure: 2.0
by Pierre-Yves Chibon 22 Apr '16

22 Apr '16

Good Morning everyone, The last release of pagure was on March 1st, since then there has been very many changes to it, improvements and bug fixes together. In fact there has been so many changes, that it wasn't worth making this new release 1.3 and it is instead a 2.0. One of the main reason for bumping the major version number is the change to support pseudo-namespace. Pseudo-namespace means that while we do not have actual namespace, we do allow up to one slash '/' in a project name. Allowing for example `rpms/firefox` or `docker/pagure` (and you see where I'm going with this idea). For this change, we broke a couple of URLs: https://stg.pagure.io/pagure/053d8cc95fcd50c23a8b0a7f70e55f8d1cc7aebb became: https://stg.pagure.io/pagure/c/053d8cc95fcd50c23a8b0a7f70e55f8d1cc7aebb (Note the added /c/ in it) We introduced a backward compatibility fix for this change, but only if you use the full git hash (ie the two links above will work). While if you use a abbreviated version of the git hash: https://stg.pagure.io/pagure/053d8cc it will not redirect you to https://stg.pagure.io/pagure/c/053d8cc If you run your own instance of pagure, the structure of the git repos has also been changed, be sure to read the UPGRADING document: https://pagure.io/pagure/blob/master/f/UPGRADING.rst I have already noticed a couple of missing elements in there, I will add them to it shortly. The version 2.0 is currently, happily running in staging as far as I could see. I will do some more testing and I could use more people poking at it to see if everything works fine. Due to the length of the changelog, I'll conclude here and place it underneath. Happy hacking everyone, Pierre Changelog: * Fri Apr 22 2016 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 2.0-1 - Update to 2.0 - Rework the initial comment of a PR, making it less a comment and more something that belong to the PR itself - Fix showing or not the fork button when editing a comment on an issue or a PR and fix the highlighted tab when editing comment of an issue (Oliver Gutierrez) - Fix the count of comments shown on the page listing all the PRs to include only the comments and not the notifications (farhaanbukhsh) - In the settings page explain that API keys are personal (Lubomír Sedlář) - Rework the fedmsg message sent upon pushing commits, one message per push instead of one message per commit - Mark the page next/previous as disabled when they are (on browse pages) - Avoid the logout/login loop when logging out - Support rendering file with a `.markdown` extension - Fix the layout of the password change branch - Improve the documentation, add overview graphs, expand the usage section, improve the overview description - Fix checking if the user is an admin of a project or not (which was making the user experience confusing as they sometime had the fork button and sometime not) - Fix the pagination on the browse pages when the results are sorted - Disable the Commit and Files tabs if a repo is new - Update the pagure logo to look better (Ryan Lerch) - Allow anyone to fork any project (Ryan Lerch) - Fix searching on the browse pages by preventing submission of the 'enter' key (Ryan Lerch) - Rework the issue page to be a single, large form allowing to update the meta-data and comment in one action and fixing updating the page via SSE - Turn off the project's documentation by default to empty `Docs` tab leading to nothing - Fill the initial comment with the body of the commit message if the PR only has one commit (Ryan Lerch) - Add a plugin/git hook allowing to disable non fast-forward pushes on a branch basis - Fix asynchronous inline comments in PR by fixing the URL to which the form is submitted - Add a plugin/git hook allowing to trigger build on readthedocs.org upon git push, with the possibility to restrict the trigger to only certain branches - Automatically scroll to the highlighted range when viewing a file with a selection (Lubomír Sedlář) - Indicate the project's creation date in the overview page (Anthony Lackey) - Clear the `preview` field after adding a comment via SSE - Adjust the unit-tests for the change in behavior in pygments 2.1.3 - Fix listing all the request when the status is True and do not convert to text request.closed_at if it is in fact None - Improved documentation - Attempt to fix the error `too many open files` on the EventSource Server - Add a new param to runserver.py to set the host (Ryan Lerch) - Fix the of the Docs tab and the Fork button with rounded corners (Pedro Lima) - Expand the information in the notifications message when a PR is updated (Ryan Lerch) - Fix hidding the reply buttons when users are not authenticated (Paul W. Frields) - Improve the description of the git hooks (Lubomír Sedlář) - Allow reply to a notification of pagure and setting the reply email address as Cc - In the fedmsg git hook, publish the username of all the users who authored the commits pushed - Add an activity page/feed for each project using the information retrieved from datagrepper (Ryan Lerch) - Fix showing lightweight tags in the releases page (Ryan Lerch) - Fix showing the list of branches when viewing a file - Add priorities to issues, with the possibility to filter or sort them by it in the page listing them. - Add support for pseudo-namespace to pagure (ie: allow one '/' in project name with a limited set of prefix allowed) - Add a new plugin/hook to block push containing commits missing the 'Signed-off-by' line - Ensure we always use the default email address when sending notification to avoid potentially sending twice a notification - Add support for using the keyword Merge(s|d) to close a ticket or pull-request via a commit message (Patrick Uiterwijk) - Add an UPGRADING.rst documentation file explaining how to upgrade between pagure releases

2 1

Freeze break: adjust pagure's config
by Pierre-Yves Chibon 22 Apr '16

22 Apr '16

Hi all, The possibility to keep the URL backward compatible in pagure 2.0 is an option that is off by default. I would like to apply the following change to pagure's configuration file to turn it on. diff --git a/ roles/pagure/frontend/templates/pagure.cfg b/ roles/pagure/frontend/templates/pagure.cfg index 77cfdfc..2748861 100644 --- a/ roles/pagure/frontend/templates/pagure.cfg +++ b/ roles/pagure/frontend/templates/pagure.cfg @@ -196,3 +196,10 @@ SSH_KEYS = { } } {% endif %} + +# Allow the backward compatiblity endpoints for the old URLs schema to +# see the commits of a repo. This is only interesting if you pagure instance +# was running since before version 1.3 and if you care about backward +# compatibility in your URLs. +OLD_VIEW_COMMIT_ENABLED = True + The change is already done manually in stg so that we could check that it is working, but I would like to apply it in ansible for both stg and prod since it will not have any impact for prod until we upgrade to 2.0. +1s? Thanks, Pierre

3 3

Freeze break: add sysadmin-badges to batcave01
by Kevin Fenzi 22 Apr '16

22 Apr '16

Greetings. I'd like to add sysadmin-badges to batcave's fas groups. This would allow people who are sysadmin-badges access to batcave to do things. diff --git a/inventory/group_vars/batcave b/inventory/group_vars/batcave index 114cffa..e47cccd 100644 --- a/inventory/group_vars/batcave +++ b/inventory/group_vars/batcave @@ -8,7 +8,7 @@ tcp_ports: [ 80, 443 ] # Neeed for rsync from log01 for logs. custom_rules: [ '-A INPUT -p tcp -m tcp -s 10.5.126.13 --dport 873 -j ACCEPT', '-A INPUT -p tcp -m tcp -s 192.168.1.59 --dport 873 -j ACCEPT' ] -fas_client_groups: sysadmin-ask,sysadmin-build,sysadmin-cvs,sysadmin-main,sysadmin-web,sysadmin-noc,sysadmin-hosted,sysadmin-releng,sysadmin-qa,sysadmin-tools,sysadmin-cloud,sysadmin-bot,sysadmin-centos,sysadmin-koschei,sysadmin-datanommer,sysadmin-fedimg,fi-apprentice,sysadmin-regcfp +fas_client_groups: sysadmin-ask,sysadmin-build,sysadmin-cvs,sysadmin-main,sysadmin-web,sysadmin-noc,sysadmin-hosted,sysadmin-releng,sysadmin-qa,sysadmin-tools,sysadmin-cloud,sysadmin-bot,sysadmin-centos,sysadmin-koschei,sysadmin-datanommer,sysadmin-fedimg,fi-apprentice,sysadmin-regcfp,sysadmin-badges ansible_base: /srv/web/infra freezes: false +1s? kevin

4 4

Freeze Break Request: mark WD drives as not multipath
by Patrick Uiterwijk 21 Apr '16

21 Apr '16

Could I get +1s for this patch set? This will blacklist drives from WD from becoming multipath devices, which will fix the newly replaced drive in buildvmhost-11.

3 3

Freeze Break request: readd pkgs02 to backups
by Kevin Fenzi 20 Apr '16

20 Apr '16

A bit before the freeze, the pkgs02 backups of /srv failed. This is the largest and most anoying backup we have since it's tons and tons and tons of git repos with small files in it. After the freeze, I wonder if it wouldn't be worthwhile to run a 'git gc' on all our pkgs git repos. This would make git pack things into fewer files and in general make it happier for backups and maintainers. I removed it from backups, ran a cleanup on the failed backup and then got a new clean backup (which took several days). So, we should be able to re-add it into our normal backups now: diff --git a/inventory/backups b/inventory/backups index c7d4193..e6e7d30 100644 --- a/inventory/backups +++ b/inventory/backups @@ -13,7 +13,7 @@ batcave01.phx2.fedoraproject.org infinote.fedoraproject.org pagure01.fedoraproject.org people02.fedoraproject.org -#pkgs02.phx2.fedoraproject.org +pkgs02.phx2.fedoraproject.org log01.phx2.fedoraproject.org qadevel.qa.fedoraproject.org:222 db-qa01.qa.fedoraproject.org +1s? kevin

3 4

Hello to the Fedora Infrastructure!
by Miguel M. 20 Apr '16

20 Apr '16

Sending this email as my Hello, world introduction post. Name: Miguel Moll IRC and FAS handle: radioact1ve Skills: Software development with Go, Python, Ruby and C#. Also an intermediate level sysadmin. I want to help give back to the community that has given me so much already. Think it's a good time to start contributing where ever I can be of most value. I'm here shadowing the mailing list and IRC. As a side note, would anyone be able to add my FAS account to the 'fi-apprentice' group? That way I can keep looking around without breaking anything. Thanks for your time and looking forward to contributing! -- Miguel Moll miguel.moll(a)gmail.com

3 2

Plan for tomorrow's Fedora Infrastructure meeting - 2016-04-21
by Kevin Fenzi 20 Apr '16

20 Apr '16

The infrastructure team will be having it's weekly meeting tomorrow, 2016-04-21 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. kevin -- = Introduction = This shared document is for the next fedora infrastructure meeting. We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2016-04-21) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson #topic New folks introductions / Apprentice feedback = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info Freed up space on log01, should be good for a bit longer now - kevin #info db-koji01 moved to faster virthost and local storage - kevin #info daily emails from ask on the way to being fixed - kevin #info We are now in Fedora 24 Beta freeze! - everyone #info rkhunter template updated to use inventory_hostname - kevin #info pkgs02 /srv backups hopefully on track - kevin #info composer.stg nfs mounts fixed up - kevin #info = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus or decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic url shorteners - kevin #topic = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the info in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) Schedule: unknown. Sign up now? #topic Learn about: = Meeting end stuff = #topic Open Floor #endmeeting

1 0

Freeze Break request: sync up fedmsg rules
by Kevin Fenzi 20 Apr '16

20 Apr '16

Greetings. I'd like to run: time ansible-playbook /srv/web/infra/ansible/master.yml -t fedmsgdconfig,fedmsgmonitor ie, the fedmsgdconfig and fedmsgmonitor tags over all hosts. The reason is that we added a new fedmsg cert for the not yet active ppc-hub. So, this shouldn't change much at all, but it will make the daily ansible check-diff report much more readable and apparent when something is changed. +1s? kevin

4 3

Freeze break: change ansible_fqdn to inventory_hostname in rkhunter template
by Kevin Fenzi 19 Apr '16

19 Apr '16

Greetings. I'd like to apply the following and run the rhunter template over everything. This is targeted at host1plus however. At most this will impact rkhunter and nothing else. ansible_fqdn gets it's value by looking at the ip address of the machine and doing a reverse dns lookup. Mostly this is fine, but in some cases (like host1plus) the reverse dns is not controlled by us and is not as you might expect: $ host host1plus01.fedoraproject.org host1plus01.fedoraproject.org has address 5.175.150.48 host 5.175.150.48 48.150.175.5.in-addr.arpa domain name pointer gdm4.unidis.com.br. This results in the above template looking and seeing that "gdm4.unidis.com.br" is not in fact in the virthosts group and thus shouldn't have the line that allows /dev/shm/spice* files, so it sends a alert about it being there. (after freeze we should probibly go remove all uses of ansible_fqdn) diff --git a/roles/rkhunter/templates/rkhunter.conf.j2 b/roles/rkhunter/templates/rkhunter.conf.j2 index 7b76695..35e5576 100644 --- a/roles/rkhunter/templates/rkhunter.conf.j2 +++ b/roles/rkhunter/templates/rkhunter.conf.j2 @@ -386,7 +386,7 @@ ALLOWDEVFILE=/dev/shm/fmn-cache.dbm ALLOWDEVFILE=/dev/shm/squid-squid-page-pool.shm ALLOWDEVFILE=/dev/shm/squid-cache_mem.shm {% endif %} -{% if ansible_fqdn in groups['virtservers'] or ansible_fqdn in groups['openqa-workers'] or ansible_fqdn in groups['openqa-stg-workers'] %} +{% if inventory_hostname in groups['virtservers'] or inventory_hostname in groups['openqa-workers'] or inventory_hostname in groups['openqa-stg-workers'] %} # libvirt spice device makes a /dev/shm/spice file ALLOWDEVFILE=/dev/shm/spice.* {% endif %} +1s? kevin

3 3

Re: [ansible] Remove mirror01.prgmr.com, it's now the same as mirror.prgmr.com
by Adrian Reber 19 Apr '16

19 Apr '16

On Fri, Apr 15, 2016 at 02:01:34PM +0000, Nick Bebout wrote: > diff --git a/roles/rsyncd/files/rsyncd.conf.download-ibiblio b/roles/rsyncd/files/rsyncd.conf.download-ibiblio > index 15375aa..478bf48 100644 > --- a/roles/rsyncd/files/rsyncd.conf.download-ibiblio > +++ b/roles/rsyncd/files/rsyncd.conf.download-ibiblio > @@ -68,7 +68,7 @@ refuse options = checksum > list = no > uid = 263 > gid = 263 > - hosts allow = jobbot1.ibiblio.org 200.17.202.1/28 zeus1.kernel.org zeus2.kernel.org zeus3.kernel.org zeus4.kernel.org 149.20.20.132 204.152.191.36 199.6.1.170 130.239.17.3 sinclair.wpi.edu bonaparte.hrz.tu-chemnitz.de josephine.hrz.tu-chemnitz.de mirror.speedpartner.de rsyncer.ftp.heanet.ie archive.linux.duke.edu lists.us.dell.com auslistsprd01.us.dell.com auslistsdr01.us.dell.com 198.129.224.34 mirror.hiwaay.net sagres.c3sl.ufpr.br mail.fedoraunity.org scrye.com odysseus.fi.muni.cz odysseus.linux.cz rhlx01.hs-esslingen.de ftp.nrc.ca zaphod.gtlib.gatech.edu 128.171.104.148 129.21.171.98 torrent01.fedoraproject.org torrent02.fedoraproject.org sunsite.mff.cuni.cz sunsite.ms.mff.cuni.cz ultra.linux.cz ftp.cz.kernel.org 202.158.214.12 speculum.rbc.ru 71.19.151.18 152.19.134.145 152.19.134.195 mirrors.mit.edu solar-one.mit.edu 10.64.10.11 mirrors.xmission.com 182.255.111.7 2001:388:1:4066:225:90ff:fec7:777e mirror.prgmr.com mirror01.prgmr.com tiz-korg-mirror.kernel.org sfo-korg-mi > rror.kernel.org 129.7.128.189 129.7.128.190 129.101.198.59 frisal.switch.ch 208.96.144.70 208.96.144.16 > + hosts allow = jobbot1.ibiblio.org 200.17.202.1/28 zeus1.kernel.org zeus2.kernel.org zeus3.kernel.org zeus4.kernel.org 149.20.20.132 204.152.191.36 199.6.1.170 130.239.17.3 sinclair.wpi.edu bonaparte.hrz.tu-chemnitz.de josephine.hrz.tu-chemnitz.de mirror.speedpartner.de rsyncer.ftp.heanet.ie archive.linux.duke.edu lists.us.dell.com auslistsprd01.us.dell.com auslistsdr01.us.dell.com 198.129.224.34 mirror.hiwaay.net sagres.c3sl.ufpr.br mail.fedoraunity.org scrye.com odysseus.fi.muni.cz odysseus.linux.cz rhlx01.hs-esslingen.de ftp.nrc.ca zaphod.gtlib.gatech.edu 128.171.104.148 129.21.171.98 torrent01.fedoraproject.org torrent02.fedoraproject.org sunsite.mff.cuni.cz sunsite.ms.mff.cuni.cz ultra.linux.cz ftp.cz.kernel.org 202.158.214.12 speculum.rbc.ru 71.19.151.18 152.19.134.145 152.19.134.195 mirrors.mit.edu solar-one.mit.edu 10.64.10.11 mirrors.xmission.com 182.255.111.7 2001:388:1:4066:225:90ff:fec7:777e mirror.prgmr.com tiz-korg-mirror.kernel.org sfo-korg-mirror.kernel.org 129 > .7.128.189 129.7.128.190 129.101.198.59 frisal.switch.ch 208.96.144.70 208.96.144.16 These diffs of the rsync ACLs are not very useful. For me it is almost impossible to see if and what has changed. It also seems we have to maintain the ACL in 4 or 5 different files. Can the rsync ACL not be handled in a more ansible way? I don't know much about ansible but wouldn't it be possible to maintain the ACL one time with something like this: - name rsync acl template: some template with a loop statement over items with_items: - ip1 - ip2 - host1 - host2 That would make the diffs readable any maybe we could maintain the rsync ACL in only one place. Before trying to implement it I wanted to see if there are some better ideas/ways to implement this 'correctly'. Adrian

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure April 2016