infrastructure March 2019

infrastructure@lists.fedoraproject.org

40 participants
97 discussions

About JS framework
by Pierre-Yves Chibon 14 Sep '22

14 Sep '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

15 19

Fedora 30 Beta freeze now in efect
by Kevin Fenzi 03 Apr '19

03 Apr '19

Greetings. we are now in the infrastructure freeze leading up to the Fedora 30 Beta release. This is a pre release freeze. We do this to ensure that our infrastructure is stable and ready to release the Fedora 30 Beta when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git ansible/scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2019-03-26 (or later if release slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, kevin

1 1

[PATCH] distgit/pagure: Increase the cross-project ACLs
by Pierre-Yves Chibon 02 Apr '19

02 Apr '19

This just makes pagure accept to generate project-less API tokens with these two ACLs. Signed-off-by: Pierre-Yves Chibon <pingou(a)pingoured.fr> --- roles/distgit/pagure/templates/pagure.cfg | 2 -- 1 file changed, 2 deletions(-) diff --git a/roles/distgit/pagure/templates/pagure.cfg b/roles/distgit/pagure/templates/pagure.cfg index a66b3278b..385b80464 100644 --- a/roles/distgit/pagure/templates/pagure.cfg +++ b/roles/distgit/pagure/templates/pagure.cfg @@ -273,10 +273,8 @@ USER_ACLS = [ CROSS_PROJECT_ACLS = [ 'fork_project', 'modify_project', -{% if env == 'staging' %} 'pull_request_create', 'pull_request_comment', -{% endif %} ] ADMIN_API_ACLS = [ -- 2.20.1

4 3

FBR: Unfreeze RelEng to push updates
by Mohan Boddu 01 Apr '19

01 Apr '19

Hello, I dont know if this FBR is needed or not, but now that we have a GOLD compose for F30 Beta and all the requested f30 builds pushed stable and I cloned f30-Beta tag from f30, now we can start pushing updates to F30 stable as well along with the others. +1's please diff --git a/vars/all/RelEngFrozen.yaml b/vars/all/RelEngFrozen.yaml index bd7553f..72af779 100644 --- a/vars/all/RelEngFrozen.yaml +++ b/vars/all/RelEngFrozen.yaml @@ -1 +1 @@ -RelEngFrozen: True +RelEngFrozen: False Thanks.

3 2

Future of fedora-packages
by Clement Verna 01 Apr '19

01 Apr '19

Hi all, fedora-packages [0] code base is showing its age. The code base and the technology stack (Turbogears2 [1] web framework and the Moksha [2] middleware) is currently not ready for Python3 and I am not planning to do the work required to make it Python3 compatible, so the application will stop working when Fedora 29 is EOL. In order to keep the service running, I have started a Proof Of Concept (fedora-search [3]) to replace the backend of the application. Fedora-search would be a REST API service offering full test search API. Such a service would then be available for other application to use, fedora-packages would then become a frontend only application using the service provided by fedora-search. While the POC shows that this is a viable solution, I don't think that we should be proceeding that way, for the simple reason that this add yet another code base to maintain, I think we should use this opportunity to consider using Elasticsearch instead of maintaining our own "search engine". I think that Elasticsearch offers quite a few advantages : - Powerful Query language - Python bindings - Javascript bindings - Can be deployed in our infrastructure or used as a service - Can be useful for other applications ( docs.fp.o, pagure, ??) So what is the general feeling about using Elasticsearch in our infrastructure ? Should we look at deploying a cluster in our infra / Should we approach the Council to see if we can get founding to have this service hosted by Elastic ? Thanks Clément [0] - https://apps.fedoraproject.org/packages/ [1] - http://www.turbogears.org/ [2] - https://mokshaproject.github.io/mokshaproject.net/ [3] - https://github.com/fedora-infra/fedora-search

13 32

[Freeze Break Request] Add zchunk support to updates and updates-testing repositories
by Jonathan Dieter 31 Mar '19

31 Mar '19

This adds zchunk support for the updates and updates-testing repositories for both rpms and modularity. We already have zchunk metadata being generated for the fedora repository. I'd like to get this in before Beta comes out so Beta users will have zchunk-enabled updates-testing repositories when Beta is released. I am making the assumption that a zchunk-enabled createrepo_c (0.12.0-2 or later) is available on the builders (I think I'm safe making that assumption, since zchunk metadata is already being generated for some repos). I have *not* tested this patch, because I'm not sure how I'd go about doing so. If we don't have any test builders, my suggestion would be to wait until no compose is running, and then run this play on a builder, verifying that the generated pungi configuration is valid for both f29 and f30, with no createrepo_extra_args in f29. Signed-off-by: Jonathan Dieter <jdieter(a)gmail.com> --- roles/bodhi2/backend/templates/pungi.module.conf.j2 | 3 +++ roles/bodhi2/backend/templates/pungi.rpm.conf.j2 | 3 +++ 2 files changed, 6 insertions(+) diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2 index bb021eb13..7dad35403 100644 --- a/roles/bodhi2/backend/templates/pungi.module.conf.j2 +++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2 @@ -59,6 +59,9 @@ greedy_method = 'build' createrepo_c = True createrepo_checksum = 'sha256' createrepo_deltas = False +[% if release.version_int >= 30 %] +createrepo_extra_args = ['--zck', '--zck-dict-dir=/usr/share/fedora-repo-zdicts/f[[ release.version_int ]]'] +[% endif %] #jigdo create_jigdo = False diff --git a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 index 8d9e9a3f2..020736aee 100644 --- a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 +++ b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 @@ -66,6 +66,9 @@ createrepo_deltas = [ ('^Everything$', {'*': True}) ] createrepo_database = True +[% if release.version_int >= 30 %] +createrepo_extra_args = ['--zck', '--zck-dict-dir=/usr/share/fedora-repo-zdicts/f[[ release.version_int ]]'] +[% endif %] # CHECKSUMS media_checksums = ['sha256']

4 21

FBR: clean up disk space on mirrors
by Stephen John Smoogen 31 Mar '19

31 Mar '19

1. Remove F29beta from mirrors so mirrors can keep up /pub/fedora/linux/releases/test/29_Beta/ 2. Move F27 to archives a. Have mirrormanager point to /pub/archives/fedora/linux/27 b. Remove /pub/fedora/linux/27 -- Stephen J Smoogen.

4 7

FBR: some test machines
by Kevin Fenzi 31 Mar '19

31 Mar '19

Greetings. I'd like to apply the following to ansible, then run maintainer-test playbook (not frozen) and then noc playbook (frozen). This will add a f30-test instance in old cloud (x86_64) and a pair of aarch64 ones (f29) on two virthosts in the new community openshift area. +1s? kevin --

3 4

[FBR] cache headers for emojione.sprites.png
by Alessandro Lorenzi 30 Mar '19

30 Mar '19

Hi all, I've wrote two patch for issue #7629. pagure frontend: caching emojione sprites for 1 week roles/pagure/frontend/templates/0_pagure.conf | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/roles/pagure/frontend/templates/0_pagure.conf b/roles/pagure/frontend/templates/0_pagure.conf index 50c2d6e9b..87009fce0 100644 --- a/roles/pagure/frontend/templates/0_pagure.conf +++ b/roles/pagure/frontend/templates/0_pagure.conf @@ -78,6 +78,12 @@ WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-na Alias /static /usr/lib/python2.7/site-packages/pagure/static/ + <Location "/static/vendor/emojione/emojione.sprites.png"> + ExpiresActive On + ExpiresDefault "access plus 1 week" + Header append Cache-Control "public" + </Location> + SetEnv GIT_PROJECT_ROOT /srv/git/repositories <Location /> @@ -145,6 +151,12 @@ WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-na Alias /static /usr/lib/python2.7/site-packages/pagure/static/ + <Location "/static/vendor/emojione/emojione.sprites.png"> + ExpiresActive On + ExpiresDefault "access plus 1 week" + Header append Cache-Control "public" + </Location> + <Location /> WSGIProcessGroup paguredocs <IfModule mod_authz_core.c> -- 2.20.1 Site src.fp.org: caching emojione sprites for 1 week --- roles/fedora-web/src/files/expires.conf | 5 +++++ roles/fedora-web/src/tasks/main.yml | 12 ++++++++++++ roles/httpd/website/templates/website.conf | 5 ++++- 4 files changed, 22 insertions(+), 2 deletions(-) create mode 100644 roles/fedora-web/src/files/expires.conf diff --git a/roles/fedora-web/src/files/expires.conf b/roles/fedora-web/src/files/expires.conf new file mode 100644 index 000000000..199080880 --- /dev/null +++ b/roles/fedora-web/src/files/expires.conf @@ -0,0 +1,5 @@ +<Location "/static/vendor/emojione/emojione.sprites.png"> + ExpiresActive On + ExpiresDefault "access plus 1 week" + Header append Cache-Control "public" +</Location> diff --git a/roles/fedora-web/src/tasks/main.yml b/roles/fedora-web/src/tasks/main.yml index d7616e84c..fbe8e87d1 100644 --- a/roles/fedora-web/src/tasks/main.yml +++ b/roles/fedora-web/src/tasks/main.yml @@ -19,3 +19,15 @@ tags: - fedora-web - fedora-web/src + +- name: Copy some config files for {{website}} + copy: > + src={{item}} dest=/etc/httpd/conf.d/{{website}}/{{item}} + owner=root group=root mode=0644 + with_items: + - expires.conf + notify: + - reload proxyhttpd + tags: + - fedora-web + - fedora-web/src diff --git a/roles/httpd/website/templates/website.conf b/roles/httpd/website/templates/website.conf index 2a6241a77..931655e6a 100644 --- a/roles/httpd/website/templates/website.conf +++ b/roles/httpd/website/templates/website.conf @@ -20,6 +20,10 @@ Alias /static/hotspot.txt /srv/web/hotspot.txt {% endif %} +{% if site_name == "src.fedoraproject.org" %} + Include "conf.d/src.fedoraproject.org/expires.conf" +{% endif %} + {% if sslonly %} RewriteEngine On RewriteCond %{REQUEST_URI} !/.well-known/acme-challenge/.* @@ -85,4 +89,3 @@ Include "conf.d/{{ site_name }}/*.conf" </VirtualHost> {% endif %} - -- 2.20.1 -- Alessandro Lorenzi

3 2

[FBR] proxies: fix logic for websockets in reversepassproxy
by Tim Flink 29 Mar '19

29 Mar '19

The last patch I did for this added some bits that didn't need to be there and made a bad assumption about the default value for remotepath for the reversepassproxy.conf template. This ended up with unneccesarry complication in the ws balancers and a unintended RewriteCond for any declared reversepassproxy that didn't redefine remotepath. This patch fixes the bad assumptions and removes the cruft that didn't actually do or fix anything --- roles/httpd/reverseproxy/templates/reversepassproxy.conf | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/roles/httpd/reverseproxy/templates/reversepassproxy.conf b/roles/httpd/reverseproxy/templates/reversepassproxy.conf index 1e4afe0..1ea0b97 100644 --- a/roles/httpd/reverseproxy/templates/reversepassproxy.conf +++ b/roles/httpd/reverseproxy/templates/reversepassproxy.conf @@ -29,25 +29,17 @@ SSLProxyEngine On <Proxy "balancer://{{balancer_name}}-websocket"> {% for member in balancer_members %} {% if http_not_https_yes_this_is_insecure_and_i_feel_bad %} - {% if remotepath is defined %} - BalancerMember "ws://{{ member }}{{ remotepath }} - {% else %} BalancerMember "ws://{{ member }}" - {% endif %} - {% else %} - {% if remotepath is defined %} - BalancerMember "wss://{{ member }}{{ remotepath }} {% else %} BalancerMember "wss://{{ member }}" {% endif %} - {% endif %} {% endfor %} </Proxy> RewriteEngine on RewriteCond %{HTTP:Upgrade} ^WebSocket$ [NC] RewriteCond %{HTTP:Connection} Upgrade [NC] -{% if remotepath is defined %} +{% if remotepath != "/" %} RewriteCond %{REQUEST_URI} ^{{ remotepath }}/(.)* {% endif %} RewriteRule .* "balancer://{{ balancer_name }}-websocket%{REQUEST_URI}" [P] -- 1.8.3.1

2 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure March 2019