copr-devel June 2013

copr-devel@lists.fedorahosted.org

11 participants
60 discussions

Re: Network setup of builders
by Rex Dieter 26 Jun '13

26 Jun '13

On 06/26/2013 02:25 PM, Miroslav Suchy wrote: > On 06/26/2013 04:53 PM, seth vidal wrote: >> 1. remember that the user can add random repos to their mock configs in >> addition to the urls for their own srpms. Those repos can be from >> anywhere - not just from coprs. This is on purpose - so we can support >> a wide variety of systems and packages - w/ buildreqs from all over. > > Why we have such requirement? Per first paragraph of http://fedoraproject.org/wiki/Category:Copr "... to help make building and managing third party package repositories *easy*." (emphasis mine). Without this feature, copr's would lose a lot of what makes this easy. -- rex

1 0

Discussion: Using copr for building SCLs
by Bohuslav Kabrda 26 Jun '13

26 Jun '13

Hi list, I'd like to start yet another discussion about some functionality of copr, here it is: One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']). I'm trying to think of a way how to do this: - We would need to allow users to create their own chroots (or more precisely allow them to do certain modifications). - We would need to be able to send these chroots from frontend to backend. - We would need some mechanism for sane naming, so that we can distinguish between builds of collections for different releases and architectures. E.g. ruby193-x86_64 is not enough, we need to also store the information about os name (epel/fedora) and os version (epel 6, fedora 18, ...). My proposal: - Use chroot names like ruby193_fedora-18-x86_64. - Allow users to create these chroots by modifying config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name). - Store the modified information in the MockChroot table and create these on backend using communication channel proposed in [2] (parts of it already implemented in frontend). - Allow selecting these on copr creation/modification as more options for mock chroots, by default hidden under some kind of fancy clickable JS dropdown. Thoughts? -- Regards, Bohuslav "Slavek" Kabrda. [1] https://fedorahosted.org/SoftwareCollections/ [2] https://fedorahosted.org/copr/ticket/8

3 8

[copr] master: add python-flexmock to deps of frontend (2c30fb0)
by Miroslav Suchý 26 Jun '13

26 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 2c30fb0d407ea533f7a8921f90bbed051b7311bd Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Tue Jun 25 14:43:50 2013 +0200 add python-flexmock to deps of frontend addressing: _____________________________________ ERROR collecting tests/test_views/test_coprs_ns/test_coprs_general.py ______________________________________ tests/test_views/test_coprs_ns/test_coprs_general.py:4: in <module> > from flexmock import flexmock E ImportError: No module named flexmock >--------------------------------------------------------------- copr.spec | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/copr.spec b/copr.spec index 8ac71c8..19c40f4 100644 --- a/copr.spec +++ b/copr.spec @@ -58,6 +58,7 @@ Requires: python-psycopg2 Requires: python-whoosh # for tests: Requires: pytest +Requires: python-flexmock %description frontend COPR is lightway Koji. It allows you to create new project in WebUI, and

3 2

[copr] master: remove sentence which does not have meaning (3870790)
by Miroslav Suchý 25 Jun '13

25 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 38707907e468d7314ad27056934576e0d1dee5c0 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Tue Jun 25 13:56:55 2013 +0200 remove sentence which does not have meaning "you are you" does not have sense. I tried to rephrase it, but it is probably better without this sentece. I think we do not need to explain why token expires. >--------------------------------------------------------------- coprs_frontend/coprs/templates/api.html | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/coprs_frontend/coprs/templates/api.html b/coprs_frontend/coprs/templates/api.html index c57e49f..6501147 100644 --- a/coprs_frontend/coprs/templates/api.html +++ b/coprs_frontend/coprs/templates/api.html @@ -13,8 +13,7 @@ should not be shared!. - The API token is valid for {{ config['API_TOKEN_EXPIRATION'] }} days after you generated it, this - in order to ensure you are you every {{ config['API_TOKEN_EXPIRATION'] }} days. + The API token is valid for {{ config['API_TOKEN_EXPIRATION'] }} days after you generated it. {% if g.user %}

1 0

[copr] master: change api token expiration to 120 days and make it configurable (171dd27)
by Miroslav Suchý 25 Jun '13

25 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 171dd27b1dd78072766f3828975f78226de20049 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Tue Jun 25 13:50:59 2013 +0200 change api token expiration to 120 days and make it configurable >--------------------------------------------------------------- coprs_frontend/coprs/templates/api.html | 4 ++-- coprs_frontend/coprs/views/api_ns/api_general.py | 2 +- coprs_frontend/coprs/views/misc.py | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/coprs_frontend/coprs/templates/api.html b/coprs_frontend/coprs/templates/api.html index 0ea30b2..c57e49f 100644 --- a/coprs_frontend/coprs/templates/api.html +++ b/coprs_frontend/coprs/templates/api.html @@ -13,8 +13,8 @@ should not be shared!. - The API token is valid for 30 days after you generated it, this - in order to ensure you are you every 30 days. + The API token is valid for {{ config['API_TOKEN_EXPIRATION'] }} days after you generated it, this + in order to ensure you are you every {{ config['API_TOKEN_EXPIRATION'] }} days. {% if g.user %} diff --git a/coprs_frontend/coprs/views/api_ns/api_general.py b/coprs_frontend/coprs/views/api_ns/api_general.py index 6d90033..7130509 100644 --- a/coprs_frontend/coprs/views/api_ns/api_general.py +++ b/coprs_frontend/coprs/views/api_ns/api_general.py @@ -38,7 +38,7 @@ def api_new_token(): user.api_token = helpers.generate_api_token( flask.current_app.config['API_TOKEN_LENGTH']) user.api_token_expiration = datetime.date.today() \ - + datetime.timedelta(days=30) + + datetime.timedelta(days=flask.current_app.config['API_TOKEN_EXPIRATION']) db.session.add(user) db.session.commit() return flask.redirect(flask.url_for('api_ns.api_home')) diff --git a/coprs_frontend/coprs/views/misc.py b/coprs_frontend/coprs/views/misc.py index 7ffad6b..ea79640 100644 --- a/coprs_frontend/coprs/views/misc.py +++ b/coprs_frontend/coprs/views/misc.py @@ -56,7 +56,7 @@ def create_or_login(resp): models.User.openid_name == resp.identity_url).first() if not user: # create if not created already expiration_date_token = datetime.date.today() \ - + datetime.timedelta(days=30) + + datetime.timedelta(days=flask.current_app.config['API_TOKEN_EXPIRATION']) copr64 = base64.b64encode('copr') + '##' user = models.User(openid_name = resp.identity_url, mail = resp.email, api_login = copr64 + helpers.generate_api_token(

1 0

refactoring backend build spawning
by Seth Vidal 25 Jun '13

25 Jun '13

I've been noodling around with how to allow for arbitrary things to be run prior to the actual building of the pkgs. This is to allow for a wide range of 'sources' of pkgs. I talked with Slavek at summit briefly and we talked about a way to handle the giturl-of-doom or turning various arbitrary blobs into srpms. The gist of it is this - we have a place we can do it in the code we have but we will need to change how builds are being spawned. Right now we just a direct ansible call to execute mockchain and get the srpms building. This is simple and immediate but it means we have to modify code every time we want to add another step. I'm thinking we use a playbook instead of a direct call. This helps us b/c it means the playbook is easier to modify and we can get the log results out of its objects. for anyone not familiar with ansible playbooks see here: http://www.ansibleworks.com/docs/playbooks.html Here's is the framework I was thinking of: playbook: play - setup/create builder - provision it - add mock configs, etc play - verify state of the builder play - download srpms/pkgs/giturls/etc - make into srpms (optionally) - check integrity of srpms - check licenses - essentially run the rpmlint license checker - attempt buildorder sort? (optional but could be handy for a number of cases - this is using this code: http://skvidal.fedorapeople.org/misc/buildorder/ - mockchain build srpms - async (perhaps to make it easier to timeout - harder to retrieve logs) play - download results play - destroy builder This playbook would let us modify the playbook for the type of system (for example the mechanism to spin up an arm-builder is very very different from an x86[_64] vm. The way I figure this would allow us to insert a set of random tasks into the entire build process w/o having to add more code - just add more plays into the playbook. thoughts? -sv

2 2

Access to copr-be.cloud.fedoraproject.org
by Miroslav Suchý 24 Jun '13

24 Jun '13

Can somebody give me access to copr-be.cloud.fedoraproject.org please? My public ssh key is already in copr-fe.cloud.fedoraproject.org so you can just copy it from there. Thanks in advance. -- Miroslav Suchy Red Hat Systems Management Engineering

2 1

[copr] master: create_chroot must be run as copr-fe user (76a1199)
by Miroslav Suchý 24 Jun '13

24 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 76a1199e18746b4239f571fe71415585f0d07529 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Mon Jun 24 16:47:03 2013 +0200 create_chroot must be run as copr-fe user addressing: mod_wsgi (pid=1279): Exception occurred processing WSGI script '/usr/share/copr/coprs_frontend/application'. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__ return self.wsgi_app(environ, start_response) File "/usr/lib/python2.7/site-packages/flask/app.py", line 1689, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/usr/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/share/copr/coprs_frontend/coprs/views/misc.py", line 120, in decorated_function return f(*args, **kwargs) File "/usr/share/copr/coprs_frontend/coprs/views/coprs_ns/coprs_general.py", line 89, in copr_new db.session.commit() File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 149, in do return getattr(self.registry(), name)(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 720, in commit self.transaction.commit() File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 360, in commit self.session.dispatch.after_commit(self.session) File "/usr/lib64/python2.7/site-packages/sqlalchemy/event.py", line 409, in __call__ fn(*args, **kw) File "/usr/lib/python2.7/site-packages/flask_sqlalchemy.py", line 185, in after_commit models_committed.send(session.app, changes=d.values()) File "/usr/lib/python2.7/site-packages/blinker/base.py", line 220, in send for receiver in self.receivers_for(sender)] File "/usr/lib/python2.7/site-packages/flask_whooshee.py", line 174, in on_commit writer = wh.index.writer() File "/usr/lib/python2.7/site-packages/whoosh/filedb/fileindex.py", line 258, in writer return SegmentWriter(self, **kwargs) File "/usr/lib/python2.7/site-packages/whoosh/filedb/filewriting.py", line 136, in __init__ delay=delay): File "/usr/lib/python2.7/site-packages/whoosh/support/filelock.py", line 54, in try_for v = fn() File "/usr/lib/python2.7/site-packages/whoosh/support/filelock.py", line 98, in acquire self.fd = os.open(self.filename, flags) OSError: [Errno 13] Permission denied: '/usr/share/copr/data/whooshee/copr_user_whoosheer/MAIN_WRITELOCK' >--------------------------------------------------------------- copr-setup.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/copr-setup.txt b/copr-setup.txt index 1e12c30..f5a7ad6 100644 --- a/copr-setup.txt +++ b/copr-setup.txt @@ -83,7 +83,7 @@ COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_db --alembic alembic.ini - setenforce 0 (sorry, putting on TODO) - Add chroots. E.g: -COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_chroot fedora-18-x86_64 fedora-18-i386 + su - copr-fe -c 'OPRS_ENVIRON_PRODUCTION=1 ./manage.py create_chroot fedora-18-x86_64 fedora-18-i386' DB config

1 0

[copr] master: add note that you have to add chroots to db (cca7c7c)
by Miroslav Suchý 24 Jun '13

24 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit cca7c7ce8609a1e3606dfe07f615b944f9d2843a Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Mon Jun 24 16:23:32 2013 +0200 add note that you have to add chroots to db >--------------------------------------------------------------- copr-setup.txt | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/copr-setup.txt b/copr-setup.txt index a3dd962..1e12c30 100644 --- a/copr-setup.txt +++ b/copr-setup.txt @@ -82,6 +82,10 @@ COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_db --alembic alembic.ini - setenforce 0 (sorry, putting on TODO) +- Add chroots. E.g: +COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_chroot fedora-18-x86_64 fedora-18-i386 + + DB config =========

1 0

[copr] master: mark config.py as config so it is not overwritten during upgrade (3232414)
by Miroslav Suchý 24 Jun '13

24 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 3232414e6066a62e0c4a9f6f2713dc45a84783a3 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Mon Jun 24 16:18:09 2013 +0200 mark config.py as config so it is not overwritten during upgrade >--------------------------------------------------------------- copr.spec | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/copr.spec b/copr.spec index dd80c51..8ac71c8 100644 --- a/copr.spec +++ b/copr.spec @@ -136,6 +136,7 @@ useradd -r -g copr-fe -G copr-fe -d %{_datadir}/copr/coprs_frontend -s /bin/bash %{_datadir}/copr/coprs_frontend %ghost %{_datadir}/copr/data/copr.db +%config(noreplace)%{_datadir}/copr/coprs_frontend/coprs/config.py %files cli %doc LICENSE README.rst

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

copr-devel June 2013