copr-devel June 2013

copr-devel@lists.fedorahosted.org

11 participants
60 discussions

Packaging COPR
by Miroslav Suchý 17 Jun '13

17 Jun '13

I started to package COPR. I put in git copr.spec. Right now it actually package just copr_cli (and I added man page there). Backend and frontend are empty placeholders. I will change that in short future. I use tito [1] for packaging. Here is just quick crash course for those who does not know tito: * tito is in fedora, so just "yum install tito" * "tito build --rpm" will create package from latest git tag * "tito build --srpm" will create source package from latest git tag * "tito build --test --rpm" will create package from latest commit. * "tito tag" will bump up version in spec file and create git tag. for more info "man tito" [1] https://github.com/dgoodwin/tito#about Once COPR will be packaged I plan to: * remove those *requirements.txt because they already differ from each other and from actual code. Requirements will be decribed by "Requires" inside of spec. * later either remove copr-setup.txt or simplify it to running "setup-frontend.sh" and "setup-backend.sh". -- Miroslav Suchy Red Hat Systems Management Engineering

2 1

[copr] tag 'copr-1.1-1' created
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git New tag : copr-1.1-1 Referencing: c4e6e4b6461ae5d5393aa2a73e3d906da56cccd0

1 0

[copr] master: Automatic commit of package [copr] release [1.1-1]. (1324fe1)
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 1324fe173b0897bbf1d108cd7ca180dd648143c6 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Mon Jun 17 14:45:58 2013 +0200 Automatic commit of package [copr] release [1.1-1]. >--------------------------------------------------------------- copr.spec | 5 ++++- rel-eng/packages/copr | 1 + 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/copr.spec b/copr.spec index 10d066f..90714e4 100644 --- a/copr.spec +++ b/copr.spec @@ -1,5 +1,5 @@ Name: copr -Version: 1.0 +Version: 1.1 Release: 1%{?dist} Summary: Cool Other Package Repo @@ -106,4 +106,7 @@ useradd -r -g copr -G apache -d %{_var}/lib/copr -s /bin/bash -c "COPR user" cop %doc %{_mandir}/man1/copr-cli.1* %changelog +* Mon Jun 17 2013 Miroslav Suchý <msuchy(a)redhat.com> 1.1-1 +- new package built with tito + diff --git a/rel-eng/packages/copr b/rel-eng/packages/copr new file mode 100644 index 0000000..b21002e --- /dev/null +++ b/rel-eng/packages/copr @@ -0,0 +1 @@ +1.1-1 ./

1 0

[copr] master: W0611: 11,0: Unused import sys (885cf98)
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 885cf98ac39779d81ff51489f295a359c2efe8f4 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Fri Jun 14 14:37:52 2013 +0200 W0611: 11,0: Unused import sys >--------------------------------------------------------------- copr_cli/subcommands.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/copr_cli/subcommands.py b/copr_cli/subcommands.py index 093d124..3c4da9d 100644 --- a/copr_cli/subcommands.py +++ b/copr_cli/subcommands.py @@ -8,7 +8,6 @@ output. import ConfigParser import json import os -import sys import requests

1 0

[copr] master: initial spec file (47ef195)
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 47ef1951667dc17224469bf3aad5815858617198 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Fri Jun 14 10:27:44 2013 +0200 initial spec file At least package copr-cli is functional. Backend and frontend will need some work. >--------------------------------------------------------------- copr.spec | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 97 insertions(+), 0 deletions(-) diff --git a/copr.spec b/copr.spec new file mode 100644 index 0000000..29a5efb --- /dev/null +++ b/copr.spec @@ -0,0 +1,97 @@ +Name: copr +Version: 1.0 +Release: 1%{?dist} +Summary: Cool Other Package Repo + +Group: Applications/Productivity +License: GPLv2+ +URL: https://fedorahosted.org/copr/ +Source0: %{name}-%{version}.tar.gz + +%description +COPR is lightway Koji. It allows you to create new project in WebUI, and +submit new builds and COPR will create yum repository from latests builds. + +%package backend +Summary: Backend for COPR +Requires: ansible >= 0.9 +Requires: lighttpd +Requires: euca2ools +Requires: rsync +Requires: openssh-clients +Requires: mock +Requires: yum-utils +Requires: createrepo +Requires: python-bunch +Requires: python-requests + +%description backend +COPR is lightway Koji. It allows you to create new project in WebUI, and +submit new builds and COPR will create yum repository from latests builds. + +This package contains backend. + +%package frontend +Summary: Frontend for COPR +Requires: httpd +Requires: mod_wsgi +Requires: postgresql-server +Requires: postgresql-devel +Requires: gcc +Requires: python-flask +Requires: python-flask-openid +Requires: python-flask-wtf +Requires: python-flask-sqlalchemy +#Requires: python-virtualenv +Requires: python-blinker +Requires: python-whoosh +# for tests: +Requires: pytest + +%description frontend +COPR is lightway Koji. It allows you to create new project in WebUI, and +submit new builds and COPR will create yum repository from latests builds. + +This package contains frontend. + +%package cli +Summary: Command line interface for COPR +Requires: python-requests +%if 0%{?rhel} < 7 && 0%{?rhel} > 0 +Requires: python-argparse +%endif + +%description cli +COPR is lightway Koji. It allows you to create new project in WebUI, and +submit new builds and COPR will create yum repository from latests builds. + +This package contains command line interface. + +%prep +%setup -q + + +%build +mv copr_cli/README.rst ./ + +%install +%{__python} coprcli-setup.py install --root %{buildroot} + +%pre backend +getent group copr >/dev/null || groupadd -r copr +getent passwd copr >/dev/null || \ +useradd -r -g copr -G apache -d %{_var}/lib/copr -s /bin/bash -c "COPR user" copr +/usr/bin/passwd -l copr >/dev/null + +%files backend +%doc LICENSE README + +%files frontend +%doc LICENSE + +%files cli +%doc LICENSE README.rst +%{_bindir}/copr-cli +%{python_sitelib}/* + +%changelog

1 0

[copr] master: W0612:121, 8:listcoprs: Unused variable 'max_width' (1e2d203)
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 1e2d2030e25bafe8ed32c512d36cc1e75f374820 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Fri Jun 14 14:36:31 2013 +0200 W0612:121,8:listcoprs: Unused variable 'max_width' >--------------------------------------------------------------- copr_cli/subcommands.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/copr_cli/subcommands.py b/copr_cli/subcommands.py index dc76dfb..093d124 100644 --- a/copr_cli/subcommands.py +++ b/copr_cli/subcommands.py @@ -118,7 +118,6 @@ def listcoprs(username=None): print "|".join(_list_to_row(row, widths)) else: - max_width = len(values[0]) headers = columns[0] print headers print "-"*len(headers)

1 0

[copr] master: Initialized to use tito. (daf46b2)
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit daf46b2d450eeea882e0d904a2f05f597e4301c6 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Mon Jun 10 15:08:49 2013 +0200 Initialized to use tito. >--------------------------------------------------------------- rel-eng/packages/.readme | 3 +++ rel-eng/tito.props | 5 +++++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/rel-eng/packages/.readme b/rel-eng/packages/.readme new file mode 100644 index 0000000..8999c8d --- /dev/null +++ b/rel-eng/packages/.readme @@ -0,0 +1,3 @@ +the rel-eng/packages directory contains metadata files +named after their packages. Each file has the latest tagged +version and the project's relative directory. diff --git a/rel-eng/tito.props b/rel-eng/tito.props new file mode 100644 index 0000000..fb74f3f --- /dev/null +++ b/rel-eng/tito.props @@ -0,0 +1,5 @@ +[globalconfig] +default_builder = tito.builder.Builder +default_tagger = tito.tagger.VersionTagger +changelog_do_not_remove_cherrypick = 0 +changelog_format = %s (%ae)

1 0

[copr] master: add man page for copr-cli (8819d78)
by Miroslav Suchý 17 Jun '13

17 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 8819d78d5d3f89b3d940365560c9e1c3eb90b588 Author: Miroslav Suchý <msuchy(a)redhat.com> Date: Fri Jun 14 15:51:07 2013 +0200 add man page for copr-cli >--------------------------------------------------------------- copr.spec | 12 ++++++ man/copr-cli.1.asciidoc | 89 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+), 0 deletions(-) diff --git a/copr.spec b/copr.spec index 29a5efb..10d066f 100644 --- a/copr.spec +++ b/copr.spec @@ -8,6 +8,10 @@ License: GPLv2+ URL: https://fedorahosted.org/copr/ Source0: %{name}-%{version}.tar.gz +BuildRequires: asciidoc +BuildRequires: libxslt +BuildRequires: util-linux + %description COPR is lightway Koji. It allows you to create new project in WebUI, and submit new builds and COPR will create yum repository from latests builds. @@ -74,9 +78,15 @@ This package contains command line interface. %build mv copr_cli/README.rst ./ +# convert manages +a2x -d manpage -f manpage man/copr-cli.1.asciidoc + %install %{__python} coprcli-setup.py install --root %{buildroot} +install -d %{buildroot}%{_mandir}/man1 +install -m 644 man/copr-cli.1 %{buildroot}/%{_mandir}/man1/ + %pre backend getent group copr >/dev/null || groupadd -r copr getent passwd copr >/dev/null || \ @@ -93,5 +103,7 @@ useradd -r -g copr -G apache -d %{_var}/lib/copr -s /bin/bash -c "COPR user" cop %doc LICENSE README.rst %{_bindir}/copr-cli %{python_sitelib}/* +%doc %{_mandir}/man1/copr-cli.1* %changelog + diff --git a/man/copr-cli.1.asciidoc b/man/copr-cli.1.asciidoc new file mode 100644 index 0000000..b04ccf6 --- /dev/null +++ b/man/copr-cli.1.asciidoc @@ -0,0 +1,89 @@ +copr-cli(1) +========== +:man source: copr +:man manual: COPR + +NAME +---- +copr-cli - command line interface for COPR + + +SYNOPSIS +-------- +copr-cli [-h] [--version] {list,create,build} ... + +DESCRIPTION +----------- + +This command allow you to interact with build system COPR from command line. + +OPTIONS +------- + +-h, --help:: +show this help message and exit + +--version:: +show program's version number and exit + +ACTIONS +------- + +list [username]:: +List all the copr of the provided username. If no username is provided, +then your coprs are listed. + +create:: +Create a new copr. + +build :: +Build packages to a specified copr. + + +`copr-cli create [options]` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +usage: copr-cli create [-h] [--chroot CHROOTS] [--repo REPOS] + [--initial-pkgs INITIAL_PKGS] + [--description DESCRIPTION] + [--instructions INSTRUCTIONS] + name + +--chroot:: +Chroot to use for this copr. Can be specified multiple times. + +--repo:: +Repository to add to this copr. Can be specified multiple times. + +--initial-pkgs:: +List of packages to build in this new copr. Can be specified multiple times. + +--description:: +Description of the copr. + +--instructions:: +Instructions for the copr. + + +`copr-cli build [options]` +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +usage: copr-cli build [-h] [--memory MEMORY] [--timeout TIMEOUT] + copr pkgs [pkgs ...] + +--memory:: +TBD + +--timeout:: +TBD + +copr:: +The copr repo to build the package in + +pkgs:: +Package to build. + + +AUTHORS +------- +Miroslav Suchý <msuchy(a)redhat.com>

1 0

Introduction
by Miroslav Suchý 16 Jun '13

16 Jun '13

Hi, my name is Miroslav Suchy (with short form Mirek) and after two weeks I will be working on COPR full time. Right now I'm reading history, code and documentation. I will have a lots of question and I hope I will shortly follow with lots of contributions. Mirek

3 2

[copr] master: Repo file generation + GUI link (47725ef)
by tradej＠fedoraproject.org 14 Jun '13

14 Jun '13

Repository : http://git.fedorahosted.org/cgit/copr.git On branch : master >--------------------------------------------------------------- commit 47725ef55cdcfea42c4966f6c1bfb43fd94a7c1f Author: Tomas Radej <tradej(a)redhat.com> Date: Thu May 16 16:45:10 2013 +0200 Repo file generation + GUI link >--------------------------------------------------------------- coprs_frontend/coprs/templates/coprs/copr.repo | 6 +++ .../coprs/templates/coprs/detail/overview.html | 4 ++ .../coprs/views/coprs_ns/coprs_general.py | 33 ++++++++++++++++++++ .../test_views/test_coprs_ns/test_coprs_general.py | 32 +++++++++++++++++++ 4 files changed, 75 insertions(+), 0 deletions(-) diff --git a/coprs_frontend/coprs/templates/coprs/copr.repo b/coprs_frontend/coprs/templates/coprs/copr.repo new file mode 100644 index 0000000..7833fd9 --- /dev/null +++ b/coprs_frontend/coprs/templates/coprs/copr.repo @@ -0,0 +1,6 @@ +[{{ copr.owner.name }}-{{ copr.name }}] + +name=Copr repo for {{ copr.name }} owned by {{ copr.owner.name }} +description={{ copr.description }} +baseurl={{ url }} +skip_if_unavailable=True diff --git a/coprs_frontend/coprs/templates/coprs/detail/overview.html b/coprs_frontend/coprs/templates/coprs/detail/overview.html index a038913..d924378 100644 --- a/coprs_frontend/coprs/templates/coprs/detail/overview.html +++ b/coprs_frontend/coprs/templates/coprs/detail/overview.html @@ -9,6 +9,10 @@ <div class="shift-right">{{ copr.description_or_not_filled }}</div> <h2>Installation Instructions</h2> <div class="shift-right">{{ copr.instructions_or_not_filled}}</div> + <h2>Yum repo</h2> + <div class="shift-right"> + <a href="{{ url_for('coprs_ns.generate_repo_file', reponame=copr.owner.name+'-'+copr.name) }}"> + {{ copr.owner.name }}-{{ copr.name }}.repo</a></div> <h2>Active Releases</h2> <table class="releases"> <tr> diff --git a/coprs_frontend/coprs/views/coprs_ns/coprs_general.py b/coprs_frontend/coprs/views/coprs_ns/coprs_general.py index 9d5be98..34f62a1 100644 --- a/coprs_frontend/coprs/views/coprs_ns/coprs_general.py +++ b/coprs_frontend/coprs/views/coprs_ns/coprs_general.py @@ -272,3 +272,36 @@ def copr_legal_flag(username, coprname): db.session.commit() flask.flash('Admin was noticed about your report and will investigate the copr shortly.') return flask.redirect(flask.url_for('coprs_ns.copr_detail', username=username, coprname=coprname)) + +(a)coprs_ns.route('/repo/<reponame>.repo') +def generate_repo_file(reponame): + ''' Generate repo file for a given repo name. + Reponame = username-coprname ''' + # This solution is used because flask splits off the last part after a + # dash, therefore user-re-po resolves to user-re/po instead of user/re-po + # FAS usernames may not contain dashes, so this construction is safe. + + if '-' not in reponame: + return page_not_found('Bad repository name: {0}. Must be username-coprname'.format(reponame)) + + username, coprname = reponame.split('-', 1) + copr = None + try: + # query.one() is used since it fetches all builds, unlike query.first(). + copr = coprs_logic.CoprsLogic.get(flask.g.user, username, coprname, + with_builds=True).one() + except sqlalchemy.orm.exc.NoResultFound: + return page_not_found('Copr {0}/{1} does not exist'.format(username, coprname)) + + url = '' + for build in copr.builds: + if build.results: + url = build.results + break + + if not url: + return page_not_found('Repository not initialized: No finished builds in {0}/{1}.'.format(username, coprname)) + + response = flask.make_response(flask.render_template('coprs/copr.repo', copr=copr, url=url)) + response.mimetype='text/plain' + return response diff --git a/coprs_frontend/tests/test_views/test_coprs_ns/test_coprs_general.py b/coprs_frontend/tests/test_views/test_coprs_ns/test_coprs_general.py index 380d59d..906ece1 100644 --- a/coprs_frontend/tests/test_views/test_coprs_ns/test_coprs_general.py +++ b/coprs_frontend/tests/test_views/test_coprs_ns/test_coprs_general.py @@ -1,4 +1,5 @@ import flask +import pytest from flexmock import flexmock @@ -456,3 +457,34 @@ class TestCoprDelete(CoprsTestCase): assert 'Copr was deleted successfully' not in r.data assert not self.models.Action.query.first() assert self.models.Copr.query.filter(self.models.Copr.id==self.c1.id).first() + +class TestCoprRepoGeneration(CoprsTestCase): + @pytest.fixture + def f_custom_builds(self): + ''' Custom builds are used in order not to break the default ones ''' + self.b5 = self.models.Build(copr=self.c1, user=self.u1, chroots='fedora-18-x86_64', submitted_on=9, ended_on=200, results='foo://bar.baz', status=1) + self.b6 = self.models.Build(copr=self.c1, user=self.u1, chroots='fedora-18-x86_64', submitted_on=11) + self.b7 = self.models.Build(copr=self.c1, user=self.u1, chroots='fedora-18-x86_64', submitted_on=10, ended_on=150, results='foo://bar.baz', status=1) + + self.db.session.add_all([self.b5, self.b6, self.b7]) + + def test_fail_on_missing_dash(self): + r = self.tc.get('/coprs/repo/reponamewithoutdash.repo') + assert r.status_code == 404 + assert 'Bad repository name' in r.data + + def test_fail_on_nonexistent_copr(self): + r = self.tc.get('/coprs/repo/bogus-nonexistent-repo.repo') + assert r.status_code == 404 + assert 'bogus/nonexistent-repo does not exist' in r.data + + def test_fail_on_no_finished_builds(self, f_users, f_coprs, f_db): + r = self.tc.get( + '/coprs/repo/{0}-{1}.repo'.format(self.u1.name, self.c1.name)) + assert r.status_code == 404 + assert 'Repository not initialized' in r.data + + def test_works_on_older_builds(self, f_users, f_coprs, f_custom_builds, f_db): + r = self.tc.get('/coprs/repo/{0}-{1}.repo'.format(self.u1.name, self.c1.name)) + assert r.status_code == 200 + assert 'baseurl=foo://bar.baz' in r.data

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

copr-devel June 2013