copr-devel June 2020

copr-devel@lists.fedorahosted.org

9 participants
15 discussions

preloading git needed for COPR builds using forgemeta/scm?
by PGNet Dev 01 Jul '20

01 Jul '20

I'm building a pkg on COPR. It uses forgemeta/scm (from git) for a source. On build exec, it fails sh: git: command not found sh: git: command not found sh: git: command not found sh: git: command not found ... ( details -> https://pastebin.com/vUCVVHQq , line #1806 ) Note, the spec _does_ have "BuildRequires: git". It appears this^ may have to do with forgemeta needing git earlier? Note that foregemeta source pulls, subsequent to the error, fail ... Does COPR need an earlier preload of 'git' for forgemeta SCM use by builds? If that _is_ the case, where/how does one add a 'early' git req't for COPR builds?

2 6

Fwd: %forgemeta support for `git` tasks in checked-out code?
by Pavel Raiskup 27 Jun '20

27 Jun '20

Just forwarding this question to more appropriate list. ---------- Forwarded Message ---------- Subject: %forgemeta support for `git` tasks in checked-out code? Date: Thursday, June 25, 2020, 6:42:42 PM CEST From: PGNet Dev <pgnet.dev(a)gmail.com> To: copr-devel(a)lists.fedorahosted.org I'm building a package using multiple sources. I'm using $forgemeta to get/manage sources, as per example here: https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/#_multi… One of my sources pulls from upstream scm (github), and needs a subsequent 'git submodule--init'. Is seems that what %forgemeta pulls/tars/expands is not initilialize git repo; rather, just the relevant target's files. E.g., for a given scm source (here, ngx_brotli), the %forgemeta pull unpacks tree -L 1 -a ngx_brotli ngx_brotli ├── config ├── CONTRIBUTING.md ├── deps ├── filter ├── .gitmodules ├── LICENSE ├── README.md ├── script ├── static └── .travis.yml note, no ".git" folder, and no ability as is to `git submodule --init` in the tree. otoh, a `git clone` of the same source results, as expected, in tree -L 1 -a ngx_brotli ngx_brotli ├── config ├── CONTRIBUTING.md ├── deps ├── filter !! ├── .git ├── .gitmodules ├── LICENSE ├── README.md ├── script ├── static └── .travis.yml currently, to workaround this, I DISABLE the forge for the source in question, and do the pull & submodule init in %prep, e.g., this spec %global forgeurl0 https://github.com/nginx/nginx Version: 1.19.0 %global tag0 release-%{version} %global forgeurl1 https://github.com/openresty/headers-more-nginx-module %global tag1 master # DISABLE global forgeurl1 https://github.com/google/ngx_brotli %global tag2 master %forgemeta -i -z 0 %forgemeta -i -z 1 # DISABLE forgemeta -i -z 2 Name: %{name} %define dist %{scm0} Release: %{dist} Summary: %{name} License: BSD URL: %{forgeurl0} Source0: %{forgesource0} Source1: %{forgesource1} Source2: https://github.com/google/ngx_brotli/archive/master/ngx_brotli-master.tar.gz ... %prep pushd . cd %{_sourcedir} rm -rf ngx_brotli git clone https://github.com/google/ngx_brotli cd ngx_brotli git submodule update --init cd .. tar --remove-files -czf ngx_brotli.tar.gz ngx_brotli popd %setup -T -b 0 -n nginx-%{tag0} %setup -T -b 1 -n headers-more-nginx-module-%{tag1} ... %build export DESTDIR=%{buildroot} cd %{_builddir}/%{name}-%{tag0} ... %install cd %{_builddir}/%{name}-%{tag0} make install DESTDIR=%{buildroot} INSTALLDIRS=vendor ... appears to work, and the `git submodule --init` is taken care of correctly. but, it's messy. _is_ there a %forgemeta alternative to the manual git clone that will init/unpack the source _as_ a repo, allowing the submodule task to be exec'd? this isn't, of course, ONLY a COPR question, but one of rpmbuild more generally. _my_ goal _is_ to get it working on COPR, eventually. that said, if this^ belongs on some other list, let me know. _______________________________________________ copr-devel mailing list -- copr-devel(a)lists.fedorahosted.org To unsubscribe send an email to copr-devel-leave(a)lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/copr-devel@lists.fedorahosted.… -----------------------------------------

3 4

Re: %forgemeta scm retrieval & expansion on COPR fails to populate tarball. also appears to select unspecified/wrong chroot version ?
by PGNet Dev 26 Jun '20

26 Jun '20

On 6/25/20 10:08 AM, Pavel Raiskup wrote: >> i'll change it in my spec in any case. > > Good, just let me know it is OK. with this spec https://pastebin.com/raw/UG1ZGZGG which atm includes, %define _disable_source_fetch 0 local rpmbuild is OK, rpmbuild --verbose -ba nginx.spec tree -L 2 ../*RPMS ../RPMS └── x86_64 ├── nginx-1.19.0.fc32.x86_64.rpm ├── nginx-debuginfo-1.19.0.fc32.x86_64.rpm └── nginx-debugsource-1.19.0.fc32.x86_64.rpm ../SRPMS └── nginx-1.19.0-git.src.rpm as is local mock build, mock \ -r /etc/mock/fedora-32-x86_64.cfg \ --rebuild ../SRPMS/nginx-1.19.0-git.src.rpm ... INFO: Results and/or logs in: /var/lib/mock/fedora-32-x86_64/result ls -al /var/lib/mock/fedora-32-x86_64/root/builddir/build/SRPMS/nginx-1.19.0-git.src.rpm -rw-r--r-- 1 root mock 2.5M Jun 25 13:51 /var/lib/mock/fedora-32-x86_64/root/builddir/build/SRPMS/nginx-1.19.0-git.src.rpm pushing the spec to COPR, copr-cli build nginx-mainline \ /root/rpmbuild/SPECS/nginx.spec now "succeeds", https://copr.fedorainfracloud.org/coprs/pgfed/nginx-mainline/build/1504166/ and shows available via dnf dnf info nginx ... Name : nginx Version : 1.19.0 Release : git Architecture : src Size : 2.4 M Source : None Repository : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline Summary : nginx URL : https://github.com/nginx/nginx License : BSD Description : Nginx WebServer but note the "Architecture : src" which doesn't look right. and, attempt to install dnf install nginx Last metadata expiration check: 0:03:25 ago on Thu 25 Jun 2020 02:26:36 PM PDT. Dependencies resolved. ============================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================= Installing: nginx x86_64 1:1.18.0-1.fc32 updates 571 k Installing dependencies: fedora-logos-httpd noarch 30.0.2-4.fc32 fedora 16 k gperftools-libs x86_64 2.7-7.fc32 fedora 307 k nginx-mimetypes noarch 2.1.48-7.fc32 fedora 19 k Downgrading: nginx-filesystem noarch 1:1.18.0-1.fc32 updates 10 k Transaction Summary ============================================================================================================================= Install 4 Packages Downgrade 1 Package Total download size: 923 k Is this ok [y/N]: _still_ tries to install the 1.18 version nginx from 'updates' again, build & install _locally_ has no issues with either rpmbuild or mock build.

1 1

Re: %forgemeta scm retrieval & expansion on COPR fails to populate tarball. also appears to select unspecified/wrong chroot version ?
by PGNet Dev 25 Jun '20

25 Jun '20

On 6/25/20 10:08 AM, Pavel Raiskup wrote: >> breaks -- specifically on COPR? >> _either_ form seems to be ok with rpmbuild -- locally. > > Good question. I don't think so, but I didn't get to more deep observation yet. > > Have you tried to put `%define _disable_source_fetch 0` into your ~/.rpmmacros, > and then put %undefine into the spec, and is it still working? (copr isn't > doing anything else). nope ... I can give it a whirl. tho' not clear on what that tells me, or why it might be a 'solution' >> i'll change it in my spec in any case. > > Good, just let me know it is OK. k >> iiuc then, this needs to be fixed in online COPR before this'll work? > > We need to fix copr so it works with the %undefine variant, yes. At least if > it is valid configuration. I did not check the semantics of the macros across > distro versions, but what makes you think that "undefining" the macro is > equivalent to allow "download random spec-requested blobs from the internet"? initially, while scrounging around trying to _not_ have to manually use spectool 1st, these: https://stackoverflow.com/questions/33177450/how-do-i-get-rpmbuild-to-downl… https://jmahler.github.io/linux/2018/06/28/fedora_package_development.html https://gitlab.cern.ch/eos/eos-folly/commit/e213d77fa90019b17d5d56830156302… not saying those are right/reliable, just frequent -- and what was found. i've been working with Fedora for a whopping 3-4 days now; a bit challenging at 1st to find thorough docs / examples on this^ and of course, the 'migration' isn't, atm, helping get at useful info ...

2 2

%forgemeta support for `git` tasks in checked-out code?
by PGNet Dev 25 Jun '20

25 Jun '20

I'm building a package using multiple sources. I'm using $forgemeta to get/manage sources, as per example here: https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/#_multi… One of my sources pulls from upstream scm (github), and needs a subsequent 'git submodule--init'. Is seems that what %forgemeta pulls/tars/expands is not initilialize git repo; rather, just the relevant target's files. E.g., for a given scm source (here, ngx_brotli), the %forgemeta pull unpacks tree -L 1 -a ngx_brotli ngx_brotli ├── config ├── CONTRIBUTING.md ├── deps ├── filter ├── .gitmodules ├── LICENSE ├── README.md ├── script ├── static └── .travis.yml note, no ".git" folder, and no ability as is to `git submodule --init` in the tree. otoh, a `git clone` of the same source results, as expected, in tree -L 1 -a ngx_brotli ngx_brotli ├── config ├── CONTRIBUTING.md ├── deps ├── filter !! ├── .git ├── .gitmodules ├── LICENSE ├── README.md ├── script ├── static └── .travis.yml currently, to workaround this, I DISABLE the forge for the source in question, and do the pull & submodule init in %prep, e.g., this spec %global forgeurl0 https://github.com/nginx/nginx Version: 1.19.0 %global tag0 release-%{version} %global forgeurl1 https://github.com/openresty/headers-more-nginx-module %global tag1 master # DISABLE global forgeurl1 https://github.com/google/ngx_brotli %global tag2 master %forgemeta -i -z 0 %forgemeta -i -z 1 # DISABLE forgemeta -i -z 2 Name: %{name} %define dist %{scm0} Release: %{dist} Summary: %{name} License: BSD URL: %{forgeurl0} Source0: %{forgesource0} Source1: %{forgesource1} Source2: https://github.com/google/ngx_brotli/archive/master/ngx_brotli-master.tar.gz ... %prep pushd . cd %{_sourcedir} rm -rf ngx_brotli git clone https://github.com/google/ngx_brotli cd ngx_brotli git submodule update --init cd .. tar --remove-files -czf ngx_brotli.tar.gz ngx_brotli popd %setup -T -b 0 -n nginx-%{tag0} %setup -T -b 1 -n headers-more-nginx-module-%{tag1} ... %build export DESTDIR=%{buildroot} cd %{_builddir}/%{name}-%{tag0} ... %install cd %{_builddir}/%{name}-%{tag0} make install DESTDIR=%{buildroot} INSTALLDIRS=vendor ... appears to work, and the `git submodule --init` is taken care of correctly. but, it's messy. _is_ there a %forgemeta alternative to the manual git clone that will init/unpack the source _as_ a repo, allowing the submodule task to be exec'd? this isn't, of course, ONLY a COPR question, but one of rpmbuild more generally. _my_ goal _is_ to get it working on COPR, eventually. that said, if this^ belongs on some other list, let me know.

1 0

Re: %forgemeta scm retrieval & expansion on COPR fails to populate tarball. also appears to select unspecified/wrong chroot version ?
by PGNet Dev 25 Jun '20

25 Jun '20

On 6/25/20 9:22 AM, PGNet Dev wrote: > On 6/24/20 10:58 PM, Pavel Raiskup wrote: >> Yes, but you need to do `%define _disable_source_fetch 0` instead. >> Undefining the macro for some reason breaks rpmbuild. > > breaks -- specifically on COPR? > > _either_ form seems to be ok with rpmbuild -- locally. > > i'll change it in my spec in any case. > >> Not even to me. I had to look at the code to see what is happening: >> 1. we create rpmmacros file with some macros: >> https://pagure.io/copr/copr/blob/58af9ef7cf4b4eccd829b17a2de2afd043b95ce0/f… >> >> 2. then we do >> https://pagure.io/copr/copr/blob/58af9ef7cf4b4eccd829b17a2de2afd043b95ce0/f… > > thx for the code links! > >> First, this is wrong. We shouldn't use 'rpkg' on host for this task, >> perhaps we should just use 'mock --buildsrpm --enable-network' here. > > ah. > > iiuc then, this needs to be fixed in online COPR before this'll work? > > >

1 0

Chromium build timeout again
by 藍挺瑋 25 Jun '20

25 Jun '20

It looks like both GCC 10 and Chromium 83 increase the time required to build Chromium. I know the build time limit was just raised to 24 hours a few months ago, but recent releases of Chromium 83 started to exceed the limit on Fedora 32 and rawhide: https://copr.fedorainfracloud.org/coprs/lantw44/chromium/build/1404528/ https://copr.fedorainfracloud.org/coprs/lantw44/chromium/build/1406157/ https://copr.fedorainfracloud.org/coprs/lantw44/chromium/build/1424921/ https://copr.fedorainfracloud.org/coprs/lantw44/chromium/build/1427639/ Do you think we can raise the limit to 30 hours this time? Currently all successful builds of Chromium 83 with GCC 10 take 23 hours, which is close to the time limit.

2 3

%forgemeta scm retrieval & expansion on COPR fails to populate tarball. also appears to select unspecified/wrong chroot version ?
by PGNet Dev 25 Jun '20

25 Jun '20

hi, I've got a local .spec file (working on an nginx build) that's using %forgemeta macros with '%undefine _disable_source_fetch'. Local rpmbuild & mock builds for fedora-32-x86_64 work as expected -- sources are auto-pulled from upstream scm as intended. The created RPMs seem healthy -- from both rpmbuild & mock builds -- & they're locally installed without error, and the apps launch/exec. On submit of the known-to-work-locally .spec file to an my COPR account, copr-cli build <proj> <spec> the build quickly fails, not finding the retrieved/tar.gz'd source: "No such file or directory". _Does_ automated source_fetch function in COPR using '%undefine _disable_source_fetch' in the spec? Or is there different/additional config needed? Looking at the failed-build log, https://download.copr.fedorainfracloud.org/results/pgfed/nginx-mainline/srp… the build dies at stderr: error: Bad source: /tmp/copr-rpmbuild-s7uo_x65/nginx-release-1.19.0.tar.gz: No such file or directory Failed to execute command. Again -- works without error on local builds with the same .spec file. I notice the %forgemeta %dist tag is set as dist: .gitrelease.1.19.0.fc31 -- note the "fc31", even though the repo settings at https://copr.fedorainfracloud.org/coprs/pgfed/nginx-mainline/edit/ explicitly set/select the fc32 fedora-32-x86_64 chroot. Is %forgemeta source retrieval and subsequent tarball creation & population into BUILDROOT on COPR known to work? If so, I'm obviously missing _something_. Any suggestions/hints would be appreciated; the "No such file or directory" isn't particularly informative to me ...

2 1

Minor Copr release 2020-06-19
by Pavel Raiskup 20 Jun '20

20 Jun '20

Hey all, a new versions of copr packages were deployed now to production. There are several small bugfixes (most of them were already hot-fix patched in production before), but there are also two major things worth mentioning: 1. The build task scheduler (priority counting) was fixed once more, now for real, I believe. When you submit a background job, it _will_ be processed later than your normal builds (or at the same time, if there's capacity, there are no synchronization mechanisms, yet). Also source builds have precedence over binary RPM builds again (see issues #1414 and #1415 for more info). 2. Newly, we don't always run createrepo_c after each build (or delete request). Iff there are a concurrent createrepo_c requests on the same directory (multiple builds finished at a similar time in the same project chroot) we will likely group them into a batch and execute the createrepo_c only once for the whole batch. This saves some resources in general, but the major speedup will be in large projects (where createrepo_c used to block us historically). Especially point 2 was not really trivial. In case you experience any bugs, please report it as usual (and as always, you can regenerate the repository manually from the web-UI when necessary). Also, today's Fedora outage [1] allowed me to fix one small problem (I got a good reproducer) in our VM allocation mechanism, so the Copr's build performance shouldn't degrade over time again as it happened several times during the past few days. [1] https://pagure.io/fedora-infrastructure/issue/9051 Happy building! Pavel

3 3

Unannounced outage or some issue with the new prioritization?
by Iñaki Ucar 19 Jun '20

19 Jun '20

Hi, What happened today between 6 and 8 h UTC? The building pace dropped flat. Except for 8 tasks that basically were running for hours, no other job was taken from pending. Regards, -- Iñaki Úcar

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

copr-devel June 2020