I brought up the issue of arch-specific Requires some months ago in response to problems I experienced with xulrunner. This issue has reared its head again with gvfs on F13:
https://bugzilla.redhat.com/show_bug.cgi?id=573183
I've put up this draft:
https://fedoraproject.org/wiki/PackagingDrafts/ArchSpecificRequires
Comments welcome.
On Sat, 2010-03-13 at 15:04 -0500, Braden McDaniel wrote:
I brought up the issue of arch-specific Requires some months ago in response to problems I experienced with xulrunner. This issue has reared its head again with gvfs on F13:
https://bugzilla.redhat.com/show_bug.cgi?id=573183I've put up this draft:
https://fedoraproject.org/wiki/PackagingDrafts/ArchSpecificRequiresComments welcome.
Well, this generated no comments. That, of course, must mean that the proposal is perfect.
Meanwhile, I seem to have just experienced the sort of failure that this proposal seeks to prevent once again:
# yum install openldap-servers Loaded plugins: auto-update-debuginfo, langpacks, presto, refresh-packagekit Adding en_US to language list Found 131 installed debuginfo package(s) Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package openldap-servers.x86_64 0:2.4.21-4.fc13 set to be updated --> Processing Dependency: openldap = 2.4.21-4.fc13 for package: openldap-servers-2.4.21-4.fc13.x86_64 --> Running transaction check ---> Package openldap.i686 0:2.4.21-4.fc13 set to be updated --> Processing Dependency: libc.so.6(GLIBC_2.3.4) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.4) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.1) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libsasl2.so.2 for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.1.2) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.1.3) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libssl.so.10 for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6 for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libcrypto.so.10 for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libpthread.so.0(GLIBC_2.3.2) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libpthread.so.0(GLIBC_2.1) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libresolv.so.2(GLIBC_2.2) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.12) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.0) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libresolv.so.2 for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libpthread.so.0(GLIBC_2.0) for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libpthread.so.0 for package: openldap-2.4.21-4.fc13.i686 --> Processing Dependency: libc.so.6(GLIBC_2.3) for package: openldap-2.4.21-4.fc13.i686 --> Running transaction check ---> Package cyrus-sasl-lib.i686 0:2.1.23-11.fc13 set to be updated --> Processing Dependency: libdb-4.8.so for package: cyrus-sasl-lib-2.1.23-11.fc13.i686 ---> Package glibc.i686 0:2.12-2 set to be updated --> Processing Dependency: libfreebl3.so for package: glibc-2.12-2.i686 --> Processing Dependency: libfreebl3.so(NSSRAWHASH_3.12.3) for package: glibc-2.12-2.i686 ---> Package openssl.i686 0:1.0.0a-1.fc13 set to be updated --> Processing Dependency: libk5crypto.so.3(k5crypto_3_MIT) for package: openssl-1.0.0a-1.fc13.i686 --> Processing Dependency: libz.so.1 for package: openssl-1.0.0a-1.fc13.i686 --> Processing Dependency: libkrb5.so.3 for package: openssl-1.0.0a-1.fc13.i686 --> Processing Dependency: libgssapi_krb5.so.2 for package: openssl-1.0.0a-1.fc13.i686 --> Processing Dependency: libk5crypto.so.3 for package: openssl-1.0.0a-1.fc13.i686 --> Processing Dependency: libcom_err.so.2 for package: openssl-1.0.0a-1.fc13.i686 --> Processing Dependency: libkrb5.so.3(krb5_3_MIT) for package: openssl-1.0.0a-1.fc13.i686 --> Running transaction check ---> Package db4.i686 0:4.8.30-1.fc13 set to be updated ---> Package krb5-libs.i686 0:1.7.1-10.fc13 set to be updated --> Processing Dependency: libkeyutils.so.1(KEYUTILS_0.3) for package: krb5-libs-1.7.1-10.fc13.i686 --> Processing Dependency: libselinux.so.1 for package: krb5-libs-1.7.1-10.fc13.i686 --> Processing Dependency: libkeyutils.so.1 for package: krb5-libs-1.7.1-10.fc13.i686 ---> Package libcom_err.i686 0:1.41.10-6.fc13 set to be updated ---> Package nss-softokn-freebl.i686 0:3.12.4-19.fc13 set to be updated ---> Package zlib.i686 0:1.2.3-23.fc12 set to be updated --> Running transaction check ---> Package keyutils-libs.i686 0:1.2-6.fc12 set to be updated ---> Package libselinux.i686 0:2.0.90-5.fc13 set to be updated --> Finished Dependency Resolution
Dependencies Resolved
================================================================================ Package Arch Version Repository Size ================================================================================ Installing: openldap-servers x86_64 2.4.21-4.fc13 fedora 2.6 M Installing for dependencies: cyrus-sasl-lib i686 2.1.23-11.fc13 fedora 135 k db4 i686 4.8.30-1.fc13 updates 614 k glibc i686 2.12-2 updates 4.3 M keyutils-libs i686 1.2-6.fc12 fedora 18 k krb5-libs i686 1.7.1-10.fc13 updates 656 k libcom_err i686 1.41.10-6.fc13 fedora 34 k libselinux i686 2.0.90-5.fc13 fedora 104 k nss-softokn-freebl i686 3.12.4-19.fc13 updates 111 k openldap i686 2.4.21-4.fc13 fedora 232 k openssl i686 1.0.0a-1.fc13 updates 1.4 M zlib i686 1.2.3-23.fc12 fedora 69 k
Transaction Summary ================================================================================ Install 12 Package(s) Upgrade 0 Package(s)
Total download size: 10 M Installed size: 27 M Is this ok [y/N]:
So... What's the next step for moving this draft along?
On Fri, 2010-06-18 at 22:10 -0400, Braden McDaniel wrote:
On Sat, 2010-03-13 at 15:04 -0500, Braden McDaniel wrote:
I brought up the issue of arch-specific Requires some months ago in response to problems I experienced with xulrunner. This issue has reared its head again with gvfs on F13:
https://bugzilla.redhat.com/show_bug.cgi?id=573183I've put up this draft:
https://fedoraproject.org/wiki/PackagingDrafts/ArchSpecificRequiresComments welcome.
Well, this generated no comments. That, of course, must mean that the proposal is perfect.
I take that back. No comments here; but Matt McCutchen added comments to the wiki page (that I've now tried to address).
On Saturday 19 June 2010, Braden McDaniel wrote:
On Sat, 2010-03-13 at 15:04 -0500, Braden McDaniel wrote:
I've put up this draft: https://fedoraproject.org/wiki/PackagingDrafts/ArchSpecificRequir es
Comments welcome.
Well, this generated no comments. That, of course, must mean that the proposal is perfect.
1) "A non-noarch subpackage's dependency on its main package"
s/on its main package/on its main package or another subpackage/
2) https://fedoraproject.org/wiki/Packaging/Guidelines#Requiring_Base_Package I suggest tweaking the examples to include the (epoch and) version and release for consistency with the above.
3) https://fedoraproject.org/wiki/Packaging/NamingGuidelines#Renaming.2Freplaci... Because Provides are not ISA qualified by default, both ISA qualified and non- qualified Provides should be added where applicable and appropriate when replacing or splitting packages in order to not break dependencies. I suggest noting this both in this draft and the above NamingGuidelines entry.
On Mon, 2010-06-21 at 01:01 +0300, Ville Skyttä wrote:
On Saturday 19 June 2010, Braden McDaniel wrote:
On Sat, 2010-03-13 at 15:04 -0500, Braden McDaniel wrote:
I've put up this draft: https://fedoraproject.org/wiki/PackagingDrafts/ArchSpecificRequir es
Comments welcome.
Well, this generated no comments. That, of course, must mean that the proposal is perfect.
- "A non-noarch subpackage's dependency on its main package"
s/on its main package/on its main package or another subpackage/
Done.
I suggest tweaking the examples to include the (epoch and) version and release for consistency with the above.
https://fedoraproject.org/wiki/Packaging/NamingGuidelines#Renaming.2Freplaci... Because Provides are not ISA qualified by default, both ISA qualified and non- qualified Provides should be added where applicable and appropriate when replacing or splitting packages in order to not break dependencies. I suggest noting this both in this draft and the above NamingGuidelines entry.
I'm not very familiar with how these drafts are normally structured as they evolve; but at this point it strikes me as beneficial to reorganize the draft into sections noting its intent and mechanism, followed by descriptions of specific concrete changes to the Packaging Guidelines and the Naming Guidelines.
On Mon, 2010-06-21 at 01:01 +0300, Ville Skyttä wrote:
[snip]
I suggest tweaking the examples to include the (epoch and) version and release for consistency with the above.
I've added a section that articulates specific changes to the Guidelines, including this one.
https://fedoraproject.org/wiki/Packaging/NamingGuidelines#Renaming.2Freplaci... Because Provides are not ISA qualified by default, both ISA qualified and non- qualified Provides should be added where applicable and appropriate when replacing or splitting packages in order to not break dependencies. I suggest noting this both in this draft and the above NamingGuidelines entry.
I'm not sure this is the Right Thing in all cases. Certainly if a package's consumables can be used in a way that is not arch-specific, there should be a Provide that *is not* arch-specific. Similarly, there should be an arch-specific Provide only if the consumables can (also) be used in a way that *is* arch-specific.
But continuing to supply an arch-independent Provide just to avoid breaking dependencies is not appropriate, I think. Spec files that are using arch-independent Provides for things that get used in arch-specific ways are broken and need fixing (with respect to this proposal). While it may be true that (part of) the reason for this brokenness is that arch-specific Provides are not currently generally available, I think that just means that suppliers of such Provides and their consumers need to be fixed together.
On Tue, 2010-07-06 at 04:16 -0400, Braden McDaniel wrote:
On Mon, 2010-06-21 at 01:01 +0300, Ville Skyttä wrote:
[snip]
https://fedoraproject.org/wiki/Packaging/NamingGuidelines#Renaming.2Freplaci... Because Provides are not ISA qualified by default, both ISA qualified and non- qualified Provides should be added where applicable and appropriate when replacing or splitting packages in order to not break dependencies. I suggest noting this both in this draft and the above NamingGuidelines entry.
I'm not sure this is the Right Thing in all cases. Certainly if a package's consumables can be used in a way that is not arch-specific, there should be a Provide that *is not* arch-specific. Similarly, there should be an arch-specific Provide only if the consumables can (also) be used in a way that *is* arch-specific.
Actually... Wouldn't "Requires: foo" be satisfied by either "Provides: foo.i386" or "Provides: foo.x86_64"? Or am I mistaken about this?
On 07/06/2010 10:56 PM, Braden McDaniel wrote:
Actually... Wouldn't "Requires: foo" be satisfied by either "Provides: foo.i386" or "Provides: foo.x86_64"? Or am I mistaken about this?
If you have:
Requires: foo
In modern Fedora, packages generate autoprovides like this:
foo-1.2-3.i686.rpm ====================== foo = 1.2-3 foo(x86-32) = 1.2-3
foo-1.2-3.x86_64.rpm ====================== foo = 1.2-3 foo(x86-64) = 1.2-3
This means that either package will satisfy that Requires. If the package has:
Requires: foo%{?_isa}
At build time, %{?_isa} is evaluated to match the architecture and bitsize, so, on i386, it evaluates to:
Requires: foo(x86-32)
and on x86_64, it evaluates to:
Requires: foo(x86-64)
In this situation, only the foo package which exactly matches the architecture will have the matching provides, so there is no chance for a mismatch.
~spot
On 7/7/10 9:28 AM, Tom "spot" Callaway wrote:
On 07/06/2010 10:56 PM, Braden McDaniel wrote:
Actually... Wouldn't "Requires: foo" be satisfied by either "Provides: foo.i386" or "Provides: foo.x86_64"? Or am I mistaken about this?
If you have:
Requires: foo
In modern Fedora, packages generate autoprovides like this:
[snip]
Got it. Thanks.
On Sat, 2010-03-13 at 15:04 -0500, Braden McDaniel wrote:
I brought up the issue of arch-specific Requires some months ago in response to problems I experienced with xulrunner. This issue has reared its head again with gvfs on F13:
https://bugzilla.redhat.com/show_bug.cgi?id=573183I've put up this draft:
https://fedoraproject.org/wiki/PackagingDrafts/ArchSpecificRequiresComments welcome.
James Antill added this text re: when Requires should *not* be arch-specific:
A package has a Build-Requires on a specific arch. library (because rpmbuild evaluates the %{_isa} at .src.rpm buildtime, and not at .src.rpm => .arch.rpm build time).
I don't understand what this is saying. Can anyone put this in more lucid terms?
On 07/06/2010 01:50 AM, Braden McDaniel wrote:
James Antill added this text re: when Requires should *not* be arch-specific:
A package has a Build-Requires on a specific arch. library (because rpmbuild evaluates the %{_isa} at .src.rpm buildtime, and not at .src.rpm => .arch.rpm build time).I don't understand what this is saying. Can anyone put this in more lucid terms?
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
~spot
On Tue, 2010-07-06 at 11:40 -0400, Tom "spot" Callaway wrote:
On 07/06/2010 01:50 AM, Braden McDaniel wrote:
James Antill added this text re: when Requires should *not* be arch-specific:
A package has a Build-Requires on a specific arch. library (because rpmbuild evaluates the %{_isa} at .src.rpm buildtime, and not at .src.rpm => .arch.rpm build time).I don't understand what this is saying. Can anyone put this in more lucid terms?
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
Okay... So if I now understand correctly, this doesn't involve Requires at all.
I was never trying to address BuildRequires in this draft; but if folks feel like the addition of the ${_isa} stuff for Requires warrants some accompanying "don't do this for BuildRequires" prose, I'll add that.
On Tue, 06 Jul 2010 11:52:42 -0400, Braden wrote:
On Tue, 2010-07-06 at 11:40 -0400, Tom "spot" Callaway wrote:
On 07/06/2010 01:50 AM, Braden McDaniel wrote:
James Antill added this text re: when Requires should *not* be arch-specific:
A package has a Build-Requires on a specific arch. library (because rpmbuild evaluates the %{_isa} at .src.rpm buildtime, and not at .src.rpm => .arch.rpm build time).I don't understand what this is saying. Can anyone put this in more lucid terms?
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
Okay... So if I now understand correctly, this doesn't involve Requires at all.
The spec file's BuildRequires become the src.rpm's Requires.
On 07/06/2010 11:52 AM, Braden McDaniel wrote:
Okay... So if I now understand correctly, this doesn't involve Requires at all.
I was never trying to address BuildRequires in this draft; but if folks feel like the addition of the ${_isa} stuff for Requires warrants some accompanying "don't do this for BuildRequires" prose, I'll add that.
Yeah, I think its worth calling out, given that it will break in most practical koji uses.
~spot
tis 2010-07-06 klockan 11:40 -0400 skrev Tom "spot" Callaway:
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
~spot
This is not how it works.
rpmbuild re-evaluates the BuildRequires from the specfile inside the SRPM, it does not use the SRPM's Requires. Having %{?_isa} in BuildRequires tags is perfectly legal - and advisable to have if you want your SRPMs to be buildable in a multiarch environment.
Mattias
On 07/07/2010 02:53 AM, Mattias Ellert wrote:
tis 2010-07-06 klockan 11:40 -0400 skrev Tom "spot" Callaway:
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
~spot
This is not how it works.
rpmbuild re-evaluates the BuildRequires from the specfile inside the SRPM, it does not use the SRPM's Requires. Having %{?_isa} in BuildRequires tags is perfectly legal - and advisable to have if you want your SRPMs to be buildable in a multiarch environment.
Sorry, my bad, you're correct. rpmbuild does re-evaluate it, perhaps yum does not when koji tries to populate the buildroot?
~spot
On Wed, 2010-07-07 at 09:34 -0400, Tom "spot" Callaway wrote:
On 07/07/2010 02:53 AM, Mattias Ellert wrote:
tis 2010-07-06 klockan 11:40 -0400 skrev Tom "spot" Callaway:
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
~spot
This is not how it works.
rpmbuild re-evaluates the BuildRequires from the specfile inside the SRPM, it does not use the SRPM's Requires. Having %{?_isa} in BuildRequires tags is perfectly legal - and advisable to have if you want your SRPMs to be buildable in a multiarch environment.
Sorry, my bad, you're correct. rpmbuild does re-evaluate it, perhaps yum does not when koji tries to populate the buildroot?
I don't think yum can. It doesn't have any expansion capability for rpm macros.
-sv
On Wed, 2010-07-07 at 09:39 -0400, seth vidal wrote:
Sorry, my bad, you're correct. rpmbuild does re-evaluate it, perhaps yum does not when koji tries to populate the buildroot?
I don't think yum can. It doesn't have any expansion capability for rpm macros.
Just did some tests on this:
In a spec file I added: BuildRequires: foobar%{?_isa}
I rebuilt the srpm and I get:
rpm -qpR file.src.rpm foobar(x86-32) rpmlib(CompressedFileNames) <= 3.0.4-1
I opened the rpm using the rpm-python interface (what yum uses internally): hdr['requires'] ['foobar(x86-32)', 'rpmlib(CompressedFileNames)']
So are you suggesting that yum should look for foobar(somearchspec) and reinterpret those based on what the local system is?
-sv
On 07/07/2010 09:49 AM, seth vidal wrote:
On Wed, 2010-07-07 at 09:39 -0400, seth vidal wrote:
Sorry, my bad, you're correct. rpmbuild does re-evaluate it, perhaps yum does not when koji tries to populate the buildroot?
I don't think yum can. It doesn't have any expansion capability for rpm macros.
Just did some tests on this:
In a spec file I added: BuildRequires: foobar%{?_isa}
I rebuilt the srpm and I get:
rpm -qpR file.src.rpm foobar(x86-32) rpmlib(CompressedFileNames) <= 3.0.4-1
I opened the rpm using the rpm-python interface (what yum uses internally): hdr['requires'] ['foobar(x86-32)', 'rpmlib(CompressedFileNames)']
So are you suggesting that yum should look for foobar(somearchspec) and reinterpret those based on what the local system is?
Well, to be fair, I'm just trying to understand where the concern is here.
I'm not sure it makes sense for yum to reinterpret SRPM Requires. I suspect there are some potential situations where a spec would want to hardcode:
BuildRequires: foo(x86-32)
And wouldn't want yum re-evaluating it.
~spot
On Wed, 2010-07-07 at 09:53 -0400, Tom "spot" Callaway wrote:
So are you suggesting that yum should look for foobar(somearchspec) and reinterpret those based on what the local system is?
Well, to be fair, I'm just trying to understand where the concern is here.
That's fine - I'm just trying to make sure i'm on the right page here, too.
I'm not sure it makes sense for yum to reinterpret SRPM Requires. I suspect there are some potential situations where a spec would want to hardcode:
BuildRequires: foo(x86-32)
And wouldn't want yum re-evaluating it.
Agreed.
-sv
On Wed, 2010-07-07 at 10:12 -0400, seth vidal wrote:
On Wed, 2010-07-07 at 09:53 -0400, Tom "spot" Callaway wrote:
So are you suggesting that yum should look for foobar(somearchspec) and reinterpret those based on what the local system is?
Well, to be fair, I'm just trying to understand where the concern is here.
That's fine - I'm just trying to make sure i'm on the right page here, too.
I'm not sure it makes sense for yum to reinterpret SRPM Requires. I suspect there are some potential situations where a spec would want to hardcode:
BuildRequires: foo(x86-32)
And wouldn't want yum re-evaluating it.
Agreed.
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
On Wed, 2010-07-07 at 21:50 -0400, Braden McDaniel wrote:
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
_isa in BuildRequires doesn't work atm. and shouldn't be used. There are possible fixes, but all of them are non-trivial.
On Thu, 8 Jul 2010, James Antill wrote:
On Wed, 2010-07-07 at 21:50 -0400, Braden McDaniel wrote:
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
_isa in BuildRequires doesn't work atm. and shouldn't be used. There are possible fixes, but all of them are non-trivial.
"Doesn't work" is, err, rather vague.
ISA in BuildRequires works just fine (buildsys and all). BUT using it in Fedora infrastructure breaks the SRPM repository & its users (like yum-builddep) which are built under the assumption SRPMs are arch-independent.
- Panu -
On 07/08/2010 06:59 AM, Panu Matilainen wrote:
On Thu, 8 Jul 2010, James Antill wrote:
On Wed, 2010-07-07 at 21:50 -0400, Braden McDaniel wrote:
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
_isa in BuildRequires doesn't work atm. and shouldn't be used. There are possible fixes, but all of them are non-trivial.
"Doesn't work" is, err, rather vague.
ISA in BuildRequires works just fine (buildsys and all). BUT using it in Fedora infrastructure breaks the SRPM repository& its users (like yum-builddep) which are built under the assumption SRPMs are arch-independent.
Explicit %_isa in any "*requires:" breaks updates when a package changes its architecture (noarch <-> "arch").
My recommendation is to not use "explicit %_isa" unless really, really necessary (i.e. almost never).
Ralf
On Thu, 2010-07-08 at 08:04 +0200, Ralf Corsepius wrote:
On 07/08/2010 06:59 AM, Panu Matilainen wrote:
On Thu, 8 Jul 2010, James Antill wrote:
On Wed, 2010-07-07 at 21:50 -0400, Braden McDaniel wrote:
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
_isa in BuildRequires doesn't work atm. and shouldn't be used. There are possible fixes, but all of them are non-trivial.
"Doesn't work" is, err, rather vague.
ISA in BuildRequires works just fine (buildsys and all). BUT using it in Fedora infrastructure breaks the SRPM repository& its users (like yum-builddep) which are built under the assumption SRPMs are arch-independent.
Explicit %_isa in any "*requires:" breaks updates when a package changes its architecture (noarch <-> "arch").
This strikes me as an uncommon (if not rare) occurrence. Nonetheless...
If your package changes from noarch -> arch, the potential to maintain compatibility with dependents (without a rebuild) is highly suspect. Regardless, this is a situation that must be coordinated with downstream packages.
Changing from arch -> noarch probably stands a better chance of not introducing breakage on its own; especially if the "arch"-ness was a mistake or happenstance (e.g., the introduction of noarch subpackages gave us some such transitions). In cases where "arch"-ness is a mistake or happenstance, the recommendations in this draft do not advise dependents to use arch-specific Requires.
If there were a real transition of a *library or -devel* package from being arch-specific to arch-independent, once again I am very doubtful that compatibility with dependents could be maintained.
On Thu, 2010-07-08 at 07:59 +0300, Panu Matilainen wrote:
On Thu, 8 Jul 2010, James Antill wrote:
On Wed, 2010-07-07 at 21:50 -0400, Braden McDaniel wrote:
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
_isa in BuildRequires doesn't work atm. and shouldn't be used. There are possible fixes, but all of them are non-trivial.
"Doesn't work" is, err, rather vague.
ISA in BuildRequires works just fine (buildsys and all). BUT using it in Fedora infrastructure breaks the SRPM repository & its users (like yum-builddep) which are built under the assumption SRPMs are arch-independent.
If srpms are not arch independent then they need to stop being created as .src.rpm. I know that they have the arch of what platform they were built on stored in the arch tag internally. I also know that if the assumption that src.rpm's are not arch independent is now no longer accurate then a lot of tooling has to change.
I'm sure the following is what you'll say:
"well, we never made any promises about that and they do have an arch contained in them."
That's great - I'm glad we are getting into the legal details of what was or was not committed to and not, instead, focusing on what we're doing to what we need srpms to do.
If srpms are not arch-independent do you mean to say that we need to create separate srpms for each arch we build on?
-sv
On Thu, 8 Jul 2010, seth vidal wrote:
On Thu, 2010-07-08 at 07:59 +0300, Panu Matilainen wrote:
On Thu, 8 Jul 2010, James Antill wrote:
On Wed, 2010-07-07 at 21:50 -0400, Braden McDaniel wrote:
Well, with respect to what to do about a guideline for BuildRequires and %{?_isa}, I'm back to being confused.
Matthias' comment suggests to me that %{?_isa} should be recommended in BuildRequires for non-noarch packages; but the ensuing discussion makes me less certain of that. The result of this uncertainty is that I'm back to thinking that mention of BuildRequires should be dropped from this draft and its issues deferred to another one.
_isa in BuildRequires doesn't work atm. and shouldn't be used. There are possible fixes, but all of them are non-trivial.
"Doesn't work" is, err, rather vague.
ISA in BuildRequires works just fine (buildsys and all). BUT using it in Fedora infrastructure breaks the SRPM repository & its users (like yum-builddep) which are built under the assumption SRPMs are arch-independent.
If srpms are not arch independent then they need to stop being created as .src.rpm. I know that they have the arch of what platform they were built on stored in the arch tag internally. I also know that if the assumption that src.rpm's are not arch independent is now no longer accurate then a lot of tooling has to change.
I'm sure the following is what you'll say:
"well, we never made any promises about that and they do have an arch contained in them."
That's great - I'm glad we are getting into the legal details of what was or was not committed to and not, instead, focusing on what we're doing to what we need srpms to do.
SRPMS have never been arch-independent, using %{_isa} in BuildRequires just makes it much more visible.
There are myriads of ways for an SRPM to be arch-dependent, for example:
%ifarch ppc ppc64 BuildRequires: ppc-utils-devel %endif
%ifarch x86_64 Source0: foo-64bit.tar.gz %else Source1: foo-32bit.tar.gz %endif
I'm not aware of arch-conditional buildrequires being banned in Fedora (if they are, I stand corrected), so the first case can already create a SRPM with dependencies unresolvable outside ppc.
The arch-conditional source is crazy of course, but the spec syntax permits it.
If srpms are not arch-independent do you mean to say that we need to create separate srpms for each arch we build on?
Nobody, me included, wants that.
One option is to continue the current "it mostly works" way, and just ban the use of spec constructs that cause arch-dependent dependencies in SRPM.
Actually fixing the issue is going to require more dramatic actions though and changes to the tools. All sorts of possibilities do exist, for example:
1) Forget SRPMs, just stuff the sources, patches and spec from a given CVS tag into a %{name}-%{version}-%{release}.tar.gz tarball and distribute those instead of SRPMs. There you have a format that's guaranteed to be arch-independent all the way, and they're buildable too, just 'rpmbuild -tb <tarball>' and go.
2) Create metadata for the SRPMs per-arch, but only actually distribute one SRPM. Might involve things like splitting header and payload to separate files or something.
...or something.
If you ask me, I would go for 1). Seriously. If you want metadata to go with the tarballs, strip the header out of src.rpm's on each arch and build per-arch metadata only.
- Panu -
On Thu, 2010-07-08 at 10:20 +0300, Panu Matilainen wrote:
Actually fixing the issue is going to require more dramatic actions though and changes to the tools. All sorts of possibilities do exist, for example:
Forget SRPMs, just stuff the sources, patches and spec from a given CVS tag into a %{name}-%{version}-%{release}.tar.gz tarball and distribute those instead of SRPMs. There you have a format that's guaranteed to be arch-independent all the way, and they're buildable too, just 'rpmbuild -tb <tarball>' and go.
Create metadata for the SRPMs per-arch, but only actually distribute one SRPM. Might involve things like splitting header and payload to separate files or something.
...or something.
If you ask me, I would go for 1). Seriously. If you want metadata to go with the tarballs, strip the header out of src.rpm's on each arch and build per-arch metadata only.
Talking about 1 is probably worthwhile - hell - even just having the Source locations need to be full paths would make it worthwhile.
It might tie into some of the stuff that colin walters and I think jkeating eventually want to do pretty well, too.
definitely worth discussing.
-sv
On Thu, 2010-07-08 at 16:54 -0400, seth vidal wrote:
On Thu, 2010-07-08 at 10:20 +0300, Panu Matilainen wrote:
Actually fixing the issue is going to require more dramatic actions though and changes to the tools. All sorts of possibilities do exist, for example:
Forget SRPMs, just stuff the sources, patches and spec from a given CVS tag into a %{name}-%{version}-%{release}.tar.gz tarball and distribute those instead of SRPMs. There you have a format that's guaranteed to be arch-independent all the way, and they're buildable too, just 'rpmbuild -tb <tarball>' and go.
Create metadata for the SRPMs per-arch, but only actually distribute one SRPM. Might involve things like splitting header and payload to separate files or something.
...or something.
If you ask me, I would go for 1). Seriously. If you want metadata to go with the tarballs, strip the header out of src.rpm's on each arch and build per-arch metadata only.
Talking about 1 is probably worthwhile - hell - even just having the Source locations need to be full paths would make it worthwhile.
It might tie into some of the stuff that colin walters and I think jkeating eventually want to do pretty well, too.
definitely worth discussing.
Something to think about for the weekend:
The rpm specfile language has long been a source of pain for both users of it and the rpm developers. Maybe the goal in #1 above is a jumping-off point to start away from the specfile language entirely?
It has the virtue of not having to shoehorn things into the specfile format and risk breaking backward compat.
thoughts? -sv
On Fri, 9 Jul 2010, seth vidal wrote:
On Thu, 2010-07-08 at 16:54 -0400, seth vidal wrote:
On Thu, 2010-07-08 at 10:20 +0300, Panu Matilainen wrote:
Actually fixing the issue is going to require more dramatic actions though and changes to the tools. All sorts of possibilities do exist, for example:
Forget SRPMs, just stuff the sources, patches and spec from a given CVS tag into a %{name}-%{version}-%{release}.tar.gz tarball and distribute those instead of SRPMs. There you have a format that's guaranteed to be arch-independent all the way, and they're buildable too, just 'rpmbuild -tb <tarball>' and go.
Create metadata for the SRPMs per-arch, but only actually distribute one SRPM. Might involve things like splitting header and payload to separate files or something.
...or something.
If you ask me, I would go for 1). Seriously. If you want metadata to go with the tarballs, strip the header out of src.rpm's on each arch and build per-arch metadata only.
Talking about 1 is probably worthwhile - hell - even just having the Source locations need to be full paths would make it worthwhile.
It might tie into some of the stuff that colin walters and I think jkeating eventually want to do pretty well, too.
definitely worth discussing.
Something to think about for the weekend:
The rpm specfile language has long been a source of pain for both users of it and the rpm developers. Maybe the goal in #1 above is a jumping-off point to start away from the specfile language entirely?
Heh, I didn't have quite that far-reaching goals in mind wrt #1. I guess everybody agrees the spec "language" has ran wild a long time ago, but redesigning the "rpm build recipe" system is beyond the scope of this discussion. Thinking about spec file future belongs (and is perfectly welcome) to rpm-maint list.
The issue at hand is that although the SRPM largely looks and quacks like a duck, it's really a platypus, and the poor thing doesn't even know it.
What it's "good" for is recording information about the conditions in which a spec file was parsed in and package(s) generated. Ie when you run rpmbuild -ba [switches] <specfile> and grab the results, you can then query the SRPM for various things like the build-requires effective for this particular build (subject to installed packages, effective macros, command line switches and all), on that architecture.
In the Red Hat Linux era, separate SRPMs were actually distributed for each arch, which is how SRPM "works". Back in those days the number of packages and architectures was quite something else and duplicating a few hundred megs worth of sources wasn't such a big deal. Doing that nowadays would be plain insane.
The SRPM is simply not a good format for what its currently used for (outside the buildsystem itself, where its usage for build-dependency installation is just fine). Sure its possible to write specs in a way that produce identical buildrequires and payload in all conditions, but it requires some care and packaging policies.
I've suggested getting rid of the distributed SRPMs completely, but apparently CVS + tags isn't sufficient from legal perspective. Well, from this POV: the spec/srpm combo CANNOT GUARANTEE a source rpm contains everything used to build the package on all architectures. Quite the contrary, its all too easy to create a spec which contains different sources and patches on different architectures. It's up to packager sanity + policies (assuming policies for this exist). The good ol' tarball created from CVS tags + lookaside cache at the time of build would have that guarantee, unconditionally. And a tarball is what it appears to be and not a platypus in disguise.
Oh and FWIW, this doesn't affect the way buildsys builds binaries a single bit, it "only" affects the source repository and anything using it. So while such a change would obviously require quite a lot of changes in various places, it doesn't require rewriting the entire buildsystem or disrupt the regular "do stuff in CVS, submit build, feed babies to rawhide" loop as such.
- Panu -
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 7/7/10 11:19 PM, seth vidal wrote:
If srpms are not arch-independent do you mean to say that we need to create separate srpms for each arch we build on?
We do, sortof, during the build in koji. However what we don't do is separately store, track, and distribute those srpms. We used to way back in the day, and one of the changes I made when I took over was to unify the source rpm directories.
I cannot tell you if the srpms in each arch directory were unique or not, they could have been the same srpm just copied into each subdirectory.
- -- Jesse Keating Fedora -- Freedom² is a feature! identi.ca: http://identi.ca/jkeating
On 07/07/2010 02:53 AM, Mattias Ellert wrote:
tis 2010-07-06 klockan 11:40 -0400 skrev Tom "spot" Callaway:
Lets say you have BuildRequires: foo%{?_isa}. Koji makes the SRPM on a ppc builder, so %{_isa} is evaluated as (ppc-32). The SRPM is then handed to an x86_64 builder to build for x86_64, but it cannot meet the BuildRequires because foo(ppc-32) is not found.
~spot
This is not how it works.
rpmbuild re-evaluates the BuildRequires from the specfile inside the SRPM, it does not use the SRPM's Requires. Having %{?_isa} in BuildRequires tags is perfectly legal - and advisable to have if you want your SRPMs to be buildable in a multiarch environment.
You should to let Panu know if you've fixed this process (and probably send him the patches). This is probably the place to post them:
[Bug 554854] %{_isa} requires are propagated to source RPMs
...feel free to post the koji/yum/etc. patches there too.
packaging@lists.fedoraproject.org
-
Braden McDaniel -
James Antill -
Jesse Keating -
Mattias Ellert -
Michael Schwendt -
Panu Matilainen -
Ralf Corsepius -
Seth Vidal -
Tom Callaway -
Ville Skyttä