TL;DR: There's a proposal for nuking the list of BuildRequires exceptions from the packaging guidelines.
Important note: this is just an idea. We're not currently changing the guidelines based on this. Nothing is set in stone. It may end up that nothing happens. Don't panic.
A couple of times recently, people have floated proposals to the FPC about removing certain things from the buildroot. (Specifically, perl and gcc*.) And the bottom line for these is that while FPC tries to maintain a list of BuildRequires: exceptions, this is really up to releng and whatever rpm happens to pull in at any particular time.
This means that whatever list FPC maintains, it's going to get outdated eventually, and giving packagers the idea that they don't have to actually specify all of their dependencies restricts what releng can do and, perhaps, what dependencies the RPM package can drop without breaking builds.
So, the generally stated proposal:
I would like to get FPC out of the business of specifying this list of exceptions, and instead just indicate that packagers should completely specify their dependencies.
My specific proposal can be seen in https://fedorahosted.org/fpc/ticket/497; the draft is at https://fedoraproject.org/wiki/User:Tibbs/BuildReqDraft2. You can use the history tab to see the differences between that and the current guidelines.
Now, the real issue is in what packagers can depend upon. Obviously you have RPM and an environment necessary to build a package (which means you have to have a shell to execute the scripts that make up the RPM sections, and redhat-rpm-config, and whatever RPM happens to use to execute %patch, which I guess could be some internal library if the RPM devs wanted). But what else? That's the open question.
The draft currently says:
--- You may assume that you have everything necessary for RPM to function and process your spec file (so of course RPM is present, along with redhat-rpm-config and what is necessary for RPM to apply patches, unpack archives, and run the shell scripts which make up the spec file sections.) You should not assume any other packages are present, as RPM dependencies and anything brought into the buildroot by the build system may change over time. ---
Honestly, I'm not completely satisfied with that but I can't come up with anything better. Polite discussion is welcomed and encouraged.
- J<
The draft is still a bit wordy IMO.
For example, I don't understand why is there mentioned redhat-rpm-config, since most people have no idea what this package is good for (you can more or less count me in this group as well ;). Patching stuff etc, you don't need to care who provides the functionality, since it is hidden behind RPM macros. Most of the dependencies from the current exception list are pulled in via RPM anyway.
Also this sentence "but even then keep in mind that you should still double check your BuildRequires: because your package may still rely on the particular makeup of the build environment." is just beating a dead horse.
Otherwise big +1 for this change.
Vít
Dne 12.2.2015 v 17:38 Jason L Tibbitts III napsal(a):
TL;DR: There's a proposal for nuking the list of BuildRequires exceptions from the packaging guidelines.
Important note: this is just an idea. We're not currently changing the guidelines based on this. Nothing is set in stone. It may end up that nothing happens. Don't panic.
A couple of times recently, people have floated proposals to the FPC about removing certain things from the buildroot. (Specifically, perl and gcc*.) And the bottom line for these is that while FPC tries to maintain a list of BuildRequires: exceptions, this is really up to releng and whatever rpm happens to pull in at any particular time.
This means that whatever list FPC maintains, it's going to get outdated eventually, and giving packagers the idea that they don't have to actually specify all of their dependencies restricts what releng can do and, perhaps, what dependencies the RPM package can drop without breaking builds.
So, the generally stated proposal:
I would like to get FPC out of the business of specifying this list of exceptions, and instead just indicate that packagers should completely specify their dependencies.
My specific proposal can be seen in https://fedorahosted.org/fpc/ticket/497; the draft is at https://fedoraproject.org/wiki/User:Tibbs/BuildReqDraft2. You can use the history tab to see the differences between that and the current guidelines.
Now, the real issue is in what packagers can depend upon. Obviously you have RPM and an environment necessary to build a package (which means you have to have a shell to execute the scripts that make up the RPM sections, and redhat-rpm-config, and whatever RPM happens to use to execute %patch, which I guess could be some internal library if the RPM devs wanted). But what else? That's the open question.
The draft currently says:
You may assume that you have everything necessary for RPM to function and process your spec file (so of course RPM is present, along with redhat-rpm-config and what is necessary for RPM to apply patches, unpack archives, and run the shell scripts which make up the spec file sections.) You should not assume any other packages are present, as RPM dependencies and anything brought into the buildroot by the build system may change over time.
Honestly, I'm not completely satisfied with that but I can't come up with anything better. Polite discussion is welcomed and encouraged.
- J<
-- packaging mailing list packaging@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/packaging
It's been a long time since I first proposed this, but I had a bit of time so I took the feedback from this list and updated my draft for chopping down the BuildRequires: section in the packaging guidelines. Basically, I'm now at a single paragraph:
It is important that your package list all necessary build dependencies using the BuildRequires: tag. You may assume that enough of an environment exists for RPM to function and execute basic shell scripts, but you should not assume any other packages are present as RPM dependencies and anything brought into the buildroot by the build system may change over time.
Comments are welcome.
On Tuesday, 05 May 2015 at 01:34, Jason L Tibbitts III wrote: [...]
It is important that your package list all necessary build dependencies using the BuildRequires: tag. You may assume that enough of an environment exists for RPM to function and execute basic shell scripts, but you should not assume any other packages are present as RPM dependencies and anything brought into the buildroot by the build system may change over time.
+1. Thanks for working on this.
Regards, Dominik
5.5.2015 2.34 "Jason L Tibbitts III" tibbs@math.uh.edu kirjoitti:
You may assume that enough of an environment exists for RPM to function and execute basic shell scripts
s/RPM/rpmbuild/, and in addition to running shell scripts it is in a state where it can unpack common archive types and apply patches?
Dne 6.5.2015 v 12:34 Ville Skyttä napsal(a):
5.5.2015 2.34 "Jason L Tibbitts III" <tibbs@math.uh.edu mailto:tibbs@math.uh.edu> kirjoitti:
You may assume that enough of an environment exists for RPM to function and execute basic shell scripts
s/RPM/rpmbuild/, and in addition to running shell scripts it is in a state where it can unpack common archive types and apply patches?
"You may assume that enough of an environment exists for rpmbuild to function" might be enough. Not sure if you really want to go into details.
# dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build Using metadata from Wed May 6 13:13:11 2015 rtld(GNU_HASH) /bin/sh libm.so.6()(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libdl.so.2()(64bit) libz.so.1()(64bit) /bin/bash libc.so.6(GLIBC_2.17)(64bit) libbz2.so.1()(64bit) libnss3.so()(64bit) libpopt.so.0()(64bit) libdb-5.3.so()(64bit) liblzma.so.5()(64bit) libpopt.so.0(LIBPOPT_0)(64bit) libcap.so.2()(64bit) grep sed libacl.so.1()(64bit) liblua-5.3.so()(64bit) librpm.so.3()(64bit) gawk librpmio.so.3()(64bit) bzip2 gzip libmagic.so.1()(64bit) tar libelf.so.1()(64bit) xz binutils findutils libelf.so.1(ELFUTILS_1.0)(64bit) unzip diffutils file cpio librpmbuild.so.3()(64bit) perl-generators system-rpm-config /usr/bin/gdb-add-index rpm = 4.12.0.1-14.fc23 elfutils >= 0.128 patch >= 2.5 pkgconfig >= 1:0.24
Vít
"VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
- J<
On 05/06/2015 07:12 PM, Jason L Tibbitts III wrote:
"VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
As much as I welcome this effort, I think we need a detailed and fixed per-fedora-release package list, to be able to give packagers some helpful guidance.
That said, why can't we have a link to the list being used by mock (The packages being listed in mock's "buildsys-build") inside the FPG?
Ralf
Dne 6.5.2015 v 19:48 Ralf Corsepius napsal(a):
On 05/06/2015 07:12 PM, Jason L Tibbitts III wrote:
> "VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
As much as I welcome this effort, I think we need a detailed and fixed per-fedora-release package list, to be able to give packagers some helpful guidance.
That said, why can't we have a link to the list being used by mock (The packages being listed in mock's "buildsys-build") inside the FPG?
What would be the point of link to comps when most of the dependencies are defined by rpm-build package? Actually, the buildsys-build should contain just rpm-build and nothing more (or it could be abandoned entirely, since it would loose its purpose this way).
Moreover, if the buildsys-build contains components which are already required by rpm-build, they should be cleaned up immediately. I am speaking about bzip2, gzip, tar, xz, sed, patch, grep, gawk, findutils, diffutils, cpio, bash, these are all duplicated.
Vít
On 05/07/2015 09:21 AM, Vít Ondruch wrote:
Dne 6.5.2015 v 19:48 Ralf Corsepius napsal(a):
On 05/06/2015 07:12 PM, Jason L Tibbitts III wrote:
>> "VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
As much as I welcome this effort, I think we need a detailed and fixed per-fedora-release package list, to be able to give packagers some helpful guidance.
That said, why can't we have a link to the list being used by mock (The packages being listed in mock's "buildsys-build") inside the FPG?
What would be the point of link to comps when most of the dependencies are defined by rpm-build package?
With the FPG being changed to the proposal, I am expecting - length discussions (esp. in reviews) about whether tools "x" is guaranteed to be present in build-roots: "Is touch, ls, zip, etc. in build-root or not? (Been there, seen that many times :) )
- broken builds, because vanishing implicit BR's can trigger different sets of build conditions and thus break packages. (Bugs/discussions along the lines package A in fcX had "feature A", fcX+1 lacks it).
Actually, the buildsys-build should contain just rpm-build and nothing more (or it could be abandoned entirely, since it would loose its purpose this way).
I do not consider this to be workable, because rpm-build's deps are just arbitrary requirements and _not_ a well defined fundation of tools.
That said, I do not consider "rpm-build" to be something to be featured.
Instead, we need an explict well defined set of tools which are guaranteed to be present throughout the life-time of a release.
Implementation-wise, this could be implemented as explict BRs of rpm-build, an independent package or a package group. IMO, the appropriate party to define this set of packages is those people who define the set of packages in mock.
Moreover, if the buildsys-build contains components which are already required by rpm-build, they should be cleaned up immediately. I am speaking about bzip2, gzip, tar, xz, sed, patch, grep, gawk, findutils, diffutils, cpio, bash, these are all duplicated.
IMO, here, you are repeating my reasoning above in different words: rpm-build's deps are arbitrary.
Ralf
Dne 7.5.2015 v 09:48 Ralf Corsepius napsal(a):
On 05/07/2015 09:21 AM, Vít Ondruch wrote:
Dne 6.5.2015 v 19:48 Ralf Corsepius napsal(a):
On 05/06/2015 07:12 PM, Jason L Tibbitts III wrote:
>>> "VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
As much as I welcome this effort, I think we need a detailed and fixed per-fedora-release package list, to be able to give packagers some helpful guidance.
That said, why can't we have a link to the list being used by mock (The packages being listed in mock's "buildsys-build") inside the FPG?
What would be the point of link to comps when most of the dependencies are defined by rpm-build package?
With the FPG being changed to the proposal, I am expecting
- length discussions (esp. in reviews) about whether tools "x" is
guaranteed to be present in build-roots: "Is touch, ls, zip, etc. in build-root or not? (Been there, seen that many times :) )
No guarantee is the best guarantee.
- broken builds, because vanishing implicit BR's can trigger different
sets of build conditions and thus break packages. (Bugs/discussions along the lines package A in fcX had "feature A", fcX+1 lacks it).
You pretend like this does not happen on any level just a bit deeper then what is defined by buildsys-build or rpm-build. But the bad news is that it happens anywhere any time.
Actually, the buildsys-build should contain just rpm-build and nothing more (or it could be abandoned entirely, since it would loose its purpose this way).
I do not consider this to be workable, because rpm-build's deps are just arbitrary requirements and _not_ a well defined fundation of tools.
That said, I do not consider "rpm-build" to be something to be featured.
Instead, we need an explict well defined set of tools which are guaranteed to be present throughout the life-time of a release.
Implementation-wise, this could be implemented as explict BRs of rpm-build, an independent package or a package group. IMO, the appropriate party to define this set of packages is those people who define the set of packages in mock.
Moreover, if the buildsys-build contains components which are already required by rpm-build, they should be cleaned up immediately. I am speaking about bzip2, gzip, tar, xz, sed, patch, grep, gawk, findutils, diffutils, cpio, bash, these are all duplicated.
IMO, here, you are repeating my reasoning above in different words: rpm-build's deps are arbitrary.
As well as any defined group is arbitrary. We have probably disagree here.
Vít
On Thu, May 7, 2015 at 3:56 AM, Vít Ondruch vondruch@redhat.com wrote:
Dne 7.5.2015 v 09:48 Ralf Corsepius napsal(a):
On 05/07/2015 09:21 AM, Vít Ondruch wrote:
Dne 6.5.2015 v 19:48 Ralf Corsepius napsal(a):
On 05/06/2015 07:12 PM, Jason L Tibbitts III wrote:
>>>> "VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
As much as I welcome this effort, I think we need a detailed and fixed per-fedora-release package list, to be able to give packagers some helpful guidance.
That said, why can't we have a link to the list being used by mock (The packages being listed in mock's "buildsys-build") inside the FPG?
What would be the point of link to comps when most of the dependencies are defined by rpm-build package?
With the FPG being changed to the proposal, I am expecting
- length discussions (esp. in reviews) about whether tools "x" is
guaranteed to be present in build-roots: "Is touch, ls, zip, etc. in build-root or not? (Been there, seen that many times :) )
No guarantee is the best guarantee.
Stability and clarity help. One of the classic dependencies that gets mucked up, right now, is 'mysql-libs'', with different MySQL variants each having their own version of it, with a different name, and different interwoven dependencies. Another is when there are two parallel versions of the same component, such as 'samba' and 'samba4' had for a while.
Moreover, if the buildsys-build contains components which are already required by rpm-build, they should be cleaned up immediately. I am speaking about bzip2, gzip, tar, xz, sed, patch, grep, gawk, findutils, diffutils, cpio, bash, these are all duplicated.
IMO, here, you are repeating my reasoning above in different words: rpm-build's deps are arbitrary.
As well as any defined group is arbitrary. We have probably disagree here.
One reason to be more explicit about some of these is backwardsa compatibility. If a particular ".spec" file is still stable for Fedora or RHEL releases before the current rpmbuild contained these particular components, "zip", in particular, is not required by 'mock' baed rpmbuild environments, but some source bundles are now using it because of GPG signed git tag exports for source code, rather than published tarballs. I'm actually fairly horrified by how Red Hat is publishing their RHEL source code these days, and really wish they'd switch to using such tags rather then the very funky "read the git logs and look for the word 'import'" to denote what everyone else in the world would make a git tag, even a signed git tag these days.
Dne 6.5.2015 v 19:12 Jason L Tibbitts III napsal(a):
"VO" == Vít Ondruch vondruch@redhat.com writes:
VO> # dnf repoquery --disablerepo=* --enablerepo=rawhide --requires rpm-build
Those dependencies could change at any time. I would like for them to be able to change without the guidelines having to change with them. Obviously any change would break some package somewhere, but at least it would get one committee out of the process.
- J<
Actually that was my point, why not speak about shell, unpacking archives, applying patches etc and hence the proposed abbreviated version of guideline you leaved out from the quote :)
Vít
"VO" == Vít Ondruch vondruch@redhat.com writes:
VO> Actually that was my point, why not speak about shell, unpacking VO> archives, applying patches etc and hence the proposed abbreviated VO> version of guideline you leaved out from the quote :)
I know, I was reinforcing your point, not refuting it.
- J<
"VS" == Ville Skyttä ville.skytta@iki.fi writes:
VS> s/RPM/rpmbuild/, and in addition to running shell scripts it is in a VS> state where it can unpack common archive types and apply patches?
RPM can do that, but you should not assume how. It could link in libraries to do that without actually installing any executables it could call. Or it could call things which are not in $PATH.
- J<
On Wed, May 6, 2015 at 8:10 PM, Jason L Tibbitts III tibbs@math.uh.edu wrote:
"VS" == Ville Skyttä ville.skytta@iki.fi writes:
VS> s/RPM/rpmbuild/, and in addition to running shell scripts it is in a VS> state where it can unpack common archive types and apply patches?
RPM can do that, but you should not assume how. It could link in libraries to do that without actually installing any executables it could call. Or it could call things which are not in $PATH.
Sure (well except that it's still rpmbuild, not rpm...). I wasn't making any assumptions how it does that, actually pretty much the opposite. I just listed its features/state that can be assumed so that people won't start adding build dependencies e.g. for tar and gzip whenever they have a tgz source tarball unpacked by %setup or one for patch when they have patches to apply with %patch etc.
On Thu, Feb 12, 2015 at 9:38 AM, Jason L Tibbitts III tibbs@math.uh.edu wrote:
The draft currently says:
You may assume that you have everything necessary for RPM to function and process your spec file (so of course RPM is present, along with redhat-rpm-config and what is necessary for RPM to apply patches, unpack archives, and run the shell scripts which make up the spec file sections.) You should not assume any other packages are present, as RPM dependencies and anything brought into the buildroot by the build system may change over time.
I personally love this idea and hope a solution can be found. In my mind, there are two main goals here that are closely related (and probably even a bit intertwined/competing): 1) Reduce the number of programs that packagers "assume" is available, and 2) Give RPM maximum latitude to change implementation without breaking anything.
I think that changing the wording to say that RPM will have "capabilities" but not "any specific program to accomplish those capabilities" will satisfy the above goals.
My proposed wording would be something like this:
You may assume that RPM has the ability to function and process your spec file. This means that RPM is present it can perform the fundamental tasks (like apply patches, unpack archives, and run the shell scripts which make up the spec file sections), but how these tasks are performed (programs/libraries/etc) *cannot* be relied on. In other words, you should not assume any other packages/programs/libraries are present, as RPM dependencies and anything brought into the buildroot by the build system may change over time.
packaging@lists.fedoraproject.org
-
Dave Johansen -
Dominik 'Rathann' Mierzejewski -
Jason L Tibbitts III -
Nico Kadel-Garcia -
Ralf Corsepius -
Ville Skyttä -
Vít Ondruch