Hi all,
Reading logs from yesterdays FPC meeting [1], I think we should discuss what is actually purpose of packaging guidelines and which version of Fedora/EPEL/RHEL they actually targets.
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
1) single version of .spec file to cover the whole Red Hat ecosystem.
2) clean .spec file following the latest and greatest packaging practices.
I personally belong to the group (2) and that is for several reasons:
a) I use Rawhide on daily basis and I develop only for Rawhide. If I do changes in older Fedoras, then it is typically just bug fixes and honestly, that does not happen often (I am POC of ~200 packages and I submitted just 40 updates during last year [2]). And in fact, this is official philosophy of updates [3], not just mine.
b) I spent time developing features which should simplify packaging (for example in F27+, the RPM %setup macro can expand the .gem packages) and I want to use these technologies to simplify my life and life of others.
c) As a proven packager and person who typically does rebuild of Ruby packages, I really hate the branched .spec files where nobody knows what was the purpose of the branches, most of the branches are for obsolete and unsupported releases etc. It is quite hard to apply any improvements into such packages. Moreover it is not realistic to test them. If they were maintained, it would be different story, but the reality is different.
Don't get me wrong, I understand that there are packagers who has just handful of packages and it is better for them to maintain just single .spec file with all the branches and I don't mind them as long as the packages are really actively maintained. But this approach just don't scale and should be exception and not recommended practice.
To sum this up, my take on packaging guidelines is that *the guidelines should document the most recent practices available in Rawhide and this should be documented*. Covering all the exceptions necessary for older Fedoras (not even mentioning RHEL/EPEL) makes the guidelines unreadable and what is worse, they slow down entire development of Fedora.
Vít
[1] https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
Hi,
I totally agree with the spirit, but that would require Red Hat taking a more active role in backporting package tooling to RHEL/EPEL.
Latest guidelines are always more efficient for everyone (Red Hat employees includes), but all too often they can't be applied to EPEL because no one at Red Hat pushed the corresponding rpm, mock, rpmdevtools, macro updates to RHEL or EPEL :(
The more packages Fedora ships the more inefficient it is to require RHEL/EPEL packagers to use old guidelines in every spec files to avoid tooling backports.
Regards,
Dne 30.11.2017 v 10:35 nicolas.mailhot@laposte.net napsal(a):
Hi,
I totally agree with the spirit, but that would require Red Hat taking a more active role in backporting package tooling to RHEL/EPEL.
Latest guidelines are always more efficient for everyone (Red Hat employees includes), but all too often they can't be applied to EPEL because no one at Red Hat pushed the corresponding rpm, mock, rpmdevtools, macro updates to RHEL or EPEL :(
The more packages Fedora ships the more inefficient it is to require RHEL/EPEL packagers to use old guidelines in every spec files to avoid tooling backports.
Regards,
I think things are changing in Fedora as well as in Red Hat.
Fedora is more agile especially in regard of RPM development, but on the other hand Red Hat will be always lacking behind due to RPM being crucial piece of RHEL. On the other hand RHEL is more agile then it used to be especially due to RHSCL. But also in other aspects, such as rebases of key packages such as OpenSSL or even Kernel.
OTOH, speaking as a Ruby maintainer in Red Hat, the agility does not make the situation with guidelines easier, since we cannot anymore say: "Ah, EPEL7 was based on F19, so apply F19 guidelines". This is simply because Ruby in RHEL 7.4 newly supports the same set of macros as Fedora Rawhide, but does not support the provide/require generators due to RHEL 7 RPM limitations. And frankly, should I go to FPC asking for approving (backward compatible) guideline changes due to change in RHEL? That does not make any sense.
However, one day, if/when next RHEL comes, I want to see RHEL using the latest and greatest technologies available in Fedora. I hope I can generalize this into everybody in Red Hat. And this is another reason why the guidelines should cover only Rawhide and should promote the most recent technologies. If we are not going to use/promote the latest advancements, there would be actually no reason for next RHEL at all. We could just stay in distribution stone age forever.
Vít
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On Thu, 2017-11-30 at 09:49 +0100, Vít Ondruch wrote:
Hi all,
Reading logs from yesterdays FPC meeting [1], I think we should discuss what is actually purpose of packaging guidelines and which version of Fedora/EPEL/RHEL they actually targets.
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
single version of .spec file to cover the whole Red Hat ecosystem.
clean .spec file following the latest and greatest packaging practices.
I'm one of these people too.
I personally belong to the group (2) and that is for several reasons:
a) I use Rawhide on daily basis and I develop only for Rawhide. If I do changes in older Fedoras, then it is typically just bug fixes and honestly, that does not happen often (I am POC of ~200 packages and I submitted just 40 updates during last year [2]). And in fact, this is official philosophy of updates [3], not just mine.
b) I spent time developing features which should simplify packaging (for example in F27+, the RPM %setup macro can expand the .gem packages) and I want to use these technologies to simplify my life and life of others.
c) As a proven packager and person who typically does rebuild of Ruby packages, I really hate the branched .spec files where nobody knows what was the purpose of the branches, most of the branches are for obsolete and unsupported releases etc. It is quite hard to apply any improvements into such packages. Moreover it is not realistic to test them. If they were maintained, it would be different story, but the reality is different.
Don't get me wrong, I understand that there are packagers who has just handful of packages and it is better for them to maintain just single .spec file with all the branches and I don't mind them as long as the packages are really actively maintained. But this approach just don't scale and should be exception and not recommended practice.
To sum this up, my take on packaging guidelines is that *the guidelines should document the most recent practices available in Rawhide and this should be documented*. Covering all the exceptions necessary for older Fedoras (not even mentioning RHEL/EPEL) makes the guidelines unreadable and what is worse, they slow down entire development of Fedora.
If we want to have compatibility, then we need to improve RPM (e.g. by introducing new macro). All ruby/python/nodejs/rust packages look same, except for versions and some special hacks for packages. Tibbs and Panu were proposing some ideas how to make it better, so probably we should look into that direction.
Vít
[1] https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... rg/message/QDQ42LRLCP5NOIFSAMUDMP6ZUH3AAHKN/
- -- - -Igor Gnatenko
On Thu, Nov 30, 2017 at 09:49:14AM +0100, Vít Ondruch wrote:
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
- single version of .spec file to cover the whole Red Hat ecosystem.
- clean .spec file following the latest and greatest packaging practices.
Here's my take: how much impact on the world do you want to have? Are you doing this just for yourself, or to help other people? With that in mind, take a look at the relative use in the world of Fedora and EPEL:
https://twitter.com/mattdm/status/936243506355621888
and considering the velociraptor's comment, my anecdotal sense is that use of NAT and proxies at large institutions using enterprise distros _probably_ makes the red EPEL line even higher.
For those of you who prefer ASCII art to graphics, here's an approximation of this chart: e e e ee ee eeee eee ee eeee eeee eeee fffff fffffffffeef fffffffffff fffffffff eeeee ffffffffffff fff eeeeee eeeeeee 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Particularly in the last few years, Fedora as an OS is doing well. But use of EPEL by our downstreams is growning even more dramatically. Of course, I want Fedora OS growth, but the impact we have is _way_ beyond that, and I'd really like us to think of _Fedora_ as beyond just the base OS, too.
I definitely get the point, though, about it being annoying to have to carry ancient kludges for ten years, or to not be able to take advantage of new packaging technologies until the downstreams have caught up. I don't discount the impact we have by making improvements that eventually trickle down, too — we make our downstreams better with our fast-moving development work as well as by providing a universe of additional software.
We *could* decide to just focus on one at the exclusion of the other — drop EPEL and just work on the latest rolling release stuff. Or, we could say that we need to dial back the packaging improvements and make everyone focus on ecosystem compatibility.
Instead, I'd like to focus effort on bringing the stuff we need to the enterprise distributions more quickly. And I think we need to figure out how to make Fedora/EPEL packages on EL without needing to promise an EL-equivalent maintenance period — that's not reasonable for many Fedora packagers.
But we can make this better. Let's work with the downstreams to that. And, let's use automation to identify and correct problem areas. (Hopefully not surprisingly, this is one of my top requests for the whole Modularity effort. It should make this *easy*.)
Dne 30.11.2017 v 16:02 Matthew Miller napsal(a):
On Thu, Nov 30, 2017 at 09:49:14AM +0100, Vít Ondruch wrote:
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
- single version of .spec file to cover the whole Red Hat ecosystem.
- clean .spec file following the latest and greatest packaging practices.
Here's my take: how much impact on the world do you want to have? Are you doing this just for yourself, or to help other people? With that in mind, take a look at the relative use in the world of Fedora and EPEL:
https://twitter.com/mattdm/status/936243506355621888
and considering the velociraptor's comment, my anecdotal sense is that use of NAT and proxies at large institutions using enterprise distros _probably_ makes the red EPEL line even higher.
For those of you who prefer ASCII art to graphics, here's an approximation of this chart: e e e ee ee eeee eee ee eeee eeee eeee fffff fffffffffeef fffffffffff fffffffff eeeee ffffffffffff fff eeeeee eeeeeee 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Particularly in the last few years, Fedora as an OS is doing well. But use of EPEL by our downstreams is growning even more dramatically. Of course, I want Fedora OS growth, but the impact we have is _way_ beyond that, and I'd really like us to think of _Fedora_ as beyond just the base OS, too.
I definitely get the point, though, about it being annoying to have to carry ancient kludges for ten years, or to not be able to take advantage of new packaging technologies until the downstreams have caught up. I don't discount the impact we have by making improvements that eventually trickle down, too — we make our downstreams better with our fast-moving development work as well as by providing a universe of additional software.
We *could* decide to just focus on one at the exclusion of the other — drop EPEL and just work on the latest rolling release stuff. Or, we could say that we need to dial back the packaging improvements and make everyone focus on ecosystem compatibility.
Instead, I'd like to focus effort on bringing the stuff we need to the enterprise distributions more quickly. And I think we need to figure out how to make Fedora/EPEL packages on EL without needing to promise an EL-equivalent maintenance period — that's not reasonable for many Fedora packagers.
But we can make this better. Let's work with the downstreams to that. And, let's use automation to identify and correct problem areas. (Hopefully not surprisingly, this is one of my top requests for the whole Modularity effort. It should make this *easy*.)
The EPEL number you are presenting are bit unrelated number. You should compare how many "enhancement" and "bugfix" updates were submitted in EPEL versus Fedora (and actually you can't evaluate Fedora correctly since there are no updates in Rawhide). In my domain (Ruby packages), there are plenty of changes in packages in Fedora, mainly in Rawhide, less in stable releases. In EPEL, there typically happens just one single import and then the package stays in that state forever.
And actually, this is the biggest downside of entire EPEL, it is not obvious if EPEL should have the most recent packages or stick with one version forever. And while you might say that the package should follow upstream and be updated to the same version as in Fedora, this effort typically fails as soon as Fedora diverges too far. And then the Rawhide developers have to still deal with EPEL macros which are not useful for Fedora and not even for EPEL.
So I agree on the point that "we need to figure out how to make Fedora/EPEL packages on EL without needing to promise an EL-equivalent maintenance period". But disagree with "that's not reasonable for many Fedora packagers.", because this is in turn not reasonable for EPEL itself, since new versions of RHEL inherits from Fedora.
Vít
On Thu, Nov 30, 2017 at 04:33:07PM +0100, Vít Ondruch wrote:
The EPEL number you are presenting are bit unrelated number. You should compare how many "enhancement" and "bugfix" updates were submitted in EPEL versus Fedora (and actually you can't evaluate Fedora correctly since there are no updates in Rawhide). In my domain (Ruby packages), there are plenty of changes in packages in Fedora, mainly in Rawhide, less in stable releases. In EPEL, there typically happens just one single import and then the package stays in that state forever.
How so? I'm not counting updates — these connection counts show repodata queries, so they happen even if there are no updates. My point here wasn't amount of change, but number of users.
That said, I think EPEL _and_ Fedora OS users would benefit from being able to choose between having frequently updated Ruby packages _or_ the "single import, stays in that state barring a serious security issue" without having that choice be dependent on the base operating system.
Dne 30.11.2017 v 16:39 Matthew Miller napsal(a):
On Thu, Nov 30, 2017 at 04:33:07PM +0100, Vít Ondruch wrote:
The EPEL number you are presenting are bit unrelated number. You should compare how many "enhancement" and "bugfix" updates were submitted in EPEL versus Fedora (and actually you can't evaluate Fedora correctly since there are no updates in Rawhide). In my domain (Ruby packages), there are plenty of changes in packages in Fedora, mainly in Rawhide, less in stable releases. In EPEL, there typically happens just one single import and then the package stays in that state forever.
How so? I'm not counting updates — these connection counts show repodata queries, so they happen even if there are no updates. My point here wasn't amount of change, but number of users.
For me it matters how many times I have to touch the package in Rawhide and how many times in other branches. If I have in .spec file the conditionals for older branches but in reality I don't touch them, because I don't update the old branches, then the conditionals are not just useless but harmful, because they make the .spec file less readable, they prevent automated changes in packages etc. This way we just build technical debt.
That said, I think EPEL _and_ Fedora OS users would benefit from being able to choose between having frequently updated Ruby packages _or_ the "single import, stays in that state barring a serious security issue" without having that choice be dependent on the base operating system.
But then comes upstream which says "ah, we don't support Ruby from RHEL 7 anymore, just because we don't test it" and they put some version restriction somewhere. Then it is not EPEL or Fedora choice. This inevitably happens during the RHEL lifecycle and again, since that moment the conditions in .spec file are useless and just builds the technical debt.
Vít
On 30 November 2017 at 03:49, Vít Ondruch vondruch@redhat.com wrote:
Hi all,
Reading logs from yesterdays FPC meeting [1], I think we should discuss what is actually purpose of packaging guidelines and which version of Fedora/EPEL/RHEL they actually targets.
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
single version of .spec file to cover the whole Red Hat ecosystem.
clean .spec file following the latest and greatest packaging practices.
I personally belong to the group (2) and that is for several reasons:
a) I use Rawhide on daily basis and I develop only for Rawhide. If I do changes in older Fedoras, then it is typically just bug fixes and honestly, that does not happen often (I am POC of ~200 packages and I submitted just 40 updates during last year [2]). And in fact, this is official philosophy of updates [3], not just mine.
b) I spent time developing features which should simplify packaging (for example in F27+, the RPM %setup macro can expand the .gem packages) and I want to use these technologies to simplify my life and life of others.
c) As a proven packager and person who typically does rebuild of Ruby packages, I really hate the branched .spec files where nobody knows what was the purpose of the branches, most of the branches are for obsolete and unsupported releases etc. It is quite hard to apply any improvements into such packages. Moreover it is not realistic to test them. If they were maintained, it would be different story, but the reality is different.
Don't get me wrong, I understand that there are packagers who has just handful of packages and it is better for them to maintain just single .spec file with all the branches and I don't mind them as long as the packages are really actively maintained. But this approach just don't scale and should be exception and not recommended practice.
To sum this up, my take on packaging guidelines is that *the guidelines should document the most recent practices available in Rawhide and this should be documented*. Covering all the exceptions necessary for older Fedoras (not even mentioning RHEL/EPEL) makes the guidelines unreadable and what is worse, they slow down entire development of Fedora.
Honestly, I think the RHEL/EPEL part of your conversation is a Red Herring (aka not the real point).
You are wanting people to follow your method of packaging everything because it makes your job easier. The problem is that other packagers want you to follow their method and that makes your job harder. EPEL versions stand out like a sore thumb of wanting that but even the differences between getting something working in F25 and F28 can be quite a bunch of %if strings which makes your job harder. On the other hand, other people aren't interested in Rawhide and want all that.
The problem is an age old one and one that comes up after every couple of releases. I think the EPEL group have tried a lot of different solutions which try to help on this
1. If a package is not maintainable to the developer, stop maintaining it. At the moment, it will be removed from usage. People wanting it can get it from various archives. 2. Tibbs has gone through several herculian efforts to backport as much of the new FPC macros and such to older EPEL's so that newer schemes can work. 3. We are open to suggestions of what maintainers and developers want or do not want out of the repository.
Vít
[1] https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
[2] https://bodhi.fedoraproject.org/updates/?user=vondruch
[3] https://pagure.io/packaging-committee/issue/710
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Dne 30.11.2017 v 17:32 Stephen John Smoogen napsal(a):
On 30 November 2017 at 03:49, Vít Ondruch vondruch@redhat.com wrote:
Hi all,
Reading logs from yesterdays FPC meeting [1], I think we should discuss what is actually purpose of packaging guidelines and which version of Fedora/EPEL/RHEL they actually targets.
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
single version of .spec file to cover the whole Red Hat ecosystem.
clean .spec file following the latest and greatest packaging practices.
I personally belong to the group (2) and that is for several reasons:
a) I use Rawhide on daily basis and I develop only for Rawhide. If I do changes in older Fedoras, then it is typically just bug fixes and honestly, that does not happen often (I am POC of ~200 packages and I submitted just 40 updates during last year [2]). And in fact, this is official philosophy of updates [3], not just mine.
b) I spent time developing features which should simplify packaging (for example in F27+, the RPM %setup macro can expand the .gem packages) and I want to use these technologies to simplify my life and life of others.
c) As a proven packager and person who typically does rebuild of Ruby packages, I really hate the branched .spec files where nobody knows what was the purpose of the branches, most of the branches are for obsolete and unsupported releases etc. It is quite hard to apply any improvements into such packages. Moreover it is not realistic to test them. If they were maintained, it would be different story, but the reality is different.
Don't get me wrong, I understand that there are packagers who has just handful of packages and it is better for them to maintain just single .spec file with all the branches and I don't mind them as long as the packages are really actively maintained. But this approach just don't scale and should be exception and not recommended practice.
To sum this up, my take on packaging guidelines is that *the guidelines should document the most recent practices available in Rawhide and this should be documented*. Covering all the exceptions necessary for older Fedoras (not even mentioning RHEL/EPEL) makes the guidelines unreadable and what is worse, they slow down entire development of Fedora.
Honestly, I think the RHEL/EPEL part of your conversation is a Red Herring (aka not the real point).
What I really want to answer is the question in $SUBJECT, since the scope of guidelines is not specified anywhere. It seems that FPC itself does not know what releases they target and my guidelines update [1] is stuck in review just because of this.
The packaging style is very related of course. If the philosophy of Fedora/EPEL was "every change goes into every branch" then the guidelines should probably cover all the branches and discuss all the differences. But the update policy [2] says the opposite: "we should avoid major updates of packages within a stable release. Updates should aim to fix bugs, and not introduce features, particularly when those features would materially affect the user or developer experience". So saying that majority of people supports all the branches is against the policy IMO.
Also, if the guidelines covered all the branches, the probably Rust guidelines would not be approved yet ...
V.
[1] https://pagure.io/packaging-committee/issue/710 [2] https://fedoraproject.org/wiki/Updates_Policy#Philosophy
On Friday, 01 December 2017 at 11:39, Vít Ondruch wrote: [...]
What I really want to answer is the question in $SUBJECT, since the scope of guidelines is not specified anywhere. It seems that FPC itself does not know what releases they target and my guidelines update [1] is stuck in review just because of this.
It is my understanding that the unwritten rule is that the guidelines are targeted at rawhide and we (FPC) try to maintain notes documenting which releases need exceptions or don't support some things.
The packaging style is very related of course. If the philosophy of Fedora/EPEL was "every change goes into every branch" then the guidelines should probably cover all the branches and discuss all the differences. But the update policy [2] says the opposite: "we should avoid major updates of packages within a stable release. Updates should aim to fix bugs, and not introduce features, particularly when those features would materially affect the user or developer experience". So saying that majority of people supports all the branches is against the policy IMO.
Now you're stretching it in my opinion. "Supporting" stable releases doesn't mean releasing every update made in rawhide to stable branches as well. Fixing bugs is still "supporting" and I don't see why it would be against the policy.
Regards, Dominik
Another almost two months passed and updated Ruby guidelines are still not approved. Hence I opened FPC ticket which requests the packaging guidelines to be Rawhide only. This should hopefully help us improve the packaging:
https://pagure.io/packaging-committee/issue/742
Vít
Dne 30.11.2017 v 09:49 Vít Ondruch napsal(a):
Hi all,
Reading logs from yesterdays FPC meeting [1], I think we should discuss what is actually purpose of packaging guidelines and which version of Fedora/EPEL/RHEL they actually targets.
Apparently, there are two camps of packagers in Fedora/EPEL. Those who want:
single version of .spec file to cover the whole Red Hat ecosystem.
clean .spec file following the latest and greatest packaging practices.
I personally belong to the group (2) and that is for several reasons:
a) I use Rawhide on daily basis and I develop only for Rawhide. If I do changes in older Fedoras, then it is typically just bug fixes and honestly, that does not happen often (I am POC of ~200 packages and I submitted just 40 updates during last year [2]). And in fact, this is official philosophy of updates [3], not just mine.
b) I spent time developing features which should simplify packaging (for example in F27+, the RPM %setup macro can expand the .gem packages) and I want to use these technologies to simplify my life and life of others.
c) As a proven packager and person who typically does rebuild of Ruby packages, I really hate the branched .spec files where nobody knows what was the purpose of the branches, most of the branches are for obsolete and unsupported releases etc. It is quite hard to apply any improvements into such packages. Moreover it is not realistic to test them. If they were maintained, it would be different story, but the reality is different.
Don't get me wrong, I understand that there are packagers who has just handful of packages and it is better for them to maintain just single .spec file with all the branches and I don't mind them as long as the packages are really actively maintained. But this approach just don't scale and should be exception and not recommended practice.
To sum this up, my take on packaging guidelines is that *the guidelines should document the most recent practices available in Rawhide and this should be documented*. Covering all the exceptions necessary for older Fedoras (not even mentioning RHEL/EPEL) makes the guidelines unreadable and what is worse, they slow down entire development of Fedora.
Vít
[1] https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
[2] https://bodhi.fedoraproject.org/updates/?user=vondruch
[3] https://pagure.io/packaging-committee/issue/710
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
packaging@lists.fedoraproject.org
-
Dominik 'Rathann' Mierzejewski -
Igor Gnatenko -
Matthew Miller -
nicolas.mailhot＠laposte.net -
Stephen John Smoogen -
Vít Ondruch