I assume that subject line got your attention.
I know this is a long-standing debate and that this thread is likely to turn into an incomprehensible flamewar filled with the same tired arguments, but I'm going to make a proposal and then attempt to respond to many of those known arguments up-front (in the hopes that we can try to keep the conversation on-track). Please keep the conversation on devel@lists.fedoraproject.org . I CCed packaging@ to make them aware of this discussion.
Right now, we have a policy that essentially forbids source code from being bundled into a package. In technical terms, this means essentially that the packaging policies mandate that any code that appears more than once in the repository must be turned into a shared library and dynamically linked into any package that requires it. Any package that wants an exception to this must petition the Fedora Packaging Committee and get an explicit exemption from this policy. This process is heavyweight and sometimes inconsistent in how the decision is made.
I would like to propose that the no-bundled-libraries policy be amended as follows: "Any package that has an existing mechanism to link against a shared system library and functions correctly when doing so must link against that library and not bundle it internally. Any package whose upstream releases cannot link against a shared system library (or are incompatible with the version in Fedora) may bundle that library (without requiring a special exemption) but MUST add Provides: bundled(<libname>) = <version> in the spec file for each known bundled library.(This will allow us to track down the bundling when we need to). Package maintainers should continue attempt to engage upstream to support linking against shared system libraries wherever possible, due to the advantages it provides the package maintainer."
The reason for this proposal is relatively simple: we know the advantages to unbundling, particularly with security and resource- usage. However, the world's developer community largely *does not care*. We fought the good fight, we tried to bring people around to seeing our reasoning and we failed.
The point of software is to provide a service to an end-user. Users don't run software because it has good packaging policies, they run software because it meets a need that they have. If they can't get that software from Fedora, they *will* get it from another source (or use a different OS that doesn't get in their way). I'll take a moment to remind people that two of Fedora's Four Foundations are "Features" and "First". We want Fedora to be the most feature-complete distribution available and we want to get there before anyone else does. I would say that holding to our no-bundling policy actively defeats our efforts on that score.
Let me describe some of the advantages to bundling and to unbundling (as noted so we can hopefully skip some of the hotter parts of the flamewar). As I noted above, anything that is capable of unbundling should remain unbundled for its advantages. But things that are not currently capable (or can't be due to forwards/backwards compatibility issues, etc.) really shouldn't be forced to attempt it.
== Advantages to using shared libraries == * Security/Bugs - When a bug or security vulnerability is located in a library, it needs to be patched in only a single package in order to fix all applications using that library. * Resources - A shared library only needs to be loaded into memory once, reducing the memory requirements of the system.
== Advantages to bundling == * Guarantees that the application is running with the same set of code that upstream tested. (Fewer Fedora-specific bugs means less burden on the maintainer) * Simplifies packaging of updates. (Fedora maintainer does not need to keep tabs on unbundling patches to keep in sync for new versions) * Increases the available pool of software that can be packaged substantially (many modern languages such as Ruby and Go are realistically only functional with allowable bundling) * Did I mention the reduction in maintainer burden yet?
Thank you for reading this far. I know that this is a topic that people get highly passionate about, so please do your best to restrain your responses to reasoned statments and avoid the temptation to get angry. I'll summon up a third of our Four Foundations here: remember that Fedora is built on "Friends" too. This should be a discussion and not an argument.
On Thu, Sep 10, 2015 at 8:53 AM, Stephen Gallagher sgallagh@redhat.com wrote:
I assume that subject line got your attention.
Most definitely. :)
So it's basically the same but without FPC as a gatekeeper? Do you have any proposals for enforcement? A periodic query of Provides (bundled-foo) and a BZ requesting a review? Sometime projects enable unbundling over time.
"SG" == Stephen Gallagher sgallagh@redhat.com writes:
SG> Right now, we have a policy that essentially forbids source code SG> from being bundled into a package.
Technically we only care if that bundled code is actually compiled in.
SG> In technical terms, this means essentially that the packaging SG> policies mandate that any code that appears more than once in the SG> repository must be turned into a shared library and dynamically SG> linked into any package that requires it.
We're happy with static libraries as well, though of course dynamic libs are preferable because then you really only need to fix issues in one place.
SG> Any package that wants an exception to this must petition the Fedora SG> Packaging Committee and get an explicit exemption from this policy. SG> This process is heavyweight and sometimes inconsistent in how the SG> decision is made.
We try to be consistent, but the committee does change over time as do the opinions of the committee members. If you have examples of what you believe to be inconsistencies, please feel to raise them. I'm guessing that most of the complaints would stem from FPC allowing Firefox but not whatever other program.
SG> I would like to propose that the no-bundled-libraries policy be SG> amended as follows: "Any package that has an existing mechanism to SG> link against a shared system library and functions correctly when SG> doing so must link against that library and not bundle it SG> internally. Any package whose upstream releases cannot link against SG> a shared system library (or are incompatible with the version in SG> Fedora) may bundle that library (without requiring a special SG> exemption) but MUST add Provides: bundled(<libname>) = <version> in SG> the spec file for each known bundled library.
Which is same as the current policy, minus anyone actually caring whether this really happens.
I would not object to a weakening of the rules (and in fact I've pushed a weakening of the rules before, leading to the current situation being less strict than it was previously). I would object rather strenuously to just not bothering.
SG> The reason for this proposal is relatively simple: we know the SG> advantages to unbundling, particularly with security and resource- SG> usage. However, the world's developer community largely *does not SG> care*. We fought the good fight, we tried to bring people around to SG> seeing our reasoning and we failed.
We haven't really failed, though. Good work has come of this, and if we resist these bad habits then more good work can come of it, even to the upstreams which currently resist.
SG> The point of software is to provide a service to an end-user. Users SG> don't run software because it has good packaging policies, they run SG> software because it meets a need that they have.
One could make the same argument for open source in general.
SG> If they can't get that software from Fedora, they *will* get it from SG> another source (or use a different OS that doesn't get in their SG> way).
Exactly. Let's ship binary drivers.
I know that's something of a straw man, but my point is that we must have some principles, and must work with upstreams to attempt to get them to at least understand those principles. And we shouldn't give up on that just because someone wants some program which is easily provided by a copr anyway.
- J<
On Thu, 2015-09-10 at 11:02 -0500, Jason L Tibbitts III wrote: <snip>
"SG" == Stephen Gallagher sgallagh@redhat.com writes:
SG> If they can't get that software from Fedora, they *will* get it from SG> another source (or use a different OS that doesn't get in their SG> way).
Exactly. Let's ship binary drivers.
I know that's something of a straw man, but my point is that we must have some principles, and must work with upstreams to attempt to get them to at least understand those principles. And we shouldn't give up on that just because someone wants some program which is easily provided by a copr anyway.
Binary drivers are a different problem. We don't ship those because there are *legal* concerns preventing it. We've got plenty of stuff in COPRs right now that are not in Fedora proper *solely* because of bundling policies and not because of any hard-line external reason.
On Thu, Sep 10, 2015 at 11:02:27AM -0500, Jason L Tibbitts III wrote:
I know that's something of a straw man, but my point is that we must have some principles, and must work with upstreams to attempt to get them to at least understand those principles. And we shouldn't give up on that just because someone wants some program which is easily provided by a copr anyway.
Well, we _have_ some stated fundamental principles, and Freedom is one of them, while "de-bundle software!" actually isn't. In fact, as Stephen suggests in the initial message, debundling is often in conflict with Features and First.
That said, I do recognize that "provides high-quality packages" has also always been an underlying Fedora value even if unstated. But, I think that _that_ value should be in support of the Big Four, and in support of our mission in general, not a sacred beast for its own sake.
"MM" == Matthew Miller mattdm@fedoraproject.org writes:
MM> That said, I do recognize that "provides high-quality packages" has MM> also always been an underlying Fedora value even if unstated. But, I MM> think that _that_ value should be in support of the Big Four, and in MM> support of our mission in general, not a sacred beast for its own MM> sake.
Well, for the FPC, high quality packaging is pretty much our only mission. I'm trying to avoid veering off into hyperbole here, but if we can't be focused on our specific mission then that kind of complicates our job.
But if FESCo or the board or whoever wants to say that we no longer really care about bundling, then FPC will stop caring. Right now we've been told to care about bundling and so we developed all of this process and rules to implement that directive.
- J<
On Thu, Sep 10, 2015 at 12:39:54PM -0500, Jason L Tibbitts III wrote:
MM> That said, I do recognize that "provides high-quality packages" has MM> also always been an underlying Fedora value even if unstated. But, I MM> think that _that_ value should be in support of the Big Four, and in MM> support of our mission in general, not a sacred beast for its own MM> sake. Well, for the FPC, high quality packaging is pretty much our only mission. I'm trying to avoid veering off into hyperbole here, but if we can't be focused on our specific mission then that kind of complicates our job.
I don't think high quality packaging for its own sake _should_ be the FPC's mission. It should be high quality packaging _for the sake of the Fedora mission, values, and objectives_. Otherwise, it should be the Something-Else Packaging Committee rather than Fedora Package Committee. And that's not to pick on FPC in particular -- all Fedora Subprojects should have the overall project's goals in mind.
But if FESCo or the board or whoever wants to say that we no longer really care about bundling, then FPC will stop caring. Right now we've been told to care about bundling and so we developed all of this process and rules to implement that directive.
Yeah, I think this thread is aimed at figuring out exactly that.
On Thu, 2015-09-10 at 12:39 -0500, Jason L Tibbitts III wrote:
"MM" == Matthew Miller mattdm@fedoraproject.org writes:
MM> That said, I do recognize that "provides high-quality packages" has MM> also always been an underlying Fedora value even if unstated. But, I MM> think that _that_ value should be in support of the Big Four, and in MM> support of our mission in general, not a sacred beast for its own MM> sake.
Well, for the FPC, high quality packaging is pretty much our only mission. I'm trying to avoid veering off into hyperbole here, but if we can't be focused on our specific mission then that kind of complicates our job.
But if FESCo or the board or whoever wants to say that we no longer really care about bundling, then FPC will stop caring. Right now we've been told to care about bundling and so we developed all of this process and rules to implement that directive.
Hi Jason, I have the impression (which may be totally wrong) that you are taking the binary approach here: either we care maximally or we do not care at all.
It seem to me Stephen is making a proposal to tweak just one specific aspect of packaging rule, that is a softer enforcement model. FPC still has a truckload other good rules about packaging and nobody believes FPC should stop caring about overall package quality.
I have mixed opinions myself about allowing bundling, on the one side it makes some things worse, on the other hand, however it is sometimes a way too step barrier to entrance. I've come to the conclusion I'd rather see a softer approach with strong encouragement to use unbundled libraries and a need to justify credibly why a bundled library is used but not a hard rule against it in all cases with hard exceptions to be doled out by the FPC, I think package review should be able to handle it. This risks making somewhat harder to police egregious mis-behavior, but I am sure there are other ways to deal with offenders that willfully break reasonable rules.
HTH, Simo.
"SS" == Simo Sorce ssorce@redhat.com writes:
SS> I have the impression (which may be totally wrong) that you are SS> taking the binary approach here: either we care maximally or we do SS> not care at all.
I sure hope that's not the tack I'm taking.
SS> It seem to me Stephen is making a proposal to tweak just one SS> specific aspect of packaging rule, that is a softer enforcement SS> model. FPC still has a truckload other good rules about packaging SS> and nobody believes FPC should stop caring about overall package SS> quality.
Indeed, that sums up Stephen's proposal. Others (and specifically, the message to which I was responding) sort of pushed the realm a bit farther than just relaxing one rule. Indeed, it was pushed out to the idea that "good packaging" should be subservient to "features" and "first". Which it very may well be; that's not FPC's decision. When FPC is tasked to set out guidelines to enforce "good packaging", that's what it does. It's somewhat of a blunt instrument, because that's what it has been asked to be.
I don't happen to agree with Stephen's proposal, but I do agree that at the very least FPC needs to tighten up the language surrounding the exception process. Personally I would like to relax the rules as well, but that's just my opinion and I haven't really fleshed it out to the point that I could articulate anything reasonably.
Anyway, what I don't get is why we're to the point of tossing out the primary anti-bundling rule when FESCo has always had the power to override any FPC decision. So FPC says "this isn't good packaging" and FESCo can say "we understand, but quality packaging here is subservient to the distro's mission". That's always been the case, even when the "E" stood for "Extras", and I suspect it would have worked just fine for this situation. Instead we're here debating whether FPC should be in the business of reviewing bundling issues at all.
- J<
On Thu, Sep 10, 2015 at 02:41:13PM -0500, Jason L Tibbitts III wrote:
Anyway, what I don't get is why we're to the point of tossing out the primary anti-bundling rule when FESCo has always had the power to override any FPC decision. So FPC says "this isn't good packaging" and FESCo can say "we understand, but quality packaging here is subservient to the distro's mission". That's always been the case, even when the "E" stood for "Extras", and I suspect it would have worked just fine for this situation. Instead we're here debating whether FPC should be in the business of reviewing bundling issues at all.
I think it's because overriding a different group seems hostile, even if it isn't meant that way. And FESCo doesn't want to feel like they're second-guessing other groups all the time. But, if FESCo and FPC want to (more, I guess) explicitly spell out that FPC takes a purist approach and that it's FESCo's place to make exceptions when they serve greater Fedora goals, maybe that could work?
"MM" == Matthew Miller mattdm@fedoraproject.org writes:
MM> I think it's because overriding a different group seems hostile, MM> even if it isn't meant that way. And FESCo doesn't want to feel like MM> they're second-guessing other groups all the time.
Well, FPC even has a "bounce to FESCo clause" in the rules we follow. Every decision used to go through FESCo, but the latter decided it was a bit too much overhead. I don't really blame them.
In any case, the FESCo/FPC interaction (which I know predates the involvement of many people) was designed with this check in mind. It isn't hostile (though I'm sure some will see it that way no matter what happens).
MM> But, if FESCo and FPC want to (more, I guess) explicitly spell out MM> that FPC takes a purist approach and that it's FESCo's place to make MM> exceptions when they serve greater Fedora goals, maybe that could MM> work?
That's how it started and how at least FPC has pretty much always operated. Of course, FPC does understand that you can't do everything and does grant exemptions when they make sense to FPC. It's just that what makes sense to FPC (or at least, whatever consensus arises out of FPC discussion) might not make sense to FESCo or to the folks who just want something without worrying about FPC's restrictions.
- J<
2015-09-10 15:53 GMT+02:00 Stephen Gallagher sgallagh@redhat.com:
I assume that subject line got your attention.
I know this is a long-standing debate and that this thread is likely to turn into an incomprehensible flamewar filled with the same tired arguments, but I'm going to make a proposal and then attempt to respond to many of those known arguments up-front (in the hopes that we can try to keep the conversation on-track). Please keep the conversation on devel@lists.fedoraproject.org . I CCed packaging@ to make them aware of this discussion.
Right now, we have a policy that essentially forbids source code from being bundled into a package. In technical terms, this means essentially that the packaging policies mandate that any code that appears more than once in the repository must be turned into a shared library and dynamically linked into any package that requires it. Any package that wants an exception to this must petition the Fedora Packaging Committee and get an explicit exemption from this policy. This process is heavyweight and sometimes inconsistent in how the decision is made.
I would like to propose that the no-bundled-libraries policy be amended as follows: "Any package that has an existing mechanism to link against a shared system library and functions correctly when doing so must link against that library and not bundle it internally. Any package whose upstream releases cannot link against a shared system library (or are incompatible with the version in Fedora) may bundle that library (without requiring a special exemption) but MUST add Provides: bundled(<libname>) = <version> in the spec file for each known bundled library.(This will allow us to track down the bundling when we need to). Package maintainers should continue attempt to engage upstream to support linking against shared system libraries wherever possible, due to the advantages it provides the package maintainer."
Well, I won't discuss about benefits of unbundling vs bundling, this is a sterile conversation.
I'll just point out that a non-negligible part of Fedora packages already bundle libraries. Unless we want to mass-review and ban all those packages, I consider that Stephen's proposal to be an *improvement* to the current situation.
I would prefer that we also empower SIGs or provenpackagers to grant or not those bundling exceptions and enforce the provides thing.
If you disagree, please come up with a counter-proposal even if it means dropping half of Fedora packages. Keeping status quo is at best diverting our eyes from bad practices, at worst, hypocrite.
The reason for this proposal is relatively simple: we know the advantages to unbundling, particularly with security and resource- usage. However, the world's developer community largely *does not care*. We fought the good fight, we tried to bring people around to seeing our reasoning and we failed.
The point of software is to provide a service to an end-user. Users don't run software because it has good packaging policies, they run software because it meets a need that they have. If they can't get that software from Fedora, they *will* get it from another source (or use a different OS that doesn't get in their way). I'll take a moment to remind people that two of Fedora's Four Foundations are "Features" and "First". We want Fedora to be the most feature-complete distribution available and we want to get there before anyone else does. I would say that holding to our no-bundling policy actively defeats our efforts on that score.
Let me describe some of the advantages to bundling and to unbundling (as noted so we can hopefully skip some of the hotter parts of the flamewar). As I noted above, anything that is capable of unbundling should remain unbundled for its advantages. But things that are not currently capable (or can't be due to forwards/backwards compatibility issues, etc.) really shouldn't be forced to attempt it.
== Advantages to using shared libraries ==
- Security/Bugs - When a bug or security vulnerability is located in
a library, it needs to be patched in only a single package in order to fix all applications using that library.
- Resources - A shared library only needs to be loaded into memory
once, reducing the memory requirements of the system.
== Advantages to bundling ==
- Guarantees that the application is running with the same set of
code that upstream tested. (Fewer Fedora-specific bugs means less burden on the maintainer)
- Simplifies packaging of updates. (Fedora maintainer does not need
to keep tabs on unbundling patches to keep in sync for new versions)
- Increases the available pool of software that can be packaged
substantially (many modern languages such as Ruby and Go are realistically only functional with allowable bundling)
- Did I mention the reduction in maintainer burden yet?
Thank you for reading this far. I know that this is a topic that people get highly passionate about, so please do your best to restrain your responses to reasoned statments and avoid the temptation to get angry. I'll summon up a third of our Four Foundations here: remember that Fedora is built on "Friends" too. This should be a discussion and not an argument. -- packaging mailing list packaging@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/packaging
packaging@lists.fedoraproject.org
-
Haïkel -
Jason L Tibbitts III -
Jon Ciesla -
Matthew Miller -
Simo Sorce -
Stephen Gallagher