Hi,
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
Any comments, suggestions welcome. Orcan
Orcan Ogetbil oget.fedora@gmail.com writes:
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
I don't like Proposal #2 a bit. The particular case that is going to bite me is that this policy requires me to rebuild documentation files (.pdfs, etc) from source. Which is possible, but it vastly expands the BuildRequires footprint of the package, to say nothing of the build time. And in return for what? Nothing at all, that's what. The excuse that upstream might perhaps have failed to update those files before shipping is as lame as can be.
It should be the packager's decision how much of a package needs to be routinely rebuilt. I concur with the existing policy regarding rebuilding executables, but I don't think it should be expanded to take away packager's discretion for other things.
regards, tom lane
On Thu, 2009-06-18 at 15:17 -0400, Tom Lane wrote:
Orcan Ogetbil oget.fedora@gmail.com writes:
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
I don't like Proposal #2 a bit. The particular case that is going to bite me is that this policy requires me to rebuild documentation files (.pdfs, etc) from source. Which is possible, but it vastly expands the BuildRequires footprint of the package, to say nothing of the build time. And in return for what? Nothing at all, that's what. The excuse that upstream might perhaps have failed to update those files before shipping is as lame as can be.
Seconded. This will cause multilib conflicts in many packages, due to randomness introduced in doc builds, such as generated ids, etc. Not to mention the pointless build-time explosion. And where to stop ? configure, Makefile.in, etc are technically also pre-built files.
It should be the packager's decision how much of a package needs to be routinely rebuilt. I concur with the existing policy regarding rebuilding executables, but I don't think it should be expanded to take away packager's discretion for other things.
Seconded. The existing guideline is fine and clear enough.
On Thu, Jun 18, 2009 at 3:24 PM, Matthias Clasen wrote:
It should be the packager's decision how much of a package needs to be routinely rebuilt. I concur with the existing policy regarding rebuilding executables, but I don't think it should be expanded to take away packager's discretion for other things.
Seconded. The existing guideline is fine and clear enough.
I don't agree (on the existing guideline being fine and clear enough). As I wrote down, there is confusion. Some reviewers did not accept prebuilt doc files (pdf, for instance) in some reviews. If the Proposal 2 is not accepted, I will make a further proposal that says prebuilt data files are allowed unless stated otherwise in a particular guideline (fonts etc).
Orcan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 18.06.2009 21:45, schrieb Orcan Ogetbil:
I don't agree (on the existing guideline being fine and clear enough). As I wrote down, there is confusion. Some reviewers did not accept prebuilt doc files (pdf, for instance) in some reviews. If the Proposal 2 is not accepted, I will make a further proposal that says prebuilt data files are allowed unless stated otherwise in a particular guideline (fonts etc).
I want to suggest, that we should clarifited, that all prebuild executables (programms, libraries, Java jar files) and as an exception python eggs-infos should be removed in the %prepare section of the SPEC file.
So there should not be the need to create documentations from source.
Best Regards:
Jochen Schmitt
Matthias Clasen mclasen@redhat.com writes:
On Thu, 2009-06-18 at 15:17 -0400, Tom Lane wrote:
I don't like Proposal #2 a bit. The particular case that is going to bite me is that this policy requires me to rebuild documentation files (.pdfs, etc) from source. Which is possible, but it vastly expands the BuildRequires footprint of the package, to say nothing of the build time.
Seconded. This will cause multilib conflicts in many packages, due to randomness introduced in doc builds, such as generated ids, etc.
Hadn't thought of that, but it's a good point. Another issue is that expanding BuildRequires to cover doc toolchains is likely to create cases of circular build dependencies that don't exist today. Those situations are a *serious* PITA for the maintainers involved :-( ... so we should avoid mandating that for marginal reasons.
regards, tom lane
On Thu, 2009-06-18 at 15:55 -0400, Tom Lane wrote:
Matthias Clasen mclasen@redhat.com writes:
On Thu, 2009-06-18 at 15:17 -0400, Tom Lane wrote:
I don't like Proposal #2 a bit. The particular case that is going to bite me is that this policy requires me to rebuild documentation files (.pdfs, etc) from source. Which is possible, but it vastly expands the BuildRequires footprint of the package, to say nothing of the build time.
Seconded. This will cause multilib conflicts in many packages, due to randomness introduced in doc builds, such as generated ids, etc.
Hadn't thought of that, but it's a good point. Another issue is that expanding BuildRequires to cover doc toolchains is likely to create cases of circular build dependencies that don't exist today. Those situations are a *serious* PITA for the maintainers involved :-( ... so we should avoid mandating that for marginal reasons.
.. not to mention that at least documentation in (La)TeX can sometimes be a PITA to compile.
Does recompiling the documentation add any value? No, in case the version compiled by upstream is up to date, since the result should be equal. Using ready built documentation doesn't result in any security issues, incompatibilities or whatnot, so there's really no need to touch it until it is unsatisfactory in some meaningful sense.
On 6/18/09 2:42 PM, Orcan Ogetbil wrote:
Hi,
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
In general, this is simply a nonstarter for Doxygen-generated HTML documentation. There is nontrivial (and generally unpredictable) variance in the style sheet that Doxygen generates from one version to the next. Accordingly, the interactions between this style sheet any any that the package upstream may have added cannot be foreseen. If upstream is packaging Doxygen-generated documentation, Fedora should assume it's doing so for a good reason.
On Thu, Jun 18, 2009 at 4:59 PM, Braden McDaniel wrote:
On 6/18/09 2:42 PM, Orcan Ogetbil wrote:
Hi,
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
In general, this is simply a nonstarter for Doxygen-generated HTML documentation. There is nontrivial (and generally unpredictable) variance in the style sheet that Doxygen generates from one version to the next. Accordingly, the interactions between this style sheet any any that the package upstream may have added cannot be foreseen.
That's why the statement starts with "If possible..."
Orcan
On 6/18/09 5:03 PM, Orcan Ogetbil wrote:
On Thu, Jun 18, 2009 at 4:59 PM, Braden McDaniel wrote:
On 6/18/09 2:42 PM, Orcan Ogetbil wrote: Hi, Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately. https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries In general, this is simply a nonstarter for Doxygen-generated HTML documentation. There is nontrivial (and generally unpredictable) variance in the style sheet that Doxygen generates from one version to the next. Accordingly, the interactions between this style sheet any any that the package upstream may have added cannot be foreseen.That's why the statement starts with "If possible..."
It is "possible" in that one can perform the regeneration operation without any obvious errors. It is not "desirable" in that there is a reasonable possibility of nonobvious errors; that is, errors that are detectable only by viewing the generated documentation and furthermore perhaps only readily obvious to someone familiar with how the documentation is intended to be rendered.
The point is that, in light of this, the guideline should be that these files should *not* be regenerated unless doing so affords some tangible improvement.
On Thu, Jun 18, 2009 at 5:17 PM, Braden McDaniel wrote:
On 6/18/09 5:03 PM, Orcan Ogetbil wrote:
On Thu, Jun 18, 2009 at 4:59 PM, Braden McDaniel wrote:
On 6/18/09 2:42 PM, Orcan Ogetbil wrote:
Hi, Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
In general, this is simply a nonstarter for Doxygen-generated HTML documentation. There is nontrivial (and generally unpredictable) variance in the style sheet that Doxygen generates from one version to the next. Accordingly, the interactions between this style sheet any any that the package upstream may have added cannot be foreseen.
That's why the statement starts with "If possible..."
It is "possible" in that one can perform the regeneration operation without any obvious errors. It is not "desirable" in that there is a reasonable possibility of nonobvious errors; that is, errors that are detectable only by viewing the generated documentation and furthermore perhaps only readily obvious to someone familiar with how the documentation is intended to be rendered.
The point is that, in light of this, the guideline should be that these files should *not* be regenerated unless doing so affords some tangible improvement.
Sure, it is just a draft. Things can be added or replaced. Maybe "possible" is not the best word. What would be better? "feasible", "desirable", "affords tangible improvement" ?
Orcan
Braden McDaniel braden@endoframe.com writes:
On 6/18/09 5:03 PM, Orcan Ogetbil wrote:
That's why the statement starts with "If possible..."
... The point is that, in light of this, the guideline should be that these files should *not* be regenerated unless doing so affords some tangible improvement.
I don't think the guidelines should tell people what to do one way or the other. There might be package-specific reasons why regenerating docs is or is not a good thing (eg, this upstream does indeed have a history of forgetting to update derived doc files). I think leaving it to maintainer's discretion is exactly what we should be doing.
More generally, Fedora packagers should be assumed to be competent to make this type of decision. It's not the goal of the guidelines to tell them what to do in exact detail.
Proposal #1 (which just clarified what is considered an executable and thus subject to the already-agreed policy) seemed fine to me.
regards, tom lane
On 6/18/09 5:42 PM, Tom Lane wrote:
Braden McDanielbraden@endoframe.com writes:
On 6/18/09 5:03 PM, Orcan Ogetbil wrote:
That's why the statement starts with "If possible..."
... The point is that, in light of this, the guideline should be that these files should *not* be regenerated unless doing so affords some tangible improvement.
I don't think the guidelines should tell people what to do one way or the other. There might be package-specific reasons why regenerating docs is or is not a good thing (eg, this upstream does indeed have a history of forgetting to update derived doc files). I think leaving it to maintainer's discretion is exactly what we should be doing.
My impression is that the term "guideline" indicates that maintainers should use their discretion. A guideline indicates the default position to take in the absence of information that would lead to a different conclusion.
Otherwise they would be called "rules".
Braden McDaniel braden@endoframe.com writes:
On 6/18/09 5:42 PM, Tom Lane wrote:
I don't think the guidelines should tell people what to do one way or the other.
My impression is that the term "guideline" indicates that maintainers should use their discretion. A guideline indicates the default position to take in the absence of information that would lead to a different conclusion.
When the proposed additions involve the use of the word MUST in capital letters, one is left with the inescapable impression that the intention is to remove one's discretion.
I wouldn't have any problem with an addition that listed reasons why one might or might not want to rebuild derived files (including all the points touched on in this discussion), and then said "use your own judgment about this". That is not what the proposal reads like, though.
regards, tom lane
On Thu, Jun 18, 2009 at 6:00 PM, Tom Lane wrote:
Braden McDaniel writes:
On 6/18/09 5:42 PM, Tom Lane wrote:
I don't think the guidelines should tell people what to do one way or the other.
My impression is that the term "guideline" indicates that maintainers should use their discretion. A guideline indicates the default position to take in the absence of information that would lead to a different conclusion.
When the proposed additions involve the use of the word MUST in capital letters, one is left with the inescapable impression that the intention is to remove one's discretion.
I wouldn't have any problem with an addition that listed reasons why one might or might not want to rebuild derived files (including all the points touched on in this discussion), and then said "use your own judgment about this". That is not what the proposal reads like, though.
Then please give suggestions to improve the proposal draft. There is an obvious confusion for what can be regarded as the pre-built binary. pdf and mo files are binary but are they covered with the existing guideline? At least, this needs to be sorted out. doxygen or other stuff can be left out of the proposal...
Saying "I don't like this proposal because of X" will not help much for improving a draft. Also comments like "I don't agree with this proposal, I want to keep things as is" has little value unless it is said during an FPC vote.
On the other hand, being constructive just helps.
Thanks, Orcan
Orcan Ogetbil oget.fedora@gmail.com writes:
On Thu, Jun 18, 2009 at 6:00 PM, Tom Lane wrote:
I wouldn't have any problem with an addition that listed reasons why one might or might not want to rebuild derived files (including all the points touched on in this discussion), and then said "use your own judgment about this".
Then please give suggestions to improve the proposal draft.
I think we all just did. If that wasn't clear enough for you, then how about this one: drop it completely.
regards, tom lane
On Fri, Jun 19, 2009 at 12:51 AM, Tom Lane wrote:
Orcan Ogetbil writes:
On Thu, Jun 18, 2009 at 6:00 PM, Tom Lane wrote:
I wouldn't have any problem with an addition that listed reasons why one might or might not want to rebuild derived files (including all the points touched on in this discussion), and then said "use your own judgment about this".
Then please give suggestions to improve the proposal draft.
I think we all just did. If that wasn't clear enough for you, then how about this one: drop it completely.
regards, tom lane
Yeah, that will definitely clear off all the confusion. Thank you for your invaluable input. I will consider it.
Orcan
Orcan Ogetbil wrote:
Hi,
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
Any comments, suggestions welcome.
Hmm, both proposals are causing me head-aches, because they are trying to extend the FPG to cover topics this FPG was trying to _intentionally_ leave open.
Let me try to elaborate: This section is aiming at prohibiting shipping prebuilt _Linux_ binaries (applications and libraries). It doesn't aim at prohibiting shipping prebuild "other binaries", such as package documentation, images or movies or sound files.
A requirement to rebuild the later has never existed in Fedora. They should be "rebuildable", if appropriate/feasible/neccessary for proper operation, but this was essentially left to the discretion of individual packagers and individual packages' demands
That said, if there is something which I consider to be in need of further refinement, then its the definition of "binary" as being used in this section of the FPG.
Ralf
On Fri, 2009-06-19 at 06:39 +0200, Ralf Corsepius wrote:
A requirement to rebuild the later has never existed in Fedora. They should be "rebuildable", if appropriate/feasible/neccessary for proper operation, but this was essentially left to the discretion of individual packagers and individual packages' demands
That said, if there is something which I consider to be in need of further refinement, then its the definition of "binary" as being used in this section of the FPG.
A related item, how do you feel about files such as java script, that are generated from .java source via google web toolkit. The java script file is not binary, and while it could be edited, it's not very friendly to editing. However upstream would like to pre-generate the java script files as it removes a rather heavy build time requirement on google web toolkit. So the content /could/ be regenerated, but it doesn't need to be. Would these pre-generated files be acceptable to you in a Fedora package?
On 06/25/2009 02:00 AM, Jesse Keating wrote:
On Fri, 2009-06-19 at 06:39 +0200, Ralf Corsepius wrote:
A requirement to rebuild the later has never existed in Fedora. They should be "rebuildable", if appropriate/feasible/neccessary for proper operation, but this was essentially left to the discretion of individual packagers and individual packages' demands
That said, if there is something which I consider to be in need of further refinement, then its the definition of "binary" as being used in this section of the FPG.
A related item, how do you feel about files such as java script, that are generated from .java source via google web toolkit. The java script file is not binary, and while it could be edited, it's not very friendly to editing. However upstream would like to pre-generate the java script files as it removes a rather heavy build time requirement on google web toolkit. So the content /could/ be regenerated, but it doesn't need to be. Would these pre-generated files be acceptable to you in a Fedora package?
My answer would be no. Documentation is content. java and javascript are code.
-Toshio
On Thu, Jun 18, 2009 at 2:42 PM, Orcan Ogetbil wrote:
Hi,
Toshio asked me to write up a proposal for clarification of the removal of pre-built binaries. I came up with two proposals. I'd like to have them considered and voted separately.
https://fedoraproject.org/wiki/PackagingDrafts/Removal_of_pre-built_binaries
I withdrew the Proposal 2 which was about rebuilding all content.
Based on the input I got, I replaced it with a new proposal that aims to clarify what is meant by "binary". Also I added one sentence that says binary content is not required to be rebuilt.
Please check the new Proposal 2 from the above link. Again, suggestions for improvement of the draft welcome.
Orcan
packaging@lists.fedoraproject.org
-
Braden McDaniel -
Jesse Keating -
Jochen Schmitt -
Matthias Clasen -
Orcan Ogetbil -
Ralf Corsepius -
Susi Lehtola -
Tom Lane -
Toshio Kuratomi