So, I have a number of macros that taken together create a generic filtering system for our autoprov/dep system.
http://fedorapeople.org/~cweyl/macros.perl.local http://fedorapeople.org/~cweyl/macros.perl (suitable for ~/.rpmmacros inclusion)
As this has a not insignificant amount to do with packaging, I'm looking for some feedback both on the macros themselves as well as on the best approach to get these included by default. (File a feature and get it approved? File a bug against rpm? A small ritual sacrifice? :))
Thanks! -Chris
On Fri, May 29, 2009 at 12:18 AM, Chris Weyl cweyl@alumni.drew.edu wrote:
So, I have a number of macros that taken together create a generic filtering system for our autoprov/dep system.
http://fedorapeople.org/~cweyl/macros.perl.localhttp://fedorapeople.org/%7Ecweyl/macros.perl.local http://fedorapeople.org/~cweyl/macros.perlhttp://fedorapeople.org/%7Ecweyl/macros.perl(suitable for ~/.rpmmacros inclusion)
As this has a not insignificant amount to do with packaging, I'm looking for some feedback both on the macros themselves as well as on the best approach to get these included by default. (File a feature and get it approved? File a bug against rpm? A small ritual sacrifice? :))
Following up to myself here, I've created a feature page that I intend to submit to FESCo:
https://fedoraproject.org/wiki/Features/BetterRpmAutoReqProvFiltering
Any feedback, good or bad, would be appreciated.
-Chris
"CW" == Chris Weyl cweyl@alumni.drew.edu writes:
CW> Following up to myself here, I've created a feature page that I CW> intend to submit to FESCo:
Well, honestly I think that FESCo would just kick it back to the FPC, but if you prefer to submit it that way then I guess I'll keep it off of the FPC agenda.
- J<
On Mon, Jun 1, 2009 at 12:59 PM, Jason L Tibbitts III tibbs@math.uh.eduwrote:
"CW" == Chris Weyl cweyl@alumni.drew.edu writes:
CW> Following up to myself here, I've created a feature page that I CW> intend to submit to FESCo:
Well, honestly I think that FESCo would just kick it back to the FPC, but if you prefer to submit it that way then I guess I'll keep it off of the FPC agenda.
If taking it through the FPC is the right way to do it, then let's do it that way. I only created the feature page as I wasn't seeing any feedback as to how it should be done.
How do I get it on the FPC agenda? Simply by asking here? :)
-Chris
On Sunday 31 May 2009, Chris Weyl wrote:
https://fedoraproject.org/wiki/Features/BetterRpmAutoReqProvFiltering
Any feedback, good or bad, would be appreciated.
Jeff Johnson notes in https://bugzilla.redhat.com/show_bug.cgi?id=502403#c1 that PLD has a general mechanism for that as well already deployed. The feature page does not mention whether existing solutions have been examined, and if they have, why one of them has not been adopted but instead a new one has been implemented. I suggest adding such a section and examining at least PLD, SUSE and Mandriva.
On Thu, Jun 4, 2009 at 8:58 AM, Ville Skyttä ville.skytta@iki.fi wrote:
On Sunday 31 May 2009, Chris Weyl wrote:
https://fedoraproject.org/wiki/Features/BetterRpmAutoReqProvFiltering
Any feedback, good or bad, would be appreciated.
Jeff Johnson notes in https://bugzilla.redhat.com/show_bug.cgi?id=502403#c1 that PLD has a general mechanism for that as well already deployed. The feature page does not mention whether existing solutions have been examined, and if they have, why one of them has not been adopted but instead a new one has been implemented. I suggest adding such a section and examining at least PLD, SUSE and Mandriva.
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
I've added this, though it's brief and based on a 30-minute examination of the two:
https://fedoraproject.org/wiki/Features/BetterRpmAutoReqProvFiltering/OtherF...
The "PLD" one, as near as I can figure, is a direct implementation in the rpm5.org codebase. (Corrections welcome.)
Tibbs suggested bringing this before the FPC... It sounds like it makes sense to do it this way, though I'm still unclear as to how to get it on the agenda.
-Chris
On Thu, Jun 4, 2009 at 6:47 PM, Chris Weyl cweyl@alumni.drew.edu wrote:
I've added this, though it's brief and based on a 30-minute examination of the two:
https://fedoraproject.org/wiki/Features/BetterRpmAutoReqProvFiltering/OtherF...
And might be reasonably be termed "biased". :) Edits welcome.
-Chris
On Thu, 4 Jun 2009, Chris Weyl wrote:
On Thu, Jun 4, 2009 at 6:47 PM, Chris Weyl cweyl@alumni.drew.edu wrote: I've added this, though it's brief and based on a 30-minute examination of the two:
https://fedoraproject.org/wiki/Features/BetterRpmAutoReqProvFiltering/OtherFilte ringSystemsAnd might be reasonably be termed "biased". :) Edits welcome.
As a generic filtering mechanism this is a no-go, doing this breaks multilib as Fedora uses it:
%global _use_internal_dependency_generator 0 \
Multilib-safe filtering mechanism needs to be bolted to the internal dependency generator, patches welcome... The other alternative would be killing the multilib coloring which would "only" require splitting all relevant packages to separate -libs etc to avoid elf32/elf64 binary collisions (which the multilib coloring currently hides)
- Panu -
On Fri, Jun 5, 2009 at 12:30 AM, Panu Matilainen pmatilai@laiskiainen.orgwrote:
As a generic filtering mechanism this is a no-go, doing this breaks multilib as Fedora uses it:
%global _use_internal_dependency_generator 0 \
Multilib-safe filtering mechanism needs to be bolted to the internal dependency generator, patches welcome... The other alternative would be killing the multilib coloring which would "only" require splitting all relevant packages to separate -libs etc to avoid elf32/elf64 binary collisions (which the multilib coloring currently hides)
AFAIK disabling the internal dependency generator is the only way to filter dependencies of this nature (namely, solib provides). Is there some way we could either do the necessary file coloring external to rpm, segregate the dependency and coloring functionality, or gain access to modify the auto-prov/dep results (if not functionality) using the embedded lua interperter?
Also, if I read this correctly, the only impact disabling the internal dependency generator has on multilib is when elf32/64 binaries are actually present in the package, right?
-Chris
On Fri, 5 Jun 2009, Chris Weyl wrote:
On Fri, Jun 5, 2009 at 12:30 AM, Panu Matilainen pmatilai@laiskiainen.org wrote:
As a generic filtering mechanism this is a no-go, doing this breaks multilib as Fedora uses it:
%global _use_internal_dependency_generator 0 \
Multilib-safe filtering mechanism needs to be bolted to the internal dependency generator, patches welcome... The other alternative would be killing the multilib coloring which would "only" require splitting all relevant packages to separate -libs etc to avoid elf32/elf64 binary collisions (which the multilib coloring currently hides)
AFAIK disabling the internal dependency generator is the only way to filter dependencies of this nature (namely, solib provides).
Currently yes, hence the "patches welcome" :)
Is there some way we could either do the necessary file coloring external to rpm, segregate the dependency and coloring functionality, or gain access to modify the auto-prov/dep results (if not functionality) using the embedded lua interperter?
Doing the coloring externally would require changing the external dependency generating quite dramatically as the internal coloring and dep generation is per-file, whereas external dependencies are per package. Separating the coloring from dependency generation should be quite possible but results in yet-another package style variant. Whether it matters ... dunno. But that'd be still be working around the fact that what is really needed is a proper dependency filtering/customization mechanism that doesn't require jumping through 15 hoops.
Also, if I read this correctly, the only impact disabling the internal dependency generator has on multilib is when elf32/64 binaries are actually present in the package, right?
Yup, packages with elf32/64 binaries are the only cases where it really matters. The internal dep generator adds some other things besides the coloring: file classes and per-file dependency information that the external one cannot do, but that extra info is not really used by anything (at least in part because its not available everywhere).
- Panu -
On Wed, Jun 10, 2009 at 4:57 AM, Panu Matilainen pmatilai@laiskiainen.orgwrote:
On Fri, 5 Jun 2009, Chris Weyl wrote:
AFAIK disabling the internal dependency generator is the only way to
filter dependencies of this nature (namely, solib provides).
Currently yes, hence the "patches welcome" :)
Just making sure I was understanding that correctly :)
Is there some way we could either do the necessary file coloring external
to rpm, segregate the dependency and coloring functionality, or gain access to modify the auto-prov/dep results (if not functionality) using the embedded lua interperter?
Doing the coloring externally would require changing the external dependency generating quite dramatically as the internal coloring and dep generation is per-file, whereas external dependencies are per package. Separating the coloring from dependency generation should be quite possible but results in yet-another package style variant. Whether it matters ... dunno. But that'd be still be working around the fact that what is really needed is a proper dependency filtering/customization mechanism that doesn't require jumping through 15 hoops.
So, can we expose the generated dep info to the embedded lua interperter, the way sources/patches are, and make it possible to selectively add/remove deps from LUA as well?
The few bits of documentation I've been able to find w.r.t. LUA in RPM refers tantalizingly to "hooks", yet doesn't actually describe how to use them... So I'm unsure if this can be leveraged here. From looking at the RPM source, it also doesn't look as if the same table structures built up for sources & patches are done for deps as well.
(I'm focusing on LUA here as this seems to potentially be the sanest / easiest / most flexible way to get at this info w/o disabling the internal dep generator.)
Also, if I read this correctly, the only impact disabling the internal
dependency generator has on multilib is when elf32/64 binaries are actually present in the package, right?
Yup, packages with elf32/64 binaries are the only cases where it really matters. The internal dep generator adds some other things besides the coloring: file classes and per-file dependency information that the external one cannot do, but that extra info is not really used by anything (at least in part because its not available everywhere).
Ok, cool. :) So what I'm taking away from the above is that we can implement this system as-is without yum/rpm breakage in the vast majority of situations this is called for (that is, plugin-style packages without elf32/64 binaries not having -libs subpackages). I know it's not perfect, but it's better than the several mystical filtering incantations we have right now.
-Chris
On Wednesday 10 June 2009, Panu Matilainen wrote:
The internal dep generator adds some other things besides the coloring: file classes and per-file dependency information that the external one cannot do, but that extra info is not really used by anything (at least in part because its not available everywhere).
rpmlint >= 0.86 does use file classes. But it falls back to finding that stuff out itself if it's not available in package headers (python-magic is required for that).
On Fri, Jun 5, 2009 at 12:30 AM, Panu Matilainen pmatilai@laiskiainen.orgwrote:
As a generic filtering mechanism this is a no-go, doing this breaks multilib as Fedora uses it:
%global _use_internal_dependency_generator 0 \
One thing I thought was obvious here is that the above line is only ever invoked if %filter_setup is actually invoked. The entire macro is %defined, not %global, and is wrapped in an %expand macro, so it will only ever be evaluated if a spec actually uses it, along the lines of:
%define filter_setup %{expand: -internal-dep-overrides-here- }
-Chris
"CW" == Chris Weyl cweyl@alumni.drew.edu writes:
CW> Tibbs suggested bringing this before the FPC... It sounds like it CW> makes sense to do it this way, though I'm still unclear as to how CW> to get it on the agenda.
Well, I do post the instructions with the FPC agenda that I post to fedora-devel-list before every meeting: Read https://fedoraproject.org/wiki/Packaging/Committee#Guideline_Change_Procedur..., Make a draft Add it to https://fedoraproject.org/wiki/PackagingDrafts/DraftsTodo
But fear not, because I made sure it came up at our meeting on Tuesday. Basically, the concept is sound but we'd like to see it in the form of a set of packaging guidelines, and we'd like the changes to the perl provides behavior that are hidden down in the macro file to be split out and discussed separately.
The discussion starts at https://fedoraproject.org/wiki/Packaging:Minutes/20090602#t12:39
- J<
On Thu, Jun 4, 2009 at 9:42 PM, Jason L Tibbitts III tibbs@math.uh.eduwrote:
"CW" == Chris Weyl cweyl@alumni.drew.edu writes:
CW> Tibbs suggested bringing this before the FPC... It sounds like it CW> makes sense to do it this way, though I'm still unclear as to how CW> to get it on the agenda.
Well, I do post the instructions with the FPC agenda that I post to fedora-devel-list before every meeting: Read https://fedoraproject.org/wiki/Packaging/Committee#Guideline_Change_Procedur... , Make a draft Add it to https://fedoraproject.org/wiki/PackagingDrafts/DraftsTodo
But fear not, because I made sure it came up at our meeting on Tuesday. Basically, the concept is sound but we'd like to see it in the form of a set of packaging guidelines, and we'd like the changes to the perl provides behavior that are hidden down in the macro file to be split out and discussed separately.
The discussion starts at https://fedoraproject.org/wiki/Packaging:Minutes/20090602#t12:39
Excellent, thank you :)
I read through the minutes, and rewrote it as a guidelines draft and added it to DraftsTodo; I took out the %perl_default_filtering bits and can either propose those separately or can just take them up as potentially being included as an /etc/rpm/macros.perl in the main perl package. I also included an admonition to be careful of using this sort of filtering in a multilib situation, due to Panu's comments.
https://fedoraproject.org/wiki/PackagingDrafts/AutoProvidesAndRequiresFilter...
The revised macros (no changes, really, just split out the %perl_... bit) are out at:
http://fedorapeople.org/~cweyl/macros.filtering http://fedorapeople.org/~cweyl/macros.perl
-Chris
packaging@lists.fedoraproject.org
-
Chris Weyl -
Jason L Tibbitts III -
Panu Matilainen -
Ville Skyttä