I've added Python Virtual Provides to the list of draft guidelines.
https://fedoraproject.org/wiki/PackagingDrafts/Python
There are two parts: 1) introduce virtual provides for python eggs. Copying from the way we do rubygems, this would be:
Provides: pythonegg(SQLAlchemy) = %{version} Requires: pythonegg(SQLAlchemy) >= 0.3.11
2) introduce virtualprovides for normal python modules:
Provides: python(sqlalchemy) = %{version} Requires: python(sqlalchemy)
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
Since this will be a far reaching change, should it be discussed on fedora-devel-list first or whether we want to do both parts?
-Toshio
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines. https://fedoraproject.org/wiki/PackagingDrafts/Python
[snip]
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
To help maintain the sanity of everyone building packages, can't we just auto-generate these? I know there's the old patch floating around somewhere for python deps that we should be using but haven't yet instituted. And extending that to work with eggs also seems to make sense. And then we avoid everyone having to add stuff to their spec files
Jeremy
Jeremy Katz wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines. https://fedoraproject.org/wiki/PackagingDrafts/Python
[snip]
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
To help maintain the sanity of everyone building packages, can't we just auto-generate these? I know there's the old patch floating around somewhere for python deps that we should be using but haven't yet instituted. And extending that to work with eggs also seems to make sense. And then we avoid everyone having to add stuff to their spec files
Err, that's why I said the motivation for this was that Dave Malcolm wrote a script to do that for eggs :-)
-Toshio
On Mon, 2008-06-16 at 16:39 -0400, Toshio Kuratomi wrote:
Jeremy Katz wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines. https://fedoraproject.org/wiki/PackagingDrafts/Python
[snip]
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
To help maintain the sanity of everyone building packages, can't we just auto-generate these? I know there's the old patch floating around somewhere for python deps that we should be using but haven't yet instituted. And extending that to work with eggs also seems to make sense. And then we avoid everyone having to add stuff to their spec files
Err, that's why I said the motivation for this was that Dave Malcolm wrote a script to do that for eggs :-)
If it's being done automatically, it's not really packaging policy -- it's just part of things working :)
Jeremy
Jeremy Katz wrote:
On Mon, 2008-06-16 at 16:39 -0400, Toshio Kuratomi wrote:
Jeremy Katz wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines. https://fedoraproject.org/wiki/PackagingDrafts/Python
[snip]
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
To help maintain the sanity of everyone building packages, can't we just auto-generate these? I know there's the old patch floating around somewhere for python deps that we should be using but haven't yet instituted. And extending that to work with eggs also seems to make sense. And then we avoid everyone having to add stuff to their spec files
Err, that's why I said the motivation for this was that Dave Malcolm wrote a script to do that for eggs :-)
If it's being done automatically, it's not really packaging policy -- it's just part of things working :)
The packaging policy portion is:
* Are Virtual Provides the way to go (I say yes and so far no one's shot me down :-) * What should the virtual provides be named? * Should we do the part where we can only manage to pull Provides out automatically but not Requires? * Should we try harder to make subpackages listed?
-Toshio
On Mon, 2008-06-16 at 19:20 -0400, Toshio Kuratomi wrote:
Jeremy Katz wrote:
On Mon, 2008-06-16 at 16:39 -0400, Toshio Kuratomi wrote:
Jeremy Katz wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines. https://fedoraproject.org/wiki/PackagingDrafts/Python
[snip]
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
To help maintain the sanity of everyone building packages, can't we just auto-generate these? I know there's the old patch floating around somewhere for python deps that we should be using but haven't yet instituted. And extending that to work with eggs also seems to make sense. And then we avoid everyone having to add stuff to their spec files
Err, that's why I said the motivation for this was that Dave Malcolm wrote a script to do that for eggs :-)
If it's being done automatically, it's not really packaging policy -- it's just part of things working :)
The packaging policy portion is:
- Are Virtual Provides the way to go (I say yes and so far no one's shot
me down :-)
I don't really know how else you'd do it...
- What should the virtual provides be named?
Bikeshed :-) But the proposal matches what we do for everything else
- Should we do the part where we can only manage to pull Provides out
automatically but not Requires?
Obviously it'd be nice if we could get Requires done automatically too, but it doesn't hurt to give the provides until then.
- Should we try harder to make subpackages listed?
How so? I might be missing something here but I don't see anything that subpackages are especially relevant for? You mean like if a module is split across subpackages?
Jeremy
Jeremy Katz wrote:
On Mon, 2008-06-16 at 19:20 -0400, Toshio Kuratomi wrote:
Jeremy Katz wrote:
On Mon, 2008-06-16 at 16:39 -0400, Toshio Kuratomi wrote:
Jeremy Katz wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines. https://fedoraproject.org/wiki/PackagingDrafts/Python
[snip]
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
To help maintain the sanity of everyone building packages, can't we just auto-generate these? I know there's the old patch floating around somewhere for python deps that we should be using but haven't yet instituted. And extending that to work with eggs also seems to make sense. And then we avoid everyone having to add stuff to their spec files
Err, that's why I said the motivation for this was that Dave Malcolm wrote a script to do that for eggs :-)
If it's being done automatically, it's not really packaging policy -- it's just part of things working :)
The packaging policy portion is:
- Are Virtual Provides the way to go (I say yes and so far no one's shot
me down :-)
I don't really know how else you'd do it...
- What should the virtual provides be named?
Bikeshed :-) But the proposal matches what we do for everything else
- Should we do the part where we can only manage to pull Provides out
automatically but not Requires?
Obviously it'd be nice if we could get Requires done automatically too, but it doesn't hurt to give the provides until then.
- Should we try harder to make subpackages listed?
How so? I might be missing something here but I don't see anything that subpackages are especially relevant for? You mean like if a module is split across subpackages?
So, eggs seem pretty straightforward to me. They establish what they provide and what they depend on in their metadata. But the python() namespace is not so easy. On the PackagingDraft/Python page I mention that one thing we could try is to make a Provides: python(foo) for every module in the site-packages directories. This is relatively straightforward to automate but misses subpackages. The example I brought up is bzr-gtk::
bzr has:: %{python_sitearch}/bzrlib And so it Provides: python(bzrlib)
bzr-gtk has:: %{python_sitearch}/bzrlib/plugins/gtk And in this scheme it wouldn't have any automated Provides.
Other bzr plugins can require that bzr-gtk is installed, though, so they'll have to write their requirements on the package name instead of a virtual provide.
Another example that's not so plugin/program based is toscawidgets. It creates the directory: %{python_sitearch}/tw
Packages which create widgets that work within tosca can install into subdirectories of that so this heuristic won't find them.
-Toshio
On Tue, 2008-06-17 at 06:37 -0400, Toshio Kuratomi wrote:
So, eggs seem pretty straightforward to me. They establish what they provide and what they depend on in their metadata. But the python() namespace is not so easy. On the PackagingDraft/Python page I mention that one thing we could try is to make a Provides: python(foo) for every module in the site-packages directories. This is relatively straightforward to automate but misses subpackages. The example I brought up is bzr-gtk::
Aha, okay. I think that the right thing here is to get the first level and then we can later worry about the secondary case of subpackages. I mean, you could do Provides: python(bzrlib.plugins.gtk) or some-such, but I suspect that might get a bit out of hand. And I'm also not sure how common of a case it is to need to care about
Jeremy
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines.
https://fedoraproject.org/wiki/PackagingDrafts/Python
There are two parts:
- introduce virtual provides for python eggs. Copying from the way we
do rubygems, this would be:
Provides: pythonegg(SQLAlchemy) = %{version} Requires: pythonegg(SQLAlchemy) >= 0.3.11
- introduce virtualprovides for normal python modules:
Provides: python(sqlalchemy) = %{version} Requires: python(sqlalchemy)
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
I ran into various inconsistencies between the versions listed in the egg requires.txt file and the rpm specfile, and thought that the latter ought to be autogenerated from the former.
First pass at a script attached to bug 451228, and I posted this to fedora-python-devel-list: https://www.redhat.com/archives/fedora-python-devel-list/2008-June/msg00002....
Obvious mistake I made was to try to store the mapping from python modules to package names, instead, the script ought to generate some kind of virtual provides as in Toshio's suggestion above. Should be simple to update the script
Since this will be a far reaching change, should it be discussed on fedora-devel-list first or whether we want to do both parts?
Could we simply put the script in a place where it's available to the maintainer, so they can opt-in and have it generate the deps?
Hope this helps Dave
On Mon, 2008-06-16 at 15:56 -0400, David Malcolm wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines.
https://fedoraproject.org/wiki/PackagingDrafts/Python
There are two parts:
- introduce virtual provides for python eggs. Copying from the way we
do rubygems, this would be:
Provides: pythonegg(SQLAlchemy) = %{version} Requires: pythonegg(SQLAlchemy) >= 0.3.11
- introduce virtualprovides for normal python modules:
Provides: python(sqlalchemy) = %{version} Requires: python(sqlalchemy)
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg dependency information as part of the rpm build process.
I ran into various inconsistencies between the versions listed in the egg requires.txt file and the rpm specfile, and thought that the latter ought to be autogenerated from the former.
First pass at a script attached to bug 451228, and I posted this to fedora-python-devel-list: https://www.redhat.com/archives/fedora-python-devel-list/2008-June/msg00002....
Obvious mistake I made was to try to store the mapping from python modules to package names, instead, the script ought to generate some kind of virtual provides as in Toshio's suggestion above. Should be simple to update the script
Implemented, attached to bug 451228. This generates the following for the TurboGears egg (telling it to use the optional SQLAlchemy stuff):
# Autogenerated metadata from "requires.py /usr/lib/python2.4/site-packages/TurboGears-1.0.4.4-py2.4.egg-info SQLAlchemy" Provides: pythonegg(TurboGears) = 1.0.4.4 Requires: pythonegg(CherryPy) >= 2.3.0 Requires: pythonegg(CherryPy) < 3.0.0alpha Requires: pythonegg(DecoratorTools) >= 1.4 Requires: pythonegg(FormEncode) >= 0.7.1 Requires: pythonegg(PasteScript) >= 1.6.2 Requires: pythonegg(RuleDispatch) >= 0.5a0.dev-r2303 Requires: pythonegg(setuptools) >= 0.6c2 Requires: pythonegg(simplejson) >= 1.3 Requires: pythonegg(TurboCheetah) >= 1.0 Requires: pythonegg(TurboJson) >= 1.1.2 Requires: pythonegg(TurboKid) >= 1.0.4 Requires: pythonegg(SQLAlchemy) >= 0.3.10
Is the dash in the RuleDispatch version likely to cause problems?
Since this will be a far reaching change, should it be discussed on fedora-devel-list first or whether we want to do both parts?
Could we simply put the script in a place where it's available to the maintainer, so they can opt-in and have it generate the deps?
Hope this helps Dave
-- Fedora-packaging mailing list Fedora-packaging@redhat.com https://www.redhat.com/mailman/listinfo/fedora-packaging
On Mon, Jun 16, 2008 at 4:31 PM, David Malcolm dmalcolm@redhat.com wrote:
On Mon, 2008-06-16 at 15:56 -0400, David Malcolm wrote:
On Mon, 2008-06-16 at 14:30 -0400, Toshio Kuratomi wrote:
I've added Python Virtual Provides to the list of draft guidelines.
https://fedoraproject.org/wiki/PackagingDrafts/Python
There are two parts:
- introduce virtual provides for python eggs. Copying from the way we
do rubygems, this would be:
Provides: pythonegg(SQLAlchemy) = %{version} Requires: pythonegg(SQLAlchemy) >= 0.3.11
- introduce virtualprovides for normal python modules:
Provides: python(sqlalchemy) = %{version} Requires: python(sqlalchemy)
The motivation for this is that David Malcolm (dmalcolm) wrote a proof of concept script to show how easy it would be to extract egg
dependency
information as part of the rpm build process.
I ran into various inconsistencies between the versions listed in the egg requires.txt file and the rpm specfile, and thought that the latter ought to be autogenerated from the former.
First pass at a script attached to bug 451228, and I posted this to fedora-python-devel-list:
https://www.redhat.com/archives/fedora-python-devel-list/2008-June/msg00002....
Obvious mistake I made was to try to store the mapping from python modules to package names, instead, the script ought to generate some kind of virtual provides as in Toshio's suggestion above. Should be simple to update the script
Implemented, attached to bug 451228. This generates the following for the TurboGears egg (telling it to use the optional SQLAlchemy stuff):
# Autogenerated metadata from "requires.py /usr/lib/python2.4/site-packages/TurboGears-1.0.4.4-py2.4.egg-info SQLAlchemy" Provides: pythonegg(TurboGears) = 1.0.4.4 Requires: pythonegg(CherryPy) >= 2.3.0 Requires: pythonegg(CherryPy) < 3.0.0alpha Requires: pythonegg(DecoratorTools) >= 1.4 Requires: pythonegg(FormEncode) >= 0.7.1 Requires: pythonegg(PasteScript) >= 1.6.2 Requires: pythonegg(RuleDispatch) >= 0.5a0.dev-r2303 Requires: pythonegg(setuptools) >= 0.6c2 Requires: pythonegg(simplejson) >= 1.3 Requires: pythonegg(TurboCheetah) >= 1.0 Requires: pythonegg(TurboJson) >= 1.1.2 Requires: pythonegg(TurboKid) >= 1.0.4 Requires: pythonegg(SQLAlchemy) >= 0.3.10
Is the dash in the RuleDispatch version likely to cause problems?
It shouldn't, provided the package which this lives in has a matching provides.
-- Jes
"DM" == David Malcolm dmalcolm@redhat.com writes:
DM> Requires: pythonegg(RuleDispatch) >= 0.5a0.dev-r2303
DM> Is the dash in the RuleDispatch version likely to cause problems?
Not so much the dash as the fact that there is a huge chance that with such incredibly poor versioning practices we're likely to see things fail to sort properly in the future, leading to versions which go backwards rpm-wise.
- J<
Jason L Tibbitts III wrote:
"DM" == David Malcolm dmalcolm@redhat.com writes:
DM> Requires: pythonegg(RuleDispatch) >= 0.5a0.dev-r2303
DM> Is the dash in the RuleDispatch version likely to cause problems?
Not so much the dash as the fact that there is a huge chance that with such incredibly poor versioning practices we're likely to see things fail to sort properly in the future, leading to versions which go backwards rpm-wise.
Well, the package will already have that problem. This is the prettiest I could make cleaning up the version::
%define nonnumeric_relstring a0.dev-r2303 Name: python-ruledispatch Version: 0.5 Release: 0.1.%{nonnumeric_relstring} Provides: pythonegg(RuleDispatch) = %{version}%{nonnumeric_relstring}
Since the script Dave provided makes constructs the Provides line for us, using the script with these kinds of versions doesn't really make our lives harder.
-Toshio
"TK" == Toshio Kuratomi a.badger@gmail.com writes:
TK> Well, the package will already have that problem.
Perhaps, but then a human is involved to do things like rationalize the version or bump epoch when required. If a script is pulling this info out of the egg file then it won't have the benefit of such oversight.
I guess I'm assuming that this works like it does for Perl, where the packager needs to specify build dependencies but essentially doesn't have to specify any kind of runtime dependency.
- J<
packaging@lists.fedoraproject.org
-
David Malcolm -
Jason L Tibbitts III -
Jeremy Katz -
Jesse Keating -
Toshio Kuratomi