Re: [Fedora-packaging] Proposal: naming convention for Python 3 packages and subpackages
On Thu, 2009-10-29 at 18:42 -0400, John Dennis wrote:
On 10/29/2009 06:27 PM, David Malcolm wrote:
I rather like the idea of standardizing on a "python3-" prefix for _all_ Python 3 module packages and subpackages, even if this leads to inconsistencies with their counterparts in the python 2 stack. It would make the "threeness" of the packages stand out more.
Thoughts?
Initially this sounds good to me. Because python 2 and python 3 are incompatible it's probably important we separate them at the packaging level, this seems like a good approach as any (even with some warts on the corner cases).
However package maintainers might not like the idea of having to maintain double the number of their packages for an interim period and I could see them wanting to have just one package that installs into both the python2 and python3 library locations. Also perhaps we don't want to inflate the number of python packages by 2x. Having not followed this discussion from it's outset I'm wondering if we might want to consider allowing a single python package to support both python versions. I'm sure there are multiple reasons why this is a horrible idea, but I thought I would throw it out for consideration and let it get shot down :-)
My original proposal [1] was that python 3 should be entirely separate from python 2
As a trial run, I took a packaged python module and got it to build with python 3. See: https://bugzilla.redhat.com/show_bug.cgi?id=531648
This is the separate-srpm approach, it would live as a separate package in CVS.
As an experiment, I tried hacking up the same module's specfile so that one build generates both 2 and 3 subpackages. You can see the result here: https://bugzilla.redhat.com/show_bug.cgi?id=531895
This approach requires the package to work with both python 2 and 3, which requires both an upstream maintainer and a downstream Fedora packager that care about both 2 and 3, working from a single source tree.
For supporting both 2 and 3 with a single built RPM (or something involving symlinked .py files?) I don't think that's possible: we don't want to add a python3 dependency to python 2 modules, and the precompiled bytecode optimization of imports requires version-specific .pyc/.pyo files - see bug 531117 (automating this in rpmbuild) and bug 531102 (adding a test to rpmlint to verify that it got it correct).
Dave
[1] See: https://www.redhat.com/archives/fedora-devel-list/2009-October/msg00054.html and https://fedoraproject.org/wiki/Features/Python3F13
On 10/30/2009 01:15 AM, David Malcolm wrote:
On Thu, 2009-10-29 at 18:42 -0400, John Dennis wrote:
On 10/29/2009 06:27 PM, David Malcolm wrote:
I rather like the idea of standardizing on a "python3-" prefix for _all_ Python 3 module packages and subpackages, even if this leads to inconsistencies with their counterparts in the python 2 stack. It would make the "threeness" of the packages stand out more.
Thoughts?
for the
python-<package> -> python3-<package> py<package> -> python3-py<package> (I think we should keep the py to make it easier to locate stuff pygpgme) <package>-python -> python3-<package>
Seem good to me.
But there is a lot of packages installing stuff into /usr/lib/pythonX-Y/site-packages there don't fit 3 cases.
Ex. yum
It is an application, but also an python API used by other packages, how do we handle there cases.
I have attacted the the sorted output from
repoquery -f '/usr/lib/python2.6/site-packages/*'
Tim
On 10/30/2009 09:57 AM, Tim Lauridsen wrote:
On 10/30/2009 01:15 AM, David Malcolm wrote:
On Thu, 2009-10-29 at 18:42 -0400, John Dennis wrote:
On 10/29/2009 06:27 PM, David Malcolm wrote:
I rather like the idea of standardizing on a "python3-" prefix for _all_ Python 3 module packages and subpackages, even if this leads to inconsistencies with their counterparts in the python 2 stack. It would make the "threeness" of the packages stand out more.
Thoughts?
for the
python-<package> -> python3-<package> py<package> -> python3-py<package> (I think we should keep the py to make it easier to locate stuff pygpgme) <package>-python -> python3-<package>
Seem good to me.
But there is a lot of packages installing stuff into /usr/lib/pythonX-Y/site-packages there don't fit 3 cases.
Ex. yum
It is an application, but also an python API used by other packages, how do we handle there cases.
I have attacted the the sorted output from
repoquery -f '/usr/lib/python2.6/site-packages/*'
Tim
I have added a ordered file categorizing the packages in
Tim
On Fri, 2009-10-30 at 10:28 +0100, Tim Lauridsen wrote:
On 10/30/2009 09:57 AM, Tim Lauridsen wrote:
On 10/30/2009 01:15 AM, David Malcolm wrote:
On Thu, 2009-10-29 at 18:42 -0400, John Dennis wrote:
On 10/29/2009 06:27 PM, David Malcolm wrote:
I rather like the idea of standardizing on a "python3-" prefix for _all_ Python 3 module packages and subpackages, even if this leads to inconsistencies with their counterparts in the python 2 stack. It would make the "threeness" of the packages stand out more.
Thoughts?
for the
python-<package> -> python3-<package> py<package> -> python3-py<package> (I think we should keep the py to make it easier to locate stuff pygpgme) <package>-python -> python3-<package>
Seem good to me.
But there is a lot of packages installing stuff into /usr/lib/pythonX-Y/site-packages there don't fit 3 cases.
Ex. yum
It is an application, but also an python API used by other packages, how do we handle there cases.
I have attacted the the sorted output from
repoquery -f '/usr/lib/python2.6/site-packages/*'
Tim
I have added a ordered file categorizing the packages in
Very nice, thanks!
In my email I said "Python 3 module packages and subpackages", and I'm not being very precise about this.
Can a distinction can be drawn between an rpm that "merely" packages a python module? I think that many of our rpms have payloads that are entirely below: /usr/lib/python2.6/site-packages /usr/share/doc/NAME-VERSION I suspect that most of the packages within these lists fall into this category: "Packages starting with 'python-':" "Packages starting with 'Py' or 'py' (but not 'python-')" "Packages ending with '-python':" (most of these seem to be subpackages from a mostly non-python build)
Contrast this with the "None of the above:" category. At a quick look, most of the packages in this list appear to be using python as an implementation detail, in order to get some user-facing job done. For these, I feel we'd port them one-by-one, and the name need not change.
A complication/exception in this last category is for plugin systems and "stacks". For example, yum and its various plugins/extensions don't mention python in their name, but they're written in python 2. Similarly, Django and the various django-foo packages implement the Django web development framework, which happens to be written in python 2 (hopefully will eventually have python 3 support), and "trac-foo". My gut feeling for both of these cases is that we'd want python 2 and 3 versions for a while, so perhaps a python3- prefix is ok. That would give us e.g.: python3-trac-privateticketsplugin python3-TurboGears python3-TurboGears2 python3-yum
I noticed "wxPython" in the "none of the above" naming bucket. This one definitely feels like a support module, rather than a thing to be used in its own right (python bindings to the wxWidgets library). - wxPython3 ? - python3-wx ? etc not sure; maybe depends on upstream.
Does the "purely a module" vs "is something uservisible" vs "is a stack" distinction sound sane?
Dave
On Fri, Oct 30, 2009 at 12:11:11PM -0400, David Malcolm wrote:
Very nice, thanks!
In my email I said "Python 3 module packages and subpackages", and I'm not being very precise about this.
Can a distinction can be drawn between an rpm that "merely" packages a python module? I think that many of our rpms have payloads that are entirely below: /usr/lib/python2.6/site-packages /usr/share/doc/NAME-VERSION I suspect that most of the packages within these lists fall into this category: "Packages starting with 'python-':" "Packages starting with 'Py' or 'py' (but not 'python-')" "Packages ending with '-python':" (most of these seem to be subpackages from a mostly non-python build)
Contrast this with the "None of the above:" category. At a quick look, most of the packages in this list appear to be using python as an implementation detail, in order to get some user-facing job done. For these, I feel we'd port them one-by-one, and the name need not change.
A complication/exception in this last category is for plugin systems and "stacks". For example, yum and its various plugins/extensions don't mention python in their name, but they're written in python 2. Similarly, Django and the various django-foo packages implement the Django web development framework, which happens to be written in python 2 (hopefully will eventually have python 3 support), and "trac-foo". My gut feeling for both of these cases is that we'd want python 2 and 3 versions for a while, so perhaps a python3- prefix is ok. That would give us e.g.: python3-trac-privateticketsplugin python3-TurboGears python3-TurboGears2 python3-yum
I noticed "wxPython" in the "none of the above" naming bucket. This one definitely feels like a support module, rather than a thing to be used in its own right (python bindings to the wxWidgets library).
- wxPython3 ?
- python3-wx ?
etc not sure; maybe depends on upstream.
I'd apply your rules to allow either of the following: - python3-wxPython - python3-wx
Does the "purely a module" vs "is something uservisible" vs "is a stack" distinction sound sane?
It sounds sane but isn't really :-( let's take a few cases, yum, gourmet, setuptools, and bzr.
yum is used by most people as an application. It has both a private library in /usr/share/yum and a public library in %{python_sitelib}/yum. Numerically, most consumers of yum are plugins to yum (so they'd need to be written in the same python version as yum). However, there are users of the public library that are their ownstandalone tools. Those tools could be ported to python3 independently of yum itself.
gourmet is a gtk application. It is largely implemented as a library in %{python_sitelib}/gourmet.
setuptools is two python modules: setuptools and pkg_resources. It also has a script, easy_install. For many end users, easy_install is becoming the way that they use setuptools but for developers, pkg_resources is the primary use. (although developers are coming to use easy_install as well especially coupled with virtualenv).
bzr is a command line version control system and a python library. bzr takes plugins. The library is used in other aps that want to integrate with bzr. (bzr-eclipse, bzr-explorer, etc). Everything except a small script for running the commandline are in %{python_sitelib}/bzrlib.
Ways to deal with this: -Yum * yum could be split into a main yum package that includes /usr/bin/yum and the private library. Whether this is python2 or 3 doesn't matter yet. It would also have python-yum and python3-yum packages that has the public library available to both versions of yum. This makes sense to me because yum is well written and organized in this sense.
-gourmet * gourmet has plugins. It's also installed into the public library location. I don't know of anything that uses the libraries, though (not even external plugins). We could continue to treat this as a pure application at this time and just install a gourmet package where python2/3 is an implementation detail that does not need to be shown.
-setuptools * setuptools can be split to have a subpackage named simply "easy_install" with the easy_install script. This is a bit odd as the script is just a thin wrapper against code in the setuptools module (setuptools is not organized around public and private interfaces like yum is). In this scenario, you'd have python-setuptools, python3-setuptools, easy_install and easy_install-3 (if the maintainers of python-setuptools and python3-setuptools couldn't agree on having a single script). End users are used to getting easy_install from the setuptools package so I don't think this helps them much.
* The packages are called python-setuptools and python3-setuptools and we ignore the fact that there's a script in the package.
-bzr * UGH! I think we'll need to split these packages artificially and somewhat inefficiently: bzr package with the /usr/bin/bzr script. python3-bzrlib and python-bzrlib packages for the library. Note that if the upstream community ports to python3 all at the same time, we wouldn't have to worry about this. So probably we continue to have a monolithic bzr package until bzr/bzrlib is ported to python3. Then, if needed for other front end consumers of bzrlib, we make the split like this: python-bzrlib (python2), bzr (python3), python3-bzrlib (is this one necessary?)
-Toshio
On 10/30/2009 05:11 PM, David Malcolm wrote:
On Fri, 2009-10-30 at 10:28 +0100, Tim Lauridsen wrote:
On 10/30/2009 09:57 AM, Tim Lauridsen wrote:
On 10/30/2009 01:15 AM, David Malcolm wrote:
On Thu, 2009-10-29 at 18:42 -0400, John Dennis wrote:
On 10/29/2009 06:27 PM, David Malcolm wrote:
I rather like the idea of standardizing on a "python3-" prefix for _all_ Python 3 module packages and subpackages, even if this leads to inconsistencies with their counterparts in the python 2 stack. It would make the "threeness" of the packages stand out more.
Thoughts?
for the
python-<package> -> python3-<package> py<package> -> python3-py<package> (I think we should keep the py to make it easier to locate stuff pygpgme) <package>-python -> python3-<package>
Seem good to me.
But there is a lot of packages installing stuff into /usr/lib/pythonX-Y/site-packages there don't fit 3 cases.
Ex. yum
It is an application, but also an python API used by other packages, how do we handle there cases.
I have attacted the the sorted output from
repoquery -f '/usr/lib/python2.6/site-packages/*'
Tim
I have added a ordered file categorizing the packages in
Very nice, thanks!
In my email I said "Python 3 module packages and subpackages", and I'm not being very precise about this.
Can a distinction can be drawn between an rpm that "merely" packages a python module? I think that many of our rpms have payloads that are entirely below: /usr/lib/python2.6/site-packages /usr/share/doc/NAME-VERSION I suspect that most of the packages within these lists fall into this category: "Packages starting with 'python-':" "Packages starting with 'Py' or 'py' (but not 'python-')" "Packages ending with '-python':" (most of these seem to be subpackages from a mostly non-python build)
+1
Contrast this with the "None of the above:" category. At a quick look, most of the packages in this list appear to be using python as an implementation detail, in order to get some user-facing job done. For these, I feel we'd port them one-by-one, and the name need not change.
+1
A complication/exception in this last category is for plugin systems and "stacks". For example, yum and its various plugins/extensions don't mention python in their name, but they're written in python 2. Similarly, Django and the various django-foo packages implement the Django web development framework, which happens to be written in python 2 (hopefully will eventually have python 3 support), and "trac-foo". My gut feeling for both of these cases is that we'd want python 2 and 3 versions for a while, so perhaps a python3- prefix is ok. That would give us e.g.: python3-trac-privateticketsplugin python3-TurboGears python3-TurboGears2 python3-yum
The plan for yum is to stay with python 2.x until RHEL6 is branched and then switch to python 3.x for the yum head branch. (not writen in stone yet ) I don't think we will ever have both a python 2.x and 3.x version in the same distro, so it will fall into the previous category and can keep the name yum.
I noticed "wxPython" in the "none of the above" naming bucket. This one definitely feels like a support module, rather than a thing to be used in its own right (python bindings to the wxWidgets library).
- wxPython3 ?
- python3-wx ?
etc not sure; maybe depends on upstream.
I think i should be python3-wxPython, because people seaching for wxPython will find both and can make the right choice.
Does the "purely a module" vs "is something uservisible" vs "is a stack" distinction sound sane?
Yes.
Tim
python-devel@lists.fedoraproject.org
-
David Malcolm -
Tim Lauridsen -
Toshio Kuratomi