Greetings everyone, we had a productive session at flock and actually covered everything that was in my email message about a potential agenda. Here's the notes:
https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines
Note that we now need to:
1) Turn the notes into a guideline draft for the FPC 2) discuss some of the issues more broadly (here and in the packaging mailing list). 3) change some of the conclusions we had at flock due to that discussion. For instance, I discussed the pypy idea we had at flock (symlinking the purelib site-packages into pypy's site-packages) with one of the upstream pypy developers and he didn't feel it was a good idea. With that feedback, I think we want to go back to the drawing board with that one.
-Toshio
----- Original Message -----
Greetings everyone, we had a productive session at flock and actually covered everything that was in my email message about a potential agenda. Here's the notes:
https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines
Thanks for posting this, Toshio. Few comments: - The wheel proposal seems nice, although we should probably also cover the case where upstream _only_ distributes wheel and not tarball (shouldn't be a big deal). - Shebang lines - So, it seems that we should both check the pip shebang line and point %{__python} to %{__python2}, right? - Naming of Python modules - this section is a bit confusing for me. What is the outcome/which way does it suggest to use? (I'm personally in favour of SRPM python-setuptools and RPMS python{2,3}-setuptools - with python2-setuptools possibly having "Provides: python-setuptools").
Note that we now need to:
- Turn the notes into a guideline draft for the FPC
- discuss some of the issues more broadly (here and in the packaging mailing list).
- change some of the conclusions we had at flock due to that discussion. For instance, I discussed the pypy idea we had at flock (symlinking the purelib site-packages into pypy's site-packages) with one of the upstream pypy developers and he didn't feel it was a good idea. With that feedback, I think we want to go back to the drawing board with that one.
-Toshio
On Wed, Aug 21, 2013 at 02:09:33AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
Greetings everyone, we had a productive session at flock and actually covered everything that was in my email message about a potential agenda. Here's the notes:
https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines
Thanks for posting this, Toshio. Few comments:
- The wheel proposal seems nice, although we should probably also cover
the case where upstream _only_ distributes wheel and not tarball (shouldn't be a big deal).
I'll have to look into this a bit. I'm not certain that a wheel contains all the source. I'm almost certain that it wouldn't fit under "preferred form of modification". Depending on those, we might want to tell people tarballs, followed by source checkouts are prefered/okay. Wheels might be third on the list or might be banned as a Source: line.
- Shebang lines - So, it seems that we should both check the pip shebang
line and point %{__python} to %{__python2}, right?
Nick's idea was that pip's shebang line would be sufficient but I don't see a problem with doing both. I think that pointing %{__python} at %{__python2} will catch a few stragglers that just making sure pip is correct would not. So +1 to do both from me.
- Naming of Python modules - this section is a bit confusing for me. What
is the outcome/which way does it suggest to use? (I'm personally in favour of SRPM python-setuptools and RPMS python{2,3}-setuptools - with python2-setuptools possibly having "Provides: python-setuptools").
We decided that we'd let maintainers decide whether to ship single srpm with subpackages or multiple srpms. The implementation of subpackages was basically what you describe above. The difference in single srpm vs subpackages from the current guideline is that we'd give less direction to packagers about when to use one over the other. Currently the Guidelines say: """ The rule to choose which method [single srpm or multiple] is simple: if the python2 and python3 modules are distributed as a single tarball (many times as a single directory of source where the /usr/bin/2to3 program is used to transform the code at buildtime) then you must package them as subpackages built from a single SRPM. If they come in multiple tarballs then package them from multiple SRPMs. """
We'd change this section to say that maintainers are allowed to choose which method and then go into the sections of Pros and Cons from the current guidelines.
One thing that I've thought of -- If we're going to do another stack for pypy, how do we want to fit that in? If naming is like:
pypy2-setuptools pypy3-setuptools
Then maybe we want a separate srpm for pypy modules. Otherwise how will users know to look for python-setuptools in bugzilla.
If naming is like:
python-pypy2-setuptools python-pypy3-setuptools
The naming seems ugly and I think there's still going to be user confusion in bugzilla... So yeah -- I'm not sure if there's a naming convention for pypy modules that would make subpackages workable.
If someone can come up with a better proposal for packaging pypy modules, that might sidestep these issues.
-Toshio
One thing that I've thought of -- If we're going to do another stack for pypy, how do we want to fit that in? If naming is like:
pypy2-setuptools pypy3-setuptools
Then maybe we want a separate srpm for pypy modules. Otherwise how will users know to look for python-setuptools in bugzilla.
If naming is like:
python-pypy2-setuptools python-pypy3-setuptools
The naming seems ugly and I think there's still going to be user confusion in bugzilla... So yeah -- I'm not sure if there's a naming convention for pypy modules that would make subpackages workable.
If someone can come up with a better proposal for packaging pypy modules, that might sidestep these issues.
The latter is awful. What we actually somehow could do is to generate pypy2-foo from python-foo SRPM and patch/tweak Bugzilla to allow aliases. Then simple add (by some script) {python,pypy}{2,3}-foo to link to python-foo so users are not confused.
Miro
On Wed, Aug 21, 2013 at 12:44:45PM +0200, Miro Hrončok wrote:
The latter is awful. What we actually somehow could do is to generate pypy2-foo from python-foo SRPM and patch/tweak Bugzilla to allow aliases. Then simple add (by some script) {python,pypy}{2,3}-foo to link to python-foo so users are not confused.
We don't control bugzilla (and bugzilla is not something we really want to run because it is pretty monstrous) so probably anything that requires modifications to it is out.
We might be able to have a separate bugzilla component that is for the same git tree (as we control the git trees and the packagedb) and just assign the same users to be owner and cc to both components in bugzilla but that's a special case that would require a bit of coding in the pkgdb... so I'm loathe to say that's a viable option either....
-Toshio
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/21/2013 09:08 PM, Toshio Kuratomi wrote:
On Wed, Aug 21, 2013 at 12:44:45PM +0200, Miro Hrončok wrote:
The latter is awful. What we actually somehow could do is to generate pypy2-foo from python-foo SRPM and patch/tweak Bugzilla to allow aliases. Then simple add (by some script) {python,pypy}{2,3}-foo to link to python-foo so users are not confused.
We don't control bugzilla (and bugzilla is not something we really want to run because it is pretty monstrous) so probably anything that requires modifications to it is out.
The RH Bugzilla team work are part of the same internal tools group as me, so if there's a small targeted change that would help, I can at least talk to them about it. It would need to be fairly small though - for some strange reason, their efforts are mostly focused on "make it go faster" at the moment :)
I suspect adding component aliases wouldn't qualify as "small", though, since that's the kind of data model change that has significant repercussions throughout the UI and query generation system.
Cheers, Nick.
- -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/21/2013 07:27 PM, Toshio Kuratomi wrote:
On Wed, Aug 21, 2013 at 02:09:33AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
Greetings everyone, we had a productive session at flock and actually covered everything that was in my email message about a potential agenda. Here's the notes:
https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines
Thanks for posting this, Toshio. Few comments:
- The wheel proposal seems nice, although we should probably also
cover the case where upstream _only_ distributes wheel and not tarball (shouldn't be a big deal).
I'll have to look into this a bit. I'm not certain that a wheel contains all the source. I'm almost certain that it wouldn't fit under "preferred form of modification". Depending on those, we might want to tell people tarballs, followed by source checkouts are prefered/okay. Wheels might be third on the list or might be banned as a Source: line.
Wheels are a binary format from an upstream point of view. I certainly wouldn't consider them a suitable input format for an SRPM.
The Python world has 3 source distribution formats:
- - raw tarball - - VCS checkout - - sdist
The difference between an sdist and a raw tarball is that the sdist may be a filtered version of what's in the source repo. This may happen if the repo contains additional files that aren't needed to build and install the distribution. If an archive has a PKG-INFO file at the root, it's likely an sdist, if it doesn't, it's likely a raw tarball.
Cheers, Nick.
- -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/21/2013 10:44 AM, Toshio Kuratomi wrote:
Greetings everyone, we had a productive session at flock and actually covered everything that was in my email message about a potential agenda. Here's the notes:
https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines
A
current limitation relevant to Fedora and EPEL: pip & wheel don't yet support an equivalent to the pkg_resources parallel version support. They use virtualenv for that kind of functionality, which isn't a suitable option for system packages.
I plan to include this capability in the next iteration of the installation database spec (using an approach similar to the current setuptools approach), but we'll need to be aware of it for any packages that currently rely on parallel installs (e.g. anything with a CherryPy 2 dependency), since they won't be able to migrate to wheel based installation any time soon.
See discussion at http://mail.python.org/pipermail/distutils-sig/2013-August/022450.html for more details.
Cheers, Nick.
- -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)
python-devel@lists.fedoraproject.org