python-devel September 2012

python-devel@lists.fedoraproject.org

4 participants
3 discussions

Upstream packaging feedback
by Nick Coghlan 05 Oct '12

05 Oct '12

I'll likely be helping to guide updates to the Python packaging format standards over the coming months. While they won't hit the standard library until 3.4, there will likely be third party tool support in earlier versions (since the whole point of the exercise is to eliminate the current implementation coupling to distutils and setuptools in favour of better defined metadata standards for communication between multiple tools). The first step will be reviewing the status quo and then creating a plausible road map (as well as describing current efforts for various aspects). I've started on that here: http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packagi… One thing I would *love* to be able to enable is adding support for automatic mapping of PyPI distribution names (similar to what already exists for Perl and CPAN) where (for example), a developer could just write "Requires: python(south)" instead of having to figure out manually the name of the appropriate RPM package in Fedora. I believe that the new metadata fields defined in PEP 345 and PEP 426 should be enough to support that when generating a SPEC file from the PyPI metadata. Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane

2 3

Re: Upstream packaging feedback
by Stanley A. Klein 21 Sep '12

21 Sep '12

This is a timely discussion for me because I need to get back to packaging some tools for RHEL/Centos that we have as dependencies. The tools are available on pypi or from a git repository. They seem to be available in older versions for Fedora 14 but not in EPEL or other sites for RHEL/Centos. One issue I encountered when previously working on the packaging for Fedora was that pypi packages do not necessarily follow the Linux Filesystem Hierarchy Standard. In fact, the packages may be implicitly organized for something like Windows where documentation goes in the same directory as code. I found I needed to use pypi2rpm bdist_rpm2 and modify the spec file to get things into the right places. BTW, the tools right now use python 2.7. So, please add the FHS issue to your list. I've also found that everything seems to be in a state of flux right now, with setuptools, distribute, distutils, distutils2, and pypi2rpm/bdist_rpm2 all out there and bumping into each other. When I tried to do a yum update to my Fedora 14 system, I ran into trouble because of conflicts among these packages. I hope this all settles down soon. Stan Klein On Thu, September 13, 2012 8:00 am, Toshio Kuratomi <a.badger(a)gmail.com> wrote: > > Message: 1 > Date: Wed, 12 Sep 2012 15:14:07 -0700 > From: Toshio Kuratomi <a.badger(a)gmail.com> > To: Fedora Python SIG <python-devel(a)lists.fedoraproject.org> > Subject: Re: Upstream packaging feedback > > On Wed, Sep 12, 2012 at 01:41:22PM +1000, Nick Coghlan wrote: >> I'll likely be helping to guide updates to the Python packaging format >> standards over the coming months. While they won't hit the standard >> library until 3.4, there will likely be third party tool support in >> earlier versions (since the whole point of the exercise is to eliminate >> the current implementation coupling to distutils and setuptools in >> favour of better defined metadata standards for communication between >> multiple tools). >> >> The first step will be reviewing the status quo and then creating a >> plausible road map (as well as describing current efforts for various >> aspects). I've started on that here: >> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packagi… >> >> One thing I would *love* to be able to enable is adding support for >> automatic mapping of PyPI distribution names (similar to what already >> exists for Perl and CPAN) where (for example), a developer could just >> write "Requires: python(south)" instead of having to figure out manually >> the name of the appropriate RPM package in Fedora. >> >> I believe that the new metadata fields defined in PEP 345 and PEP 426 >> should be enough to support that when generating a SPEC file from the >> PyPI metadata. > > Hey Nick! Thanks for working on this whole thing. I haven't been looking > at things in the past year but was active on it a few years ago if you > want > to bounce any ideas around, get some idea of what's been discussed i nthe > past by whom, or talk about what Fedora-specifically is currently > doing/has > found troublesome in the past. > > Mapping of pypi distribution names was something that I looked into a > small > bit with some Canonical people at PyCon two years ago. IIRC, it was part > of > trying to map distribution packages with each other and with pypi in order > to figure out the state of python3 porting. Barry Warsaw still works > there > and might know more about what happened to it -- Allison Randal is who I > was > working with but I got the impression that she isn't working on that > anymore > so I don't know if she'll still have code around or not. > > There are numerous caveats to trying to do this, none of them > insurmountable. I believe we were trying to compare versions as well as > package names which was even tougher. Some things I can remember off the > top of my head: > > * Multiple names for a package > * pypi usually has the same name as the setup.py name field which is > also > encoded in the egg file name and metadata > * The name of the module that is imported > * One upstream package having multiple downstream names -- for instance, > Debian has setuptools and pkg_resources as two separate binary packages. > However, they're provided by the setuptools pypi entry. > * A module that is provided by multiple upstreams: For instance, > setuptools > is provided by setuptools and distribute. pexpect is provided by > pexpect > and pexpect-u. > * Some packages aren't present on pypi. Many library bindings > provided by the libraries are this way, for instance libselinux-python. > * Some libraries have conflicting names upstream (mock was one example in > the past. ming still is one: http://www.libming.org/ and > http://merciless.sourceforge.net/ > * The naming for packages in the distribution isn't always simple. In > Fedora, for instance, we have several styles and not all of them are > mutually exclusive: > * python-foo is the common case for libraries (python-docutils) > * foo-python are the majority (but not all) of libraries which are > bindings to C libraries (selinux-python) > * Fedora allows modules that have py as a prefix to not add python- to > their name. (pygtk2) > * Applications may provide libraries that other packages need but since > they're "primarily applications" they may simply bear the name of the > application. (bzr) > * Case sensitivity can get you. There are some maintainers that prefer > to > lowercase the package names and others who prefer to do exactly what > upstream did. > * dashes, periods, and other punctuation can also get you. Sometimes > those are translated into dashes, other times they're left alone, and > other times they're omitted. > * Even today, not all packages provide egg-info. For instance, a useful > python module might consist of a single .py file so upstream might not > provide a setup.py for it. We install it by copying the .py file to > site-packages. > * We'll need to differentiate between things provided for python2 and > python3 > (and python26 in EPEL) > > All that said.... you can probably sidestep some of these issues by having > python packages contain explicit virtual Provides. These might be > manually > added or automatically generated by a tool like pypi2rpm with the > maintainer editting them afterwards to make sure they didn't hit any of > the > above cornercases. > > There's some prior work done by other people: > * http://www.rpm.org/ticket/154 > * http://lists.fedoraproject.org/pipermail/packaging/2008-June/004715.html > (Despite my having written that email, the hard parts were all dmalcolm > :-) > > -Toshio >

1 0

Renaming Feature for F19 [was: Re: Switching to Python 3]
by Thomas Spura 04 Sep '12

04 Sep '12

On Tue, Aug 14, 2012 at 9:42 AM, Nick Coghlan <ncoghlan(a)redhat.com> wrote: > However, I think it's enough to place a clear upper limit on the number > of runtimes to be supported (where 'x' is the relevant minor version > packaged in the Fedora repos): CPython 2.x, PyPy 1.x, Python 3.x (with > shared site-packages) I don't know if pypy1-foo makes sense or how they want to support python2 and python3 at the same time. But I'm all for pypy1-foo to be on the save side... One thing, that comes to my mind: Should it be python2-foo or cpython2-foo? Otherwise, I went ahead and created a feature page: https://fedoraproject.org/wiki/Features/PythonNamingDependingOnImplementati… Feel free to add yourself to the "owner" list and change it, when there is something missing. I would propose, that we agree to a IRC meeting, where we can discuss possible differences or do you think anything is sorted out now and the feature is "sane" for anyone? Greetings, Tom

3 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel September 2012