python-devel December 2013

python-devel@lists.fedoraproject.org

5 participants
4 discussions

Python 3.4, ensurepip and wheels
by Bohuslav Kabrda 12 Dec '13

12 Dec '13

Hi all, so I started to work on packaging Python 3.4 beta 1 and I wanted to create ensurepip patch also acceptable for upstream - Toshio and Nick talked about that at [1]. I tried to analyze the situation and here is what I have. First some facts: - with ensurepip, Python 3.4 bundles setuptools and pip wheel archives - when ensurepip is run, it installs (or upgrades) currently installed pip from the bundled wheels A way to solve this "properly" in a distro (as mentioned in [1]) would be: - throw away bundled setuptools and pip wheels when building Python - making python3 require python-pip - patching ensurepip (this is the fun part): -- if ensurepip is run in venv: --- we will need to reconstruct the wheels and install them into venv; or just grab the RECORD files and use them as a list of files we need to copy into venv --- to reconstruct the wheels from installed RPMs, it'd be a good idea to package python-{setuptools,pip} as wheels (with the ".dist-info" directory and all the wheel goodies inside) so that we have the RECORD files --- to package setuptools and pip as wheels, they'll have to BR: python3-pip (I saw Toshio's notes from flock and I agree with the part about how Fedora should use wheels [2]) -- if ensurepip is run systemwide: --- if "--upgrade" is used, should this run "yum update python3-pip"? Now, when new X.Y version Python is introduced, we will have to: - bootstrap the new python3 interpreter to not require python3-pip, since python3-pip will require python(abi)=X.(Y-1) - then we will need to build setuptools and pip *not as wheels*, because we still won't have pip that would work with our new Python X.Y - that means boostrapping them too and building them using the current "setup.py install" approach - when we will have the new pip, we will be able to reenable python3 requires for python3-pip - and we will also be able to rebuild setuptools and pip *as wheels* There are also some more minor issues, that are worth mentioning: - We typically won't update pip after Fedora is released - if there are security issues, we will backport patches and bump the release. The problem with this is that "ensurepip --upgrade" is not able to see Fedora releases and wouldn't work in venv, leaving it vulnerable. I think we have three options here: -- Either we use some Fedora-specific RPM magic, which won't be acceptable for upstream -- Or we will use a nice feature of wheel RECORD files - they keep file hashes, so we can check venv pip against system pip and see if something has changed when "ensurepip --upgrade" is run -- Or we can just copy whole pip again into venv, regardless of what is currently there (doesn't seem very nice..., but it's simple and in the end it works as the above method) - We can't copy the *.pyc and *.pyo files from system to venv, because they have file locations hardcoded in them, that are used in tracebacks. We can solve this by just copying *.py files and running py_compile in our ensurepip. I hope I covered all the important points. Basically, we can make this work in a way acceptable for upstream, if we package setupttols and pip as wheels. It'll require some extra effort, but I think it's worth it. Thoughts? Anyone has better/simpler ideas? Thanks a lot, Slavek. [1] https://lists.fedoraproject.org/pipermail/python-devel/2013-November/000535… [2] https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines#Whee…

3 11

Proposal for Python guidelines change (in connection to Python 3 switch)
by Bohuslav Kabrda 05 Dec '13

05 Dec '13

Hi all, as you probably know, the "Python 3 as a Default" [1] change has been accepted by FESCo and is currently targeted for F22. As a part of that Change, I also wrote that we should rethink some aspects of Python packaging in Fedora. I have put together a proposal that summarizes the current state and proposes a new approach to Python packaging [2] Before I propose it to FPC, I'd like to discuss it and work on it here on lists and incorporate everybody's notes/suggestions. So thanks for reading this through and sending your comments. Slavek. -- Regards, Bohuslav "Slavek" Kabrda. [1] https://fedoraproject.org/wiki/Changes/Python_3_as_Default [2] https://fedoraproject.org/wiki/User:Bkabrda/Py2to3GuidelineChanges

4 7

Re: [Fedora-packaging] Proposal for Python guidelines change (in connection to Python 3 switch)
by Toshio Kuratomi 04 Dec '13

04 Dec '13

On Wed, Dec 04, 2013 at 01:36:53PM +0100, Dridi Boukelmoune wrote: > On Wed, Dec 4, 2013 at 1:18 PM, Bohuslav Kabrda <bkabrda(a)redhat.com> wrote: > > ----- Original Message ----- > >> Dne 4.12.2013 12:37, Bohuslav Kabrda napsal(a): > >> > (tkinter is actually a subpackage of python itself) > >> > >> I guess you know what I mean here, but to be clear: > >> > >> tkinter is only an example, we got more, like pyserial, PyYAML... > > > > Oh, I see. Some time ago, FPC has accepted a change that says, that packages with "py" in name should be prefixed with "python-" anyway [1]. Since this only applies to newly created packages, we will have to cope with this, unfortunately. So my idea of handling this would be: > > - all packages must have Provides: python-* > > - packages that weren't prefixed with "python-" previously (pyserial, PyYAML), should also carry an explicit Provides/Obsoletes for the old name. > > Sounds good? > > What about packages named without the python- prefix for consistency [1] > with other distros ? > > Which rule should prevail ? > > Dridi > > [1] https://fedoraproject.org/wiki/Packaging:NamingGuidelines#General_Naming > The python- naming guidelines take precedence as they are more specific to the situation than the general naming guideline. -Toshio

1 0

{__python2} for el6?
by Ralph Bean 04 Dec '13

04 Dec '13

Back in August, the {__python} macro was deprecated in favor of {__python2}. This is cool as it paves the way for an eventual python3 takeover. https://fedoraproject.org/w/index.php?title=Packaging:Python&diff=350838&ol… However, {__python} is still the expected macro for el6 builds. Many of us maintain single spec files that we build for Fedora and EPEL. - Does anyone have any recommendations about how to handle this? I recently started using this blurb at the top of my new spec files. Will it do? %if 0%{?rhel} && 0%{?rhel} <= 6 %{!?__python2: %global __python2 %{__python}} %{!?python2_sitelib: %global python2_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")} %{!?python2_sitearch: %global python2_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")} %endif - If we decide on a canonical approach, can it be added to the Python guidelines as a recommendation? Cheers- -Ralph

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel December 2013