python-devel June 2018

python-devel@lists.fedoraproject.org

6 participants
9 discussions

Python 3.7's Deterministic pycs
by Petr Viktorin 15 Jul '19

15 Jul '19

Hello! The first beta for Python 3.7 is out. It will hopefully get into Fedora soon as python37. After it comes out of beta, we'll upgrade python3 to it. The What's New list is at: https://docs.python.org/3.7/whatsnew/3.7.html One thing that's interesting for packagers is PEP 552: Deterministic pycs: https://www.python.org/dev/peps/pep-0552/ Let me summarize in my own words. A new opt-in mode for byte-compilation makes .pyc (bytecode cache) files depend only on the contents of the corresponding source file. If we use this, it will slow down imports, because the whole source file would need to be read and hashed in order to verify if a .pyc file is valid. (Currently, metadata like the modification time and file size is used.) To speed things up, there's an option, UNCHECKED_HASH, which skips cache validation entirely. Using this would mean that if you modify a .py source file installed by RPM, the changes wouldn't take effect (the .py contents would only be shown in tracebacks). Modifying installed files in production is extremely bad practice, of course, but it's quite useful for debugging on throw-away systems. If we adopt UNCHECKED_HASH, anyone doing it will have to remember to remove the corresponding .pyc file. Honestly, I'm not sure we want to use this in Fedora. Is anyone here into reproducible builds, to make a better argument for this? -- Petr Viktorin

5 5

Getting rid of python3- backports packages
by Miro Hrončok 02 Jul '18

02 Jul '18

During the 3.7 boostrapping of the interstitial sequence of Python 3 packages, I have noticed it includes a lot of packages that are only intended as Python 3 stdlib backports for older Pythons. https://pypi.org/project/contextlib2/ is a backport of the standard library’s contextlib module to earlier Python versions https://pypi.org/project/linecache2/ is a backport of linecache to older supported Pythons. https://pypi.org/project/pathlib2/ The goal of pathlib2 is to provide a backport of standard pathlib module which tracks the standard library module, so all the newest features of the standard pathlib can be used also on older Python versions. https://pypi.org/project/traceback2/ is a backport of traceback to older supported Pythons. https://pypi.org/project/unittest2/ is a backport of the new features added to the unittest testing framework in Python 2.7 and onwards. https://pypi.org/project/simplejson/ is the externally maintained development version of the json library included with Python 2.6 and Python 3.0, but maintains backwards compatibility with Python 2.5. https://pypi.org/project/funcsigs/ is a backport of the PEP 362 function signature features from Python 3.3 https://pypi.org/project/mock/ is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards. Now I see fairly no real reason to have python3-contextlib2, python3-linecache2 etc. packaged and maintained in Fedora. The only reason there is that other packages actually depend on them. I think we should actively change the packages not to depend on those and eventually get rid of them. This involves changing how upstreams are using those. Thoughts? Here is some dependency info: $ whatrequires python3-contextlib2 python3-futurist python3-pytest-virtualenv python3-vcrpy python-pytest-shutil python-traceback2 $ whatrequires python3-linecache2 python3-traceback2 $ whatrequires python3-pathlib2 pipenv python3-spyder python-testpath $ whatrequires python3-traceback2 python3-testtools python3-unittest2 $ whatrequires python3-unittest2 python3-testtools ProDy cloud-init compose-utils fedpkg libabigail pungi pyephem python-agate python-agate-excel python-autobahn python-billiard python-case python-funcsigs python-leather python-linecache2 python-mimerender python-mock python-pyclipper python-pykafka python-ripozo python-rsa python-s3transfer python-traceback2 python-tw2-core python-unicodecsv $ whatrequires python3-simplejson copr-cli copr-frontend copr-rpmbuild gnome-code-assistance kismon python2-pystache (huh?) python3-canonicaljson python3-cherrypy python3-cinderclient python3-cornice python3-geojson python3-jsonpickle python3-manilaclient python3-mwclient python3-neutronclient python3-novaclient python3-octaviaclient python3-osc-lib python3-oslo-serialization-tests python3-peewee python3-rabbitvcs python3-resultsdb_api python3-scales python3-tackerclient python3-troveclient python3-tw2-core python3-tweepy python3-twitter python3-uri-templates python3-x2go translate-toolkit pymol python-anyjson python-arrow python-marshmallow python-mygpoclient python-openstackclient python-sphinx $ whatrequires python3-funcsigs module-build-service python3-debtcollector python3-mock python3-octaviaclient python3-oslo-utils python3-tinyrpc python3-tinyrpc-tests odcs python-ZEO python-msgpack python-mwclient python-ripe-atlas-cousteau $ whatrequires python3-mock copr-keygen module-build-service python3-ara-tests python3-certbot python3-certbot-dns-* python3-conda python3-congressclient-tests python3-keystoneclient-tests python3-kubernetes-tests python3-magnumclient-tests python3-music21 python3-netdiff python3-octaviaclient-tests python3-osc-lib-tests python3-oslo-cache-tests python3-oslo-log-tests python3-oslo-middleware-tests python3-oslo-policy-tests python3-oslo-rootwrap-tests python3-oslo-serialization-tests python3-oslo-service-tests python3-oslo-versionedobjects python3-oslo-vmware-tests python3-oslotest python3-pytest-spec python3-pywbem python3-sphinx python3-testfixtures-tests python3-testify python3-testinfra python3-tinyrpc-tests python3-vcrpy waiverdb ...320 source packages buildrequire this one! -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 4

What is the criterion for Python 3.7 side tag merge?
by Miro Hrončok 29 Jun '18

29 Jun '18

The Python 3.7 rebuild in the f29-python side tag is currently: - 2168 built - ~170 FTBFS - ~120 blocked by dependencies that FTBFS Ideally, this would all get fixed before the merge, but that's unrealistic, there are packages that are FTBFS for unrelated reasons, etc. What is the criterion to merge the side tag? The critpath installs fine (except stuff that cannot be installed due to yum-deprecated vs yum collisions and gnome-software that has some broken dependency on PackageKit). Stuff that doesn't build and sounds important: - ceph (also FTBFS in regular rawhide) - libreoffice (also FTBFS in regular rawhide) - orca (blocked by pyatspi, looking at it ATM) - kernel-tools -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

Heads up: Python 3.7 rebuild in progress
by Miro Hrončok 26 Jun '18

26 Jun '18

I've just started to build the bootstrap sequence in a side tag (f29-python). This should not affect you mostly but if you have a Python 3 package and you are going to update it with new buildtime dependencies, please let me know or wait until this is done. The initial order is in https://github.com/hroncok/rpm-list-builder/blob/python37/python37.yaml -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 6

Ask for help with packaging of Pipenv deps
by Michal Cyprian 23 Jun '18

23 Jun '18

Hi, we added Pipenv, The higher level Python packaging tool [0], to Fedora recently. It is a nice tool and it is also officially recommended from Python.org. This is why we wanted to make it available to Fedora users as soon as possible. The upstream Pipenv project contains a lot of bundled dependencies. Some of them are not packaged for Fedora yet, so we were not able to unbundle them immediately. The set of missing packages from PyPI is following: * https://pypi.org/project/blindspin/ * https://pypi.org/project/delegator.py/ * https://pypi.org/project/first/ * https://pypi.org/project/pipreqs/ * https://pypi.org/project/pipdeptree/ * https://pypi.org/project/requirements-parser/ * https://pypi.org/project/shutilwhich/ * https://pypi.org/project/yarg/ In case you want to package some Python software for Fedora, we will appreciate your help. Packages only for Python 3 version are needed. If you pick some of the packages, please respond to this thread to prevent more packagers working on the same package. Also check if the Review Request bug for the package doesn't exist already. [0] https://src.fedoraproject.org/rpms/pipenv Michal Cyprian

2 2

Re: Heads up: Python 3.7 rebuild in progress
by Miro Hrončok 21 Jun '18

21 Jun '18

On 21.6.2018 06:12, Jerry James wrote: > On Wed, Jun 20, 2018 at 1:36 AM Miro Hrončok <mhroncok(a)redhat.com> wrote: >> Unfortunately this is blocked by failing: >> >> python-manuel >> https://koji.fedoraproject.org/koji/taskinfo?taskID=27740145 >> https://bugzilla.redhat.com/show_bug.cgi?id=1593132 > > I had planned to look at that tonight. However, just as I arrived > home from work tonight, as I was pulling into my driveway, my > neighbor's propane grill exploded, catching his deck and then his > house on fire. Fortunately there were no serious injuries. I'm glad nobody go seriously hurt. > Is there a way to do a mock build against the packages currently in f29-python? fedpkg mock-config --target f29-python This gives you a mock config you can use with mock -r or fedpkg mockbuild --root. (Untested.) -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Re: Heads up: Python 3.7 rebuild in progress
by Miro Hrončok 20 Jun '18

20 Jun '18

On 20.6.2018 05:55, Jerry James wrote: >> fedpkg build --target=f29-python > > I have fixed python-pybtex, python-pybtex-docutils, python-latexcodec, > python-sphinx-testing, and started a build of > python-sphinxcontrib-bibtex which I think has a high chance of > succeeding. Thank You. > That should unblock the python-BTrees build, which in > turn should unblock the rest of the ZODB/ZEO stack. Unfortunately this is blocked by failing: python-manuel https://koji.fedoraproject.org/koji/taskinfo?taskID=27740145 https://bugzilla.redhat.com/show_bug.cgi?id=1593132 python-trollius https://koji.fedoraproject.org/koji/taskinfo?taskID=27740570 https://bugzilla.redhat.com/show_bug.cgi?id=1593133 python-uvloop https://koji.fedoraproject.org/koji/taskinfo?taskID=27740603 https://bugzilla.redhat.com/show_bug.cgi?id=1556279 Or stuff that depends on those. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Re: Intent to drop python2-notebook
by Miro Hrončok 05 Jun '18

05 Jun '18

On 5.6.2018 18:21, Jerry James wrote: > On Tue, Jun 5, 2018 at 9:30 AM, Miro Hrončok <mhroncok(a)redhat.com > <mailto:mhroncok@redhat.com>> wrote: > > On 11.5.2018 17:54, Miro Hrončok wrote: > > python2-notebook is a leaf package (nothing in Fedora > (build)requires it). I'd like to drop it from rawhide. Any > objections? > > Dropped. > > > Uh oh. I just added both a BR and an R from sagemath to > python2-notebook a couple of days ago, because sagemath does require > it. Does it require all of it? It's not a library. Can we see what exactly is required and why maybe? How is it used - via Python imports or via the command form /usr/bin? > Can we bring it back? I can help maintain it if maintenance cycles > are an issue. We can, but please check the above first. The issue is that upstream no longer seem to care about Python 2. Last time, it had some SyntaxErrors on legacy Python. Fortunatelly, only in some tests, so I've dropped those from the py2 package, but I just didn't want to deal with this any more. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

Intent to drop python2-notebook
by Miro Hrončok 05 Jun '18

05 Jun '18

python2-notebook is a leaf package (nothing in Fedora (build)requires it). I'd like to drop it from rawhide. Any objections? This **does not** affect the ability to run Jupyter Notebook (via the jupyter-notebook command) with Python 2 kernel (from the python2-ipykernel package). -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel June 2018