During the 3.7 boostrapping of the interstitial sequence of Python 3 packages, I have noticed it includes a lot of packages that are only intended as Python 3 stdlib backports for older Pythons.
https://pypi.org/project/contextlib2/ is a backport of the standard library’s contextlib module to earlier Python versions
https://pypi.org/project/linecache2/ is a backport of linecache to older supported Pythons.
https://pypi.org/project/pathlib2/ The goal of pathlib2 is to provide a backport of standard pathlib module which tracks the standard library module, so all the newest features of the standard pathlib can be used also on older Python versions.
https://pypi.org/project/traceback2/ is a backport of traceback to older supported Pythons.
https://pypi.org/project/unittest2/ is a backport of the new features added to the unittest testing framework in Python 2.7 and onwards.
https://pypi.org/project/simplejson/ is the externally maintained development version of the json library included with Python 2.6 and Python 3.0, but maintains backwards compatibility with Python 2.5.
https://pypi.org/project/funcsigs/ is a backport of the PEP 362 function signature features from Python 3.3
https://pypi.org/project/mock/ is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.
Now I see fairly no real reason to have python3-contextlib2, python3-linecache2 etc. packaged and maintained in Fedora. The only reason there is that other packages actually depend on them.
I think we should actively change the packages not to depend on those and eventually get rid of them. This involves changing how upstreams are using those. Thoughts?
Here is some dependency info:
$ whatrequires python3-contextlib2 python3-futurist python3-pytest-virtualenv python3-vcrpy
python-pytest-shutil python-traceback2
$ whatrequires python3-linecache2 python3-traceback2
$ whatrequires python3-pathlib2 pipenv python3-spyder
python-testpath
$ whatrequires python3-traceback2 python3-testtools python3-unittest2
$ whatrequires python3-unittest2 python3-testtools
ProDy cloud-init compose-utils fedpkg libabigail pungi pyephem python-agate python-agate-excel python-autobahn python-billiard python-case python-funcsigs python-leather python-linecache2 python-mimerender python-mock python-pyclipper python-pykafka python-ripozo python-rsa python-s3transfer python-traceback2 python-tw2-core python-unicodecsv
$ whatrequires python3-simplejson copr-cli copr-frontend copr-rpmbuild gnome-code-assistance kismon python2-pystache (huh?) python3-canonicaljson python3-cherrypy python3-cinderclient python3-cornice python3-geojson python3-jsonpickle python3-manilaclient python3-mwclient python3-neutronclient python3-novaclient python3-octaviaclient python3-osc-lib python3-oslo-serialization-tests python3-peewee python3-rabbitvcs python3-resultsdb_api python3-scales python3-tackerclient python3-troveclient python3-tw2-core python3-tweepy python3-twitter python3-uri-templates python3-x2go translate-toolkit
pymol python-anyjson python-arrow python-marshmallow python-mygpoclient python-openstackclient python-sphinx
$ whatrequires python3-funcsigs module-build-service python3-debtcollector python3-mock python3-octaviaclient python3-oslo-utils python3-tinyrpc python3-tinyrpc-tests
odcs python-ZEO python-msgpack python-mwclient python-ripe-atlas-cousteau
$ whatrequires python3-mock copr-keygen module-build-service python3-ara-tests python3-certbot python3-certbot-dns-* python3-conda python3-congressclient-tests python3-keystoneclient-tests python3-kubernetes-tests python3-magnumclient-tests python3-music21 python3-netdiff python3-octaviaclient-tests python3-osc-lib-tests python3-oslo-cache-tests python3-oslo-log-tests python3-oslo-middleware-tests python3-oslo-policy-tests python3-oslo-rootwrap-tests python3-oslo-serialization-tests python3-oslo-service-tests python3-oslo-versionedobjects python3-oslo-vmware-tests python3-oslotest python3-pytest-spec python3-pywbem python3-sphinx python3-testfixtures-tests python3-testify python3-testinfra python3-tinyrpc-tests python3-vcrpy waiverdb
...320 source packages buildrequire this one!
On Sat, Jun 30, 2018, 5:59 AM Miro Hrončok mhroncok@redhat.com wrote:
During the 3.7 boostrapping of the interstitial sequence of Python 3 packages, I have noticed it includes a lot of packages that are only intended as Python 3 stdlib backports for older Pythons.
In general, I think this is a good effort however, it's not always straightforward and not always about python3. There's also at least one item on your list (simplejson) which should remain packaged in Fedora because it is useful as more than forwards compatibility.
Here's an example of how you might need to replace "import contextlib2":
try: import contextlib except ImportError: contextlib = False else: if not hasattr(contextlib, 'nullcontext'): contextlib = False if not contextlib: import contextlib2 as contextlib
For each of these libraries and usages you'll have to:
* identify what facilities of the library are being used by the dependent package. * Figure out if those facilities are present in all versions of the stdlib or not. * Figure out whether the library in question is truly a backport. I'd be hesitant to replace anything that isn't truly a backport (simplejson definitely shouldn't be treated as a backport, for instance. mock and unittest2 maybe shouldn't be treated as backports... If all Michael foord for his input)
For the latter, the goal is to identify if there's a reason to use the library for more than just it's API compatibility with newer versions of python 3. For simplejson, I know that it is faster than the stdlib json library. It may also have additional API; the stdlib has not pulled the changes from simplejson back into itself (and, iirc, is now best considered a fork of simplejson rather than just a copy).
Also, be careful about how you phrase your requests to upstream to merge your changes. We don't want to annoy upstreams with requests to use python3 stdlib modules when the backport works just as well for their code for now as too much of that will turn upstream off of python3 altogether.
-Toshio
https://pypi.org/project/contextlib2/ is a backport of the standard library’s contextlib module to earlier Python versions
https://pypi.org/project/linecache2/ is a backport of linecache to older supported Pythons.
https://pypi.org/project/pathlib2/ The goal of pathlib2 is to provide a backport of standard pathlib module which tracks the standard library module, so all the newest features of the standard pathlib can be used also on older Python versions.
https://pypi.org/project/traceback2/ is a backport of traceback to older supported Pythons.
https://pypi.org/project/unittest2/ is a backport of the new features added to the unittest testing framework in Python 2.7 and onwards.
https://pypi.org/project/simplejson/ is the externally maintained development version of the json library included with Python 2.6 and Python 3.0, but maintains backwards compatibility with Python 2.5.
https://pypi.org/project/funcsigs/ is a backport of the PEP 362 function signature features from Python 3.3
https://pypi.org/project/mock/ is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.
Now I see fairly no real reason to have python3-contextlib2, python3-linecache2 etc. packaged and maintained in Fedora. The only reason there is that other packages actually depend on them.
I think we should actively change the packages not to depend on those and eventually get rid of them. This involves changing how upstreams are using those. Thoughts?
Here is some dependency info:
$ whatrequires python3-contextlib2 python3-futurist python3-pytest-virtualenv python3-vcrpy
python-pytest-shutil python-traceback2
$ whatrequires python3-linecache2 python3-traceback2
$ whatrequires python3-pathlib2 pipenv python3-spyder
python-testpath
$ whatrequires python3-traceback2 python3-testtools python3-unittest2
$ whatrequires python3-unittest2 python3-testtools
ProDy cloud-init compose-utils fedpkg libabigail pungi pyephem python-agate python-agate-excel python-autobahn python-billiard python-case python-funcsigs python-leather python-linecache2 python-mimerender python-mock python-pyclipper python-pykafka python-ripozo python-rsa python-s3transfer python-traceback2 python-tw2-core python-unicodecsv
$ whatrequires python3-simplejson copr-cli copr-frontend copr-rpmbuild gnome-code-assistance kismon python2-pystache (huh?) python3-canonicaljson python3-cherrypy python3-cinderclient python3-cornice python3-geojson python3-jsonpickle python3-manilaclient python3-mwclient python3-neutronclient python3-novaclient python3-octaviaclient python3-osc-lib python3-oslo-serialization-tests python3-peewee python3-rabbitvcs python3-resultsdb_api python3-scales python3-tackerclient python3-troveclient python3-tw2-core python3-tweepy python3-twitter python3-uri-templates python3-x2go translate-toolkit
pymol python-anyjson python-arrow python-marshmallow python-mygpoclient python-openstackclient python-sphinx
$ whatrequires python3-funcsigs module-build-service python3-debtcollector python3-mock python3-octaviaclient python3-oslo-utils python3-tinyrpc python3-tinyrpc-tests
odcs python-ZEO python-msgpack python-mwclient python-ripe-atlas-cousteau
$ whatrequires python3-mock copr-keygen module-build-service python3-ara-tests python3-certbot python3-certbot-dns-* python3-conda python3-congressclient-tests python3-keystoneclient-tests python3-kubernetes-tests python3-magnumclient-tests python3-music21 python3-netdiff python3-octaviaclient-tests python3-osc-lib-tests python3-oslo-cache-tests python3-oslo-log-tests python3-oslo-middleware-tests python3-oslo-policy-tests python3-oslo-rootwrap-tests python3-oslo-serialization-tests python3-oslo-service-tests python3-oslo-versionedobjects python3-oslo-vmware-tests python3-oslotest python3-pytest-spec python3-pywbem python3-sphinx python3-testfixtures-tests python3-testify python3-testinfra python3-tinyrpc-tests python3-vcrpy waiverdb
...320 source packages buildrequire this one!
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproje...
On 30.6.2018 15:33, Toshio Kuratomi wrote:
For the latter, the goal is to identify if there's a reason to use the library for more than just it's API compatibility with newer versions of python 3. For simplejson, I know that it is faster than the stdlib json library. It may also have additional API; the stdlib has not pulled the changes from simplejson back into itself (and, iirc, is now best considered a fork of simplejson rather than just a copy).
Of course, digging into details about particular libraries would be needed. With simplejson, it's complicated [1].
Also, be careful about how you phrase your requests to upstream to merge your changes. We don't want to annoy upstreams with requests to use python3 stdlib modules when the backport works just as well for their code for now as too much of that will turn upstream off of python3 altogether.
Good point!
[1] https://stackoverflow.com/questions/712791/what-are-the-differences-between-...
On Sat, Jun 30, 2018 at 06:33:22AM -0700, Toshio Kuratomi wrote:
On Sat, Jun 30, 2018, 5:59 AM Miro Hrončok mhroncok@redhat.com wrote:
During the 3.7 boostrapping of the interstitial sequence of Python 3 packages, I have noticed it includes a lot of packages that are only intended as Python 3 stdlib backports for older Pythons.
In general, I think this is a good effort however, it's not always straightforward and not always about python3. There's also at least one item on your list (simplejson) which should remain packaged in Fedora because it is useful as more than forwards compatibility.
Here's an example of how you might need to replace "import contextlib2":
try: import contextlib except ImportError: contextlib = False else: if not hasattr(contextlib, 'nullcontext'): contextlib = False if not contextlib: import contextlib2 as contextlib
That's a generally useful formula if support for diverse python versions is needed. But in Fedora we know that we have python3.7 (assuming that this effort would be directed at F29+), so just the above collapses to just a single
import contextlib
Not appropriate as an upstream patch of course, but good enough for Fedora.
https://pypi.org/project/contextlib2/ is a backport of the standard library’s contextlib module to earlier Python versions
https://pypi.org/project/linecache2/ is a backport of linecache to older supported Pythons.
https://pypi.org/project/pathlib2/ The goal of pathlib2 is to provide a backport of standard pathlib module which tracks the standard library module, so all the newest features of the standard pathlib can be used also on older Python versions.
https://pypi.org/project/traceback2/ is a backport of traceback to older supported Pythons.
https://pypi.org/project/unittest2/ is a backport of the new features added to the unittest testing framework in Python 2.7 and onwards.
https://pypi.org/project/simplejson/ is the externally maintained development version of the json library included with Python 2.6 and Python 3.0, but maintains backwards compatibility with Python 2.5.
https://pypi.org/project/funcsigs/ is a backport of the PEP 362 function signature features from Python 3.3
https://pypi.org/project/mock/ is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.
Now I see fairly no real reason to have python3-contextlib2, python3-linecache2 etc. packaged and maintained in Fedora. The only reason there is that other packages actually depend on them.
I think we should actively change the packages not to depend on those and eventually get rid of them. This involves changing how upstreams are using those. Thoughts?
I think it'd be good to start with the most egregious ones (e.g. the ones that "backport" features from python 3.3), and try to kill all dependencies and then the package itself. The nice thing about this problem it can be done piecemeal.
$ whatrequires python3-mock copr-keygen module-build-service python3-ara-tests python3-certbot python3-certbot-dns-* python3-conda python3-congressclient-tests python3-keystoneclient-tests python3-kubernetes-tests python3-magnumclient-tests python3-music21 python3-netdiff python3-octaviaclient-tests python3-osc-lib-tests python3-oslo-cache-tests python3-oslo-log-tests python3-oslo-middleware-tests python3-oslo-policy-tests python3-oslo-rootwrap-tests python3-oslo-serialization-tests python3-oslo-service-tests python3-oslo-versionedobjects python3-oslo-vmware-tests python3-oslotest python3-pytest-spec python3-pywbem python3-sphinx python3-testfixtures-tests python3-testify python3-testinfra python3-tinyrpc-tests python3-vcrpy waiverdb
...320 source packages buildrequire this one!
For the ones that have the most deps, maybe it would be worth to add all the Provides to python3-libs package?
Zbyszek
On 1.7.2018 23:37, Zbigniew Jędrzejewski-Szmek wrote:> For the ones that have the most deps, maybe it would be worth to
add all the Provides to python3-libs package?
That would only make sense if the import lien would be the same. For example if we provide python3-mock form python3-libs, people would expect that `import mock` works. But in fact, what they need to do is `form unittest import mock`.
We could add a compatibility layer to support this, but that is diverging from upstream and obviously very fragile.
python-devel@lists.fedoraproject.org