python-devel November 2021

python-devel@lists.fedoraproject.org

8 participants
6 discussions

package request: PyWaffle
by Matthew Miller 20 Dec '21

20 Dec '21

Hi! I'm using this https://github.com/gyli/PyWaffle for some visualizations for Fedora Project stats. I'm kind of out of the loop on the state of the art of python packaging, and wondered if some kind Python SIG person would like to take it on for me.

4 12

Automatically generated Obsoletes tags?
by Tomáš Orsava 15 Dec '21

15 Dec '21

Hi, I'm working on a way to automatically generate Obsoletes tags for Python packages. I.e. for each `python3-foo` package, the automatic generator would create a tag `Obsoletes: python3.10-foo < X-Y` (assuming python3 version is 3.10). Currently we automatically generate only Provides tags, but not Obsoletes, and when updating to a newer "main" Python version, a user might get file conflicts during the dnf transaction (e.g. files from python3.10-requests would conflict with files from python3-requests). If we have the automatic Obsoletes, `python3-foo` packages would always take precedence over any older `python3.X-foo`, as they should. To be frank, we don't truly need this in Fedora right now: We only ship alternative Python interpreters (e.g. python3.8, python3.9), but we do not ship additional packages for these alternative Pythons (e.g. we have no python3.9-requests). However, this will be useful for downstream distributors (e.g. CentOS, RHEL, ...), or in cases where somebody builds custom packages for alternative stacks in Fedora. The downside is that we will have a few thousand (est. 3624 [1]) additional Obsoletes tags in the Fedora repos that are mostly useless. Does anyone see a problem with this? Given the amount of tags already present (e.g. 336 thousand provides [2], 80 thousand requires [3] and 7.5 thousand obsoletes [4]), I think it won't negatively affect anything, but I might be mistaken. I think it would be useful to have the generator enabled by default, but of course to account for corner cases, I'd implement an opt-out macro (similar to `%python_disable_dependency_generator`). And to make the automatic generator consistent with the manual macros, I would modify the `%py_provides` macro to also generate Obsoletes tags. What do you think? [1] repoquery --repo=rawhide --provides -a | grep '^python3\.10-' | wc -l [2] repoquery --repo=rawhide --provides -a | wc -l [3] repoquery --repo=rawhide --requires -a | wc -l [4] repoquery --repo=rawhide --obsoletes -a | wc -l All the best, Tomas Orsava

3 3

python3-augeas and python3-ansible-runner file conflict?
by Matthew Miller 23 Nov '21

23 Nov '21

Got a report on Ask Fedora(*) about an upgrade conflict, and, yep, these two have a problem: file /usr/lib/python3.10/site-packages/test/__pycache__/__init__.cpython-310.opt-1.pyc from install of python3-ansible-runner-2.0.0a1-3.fc35.noarch conflicts with file from package python3-augeas-1.1.0-2.fc35.noarch file /usr/lib/python3.10/site-packages/test/__pycache__/__init__.cpython-310.pyc from install of python3-ansible-runner-2.0.0a1-3.fc35.noarch conflicts with file from package python3-augeas-1.1.0-2.fc35.noarch ... but, also, the particular thing being installed under "test" and particularly "test/__pycache" seems dodgy to me. Are these things even supposed to be in either package? * https://ask.fedoraproject.org/t/dnf-upgrade-to-f35-transaction-error/18570

2 4

Python RPM dependency generator does not support PEP 508?
by Michel Alexandre Salim 19 Nov '21

19 Nov '21

Hi all, Davide recently updated python-fixit, and the new version introduced an ungated requirement on 'importlib-resources', which is only needed on Python 3.6 and below; this caused an FTI as nothing satisfies that dependency: https://bugzilla.redhat.com/show_bug.cgi?id=2021229 To our surprise, patching requirements.txt to have importlib-resources>=5.1.2; python_version <= '3.6' per PEP 508 (which is a fix we can upstream) does not actually work. cf: https://www.python.org/dev/peps/pep-0508/#examples We're working around it by patching out this line completely, which does work as expected, but: 1. should we support this conditional 2. where should we file this RFE? Thanks, -- Michel Alexandre Salim profile: https://keyoxide.org/michel@michel-slm.name

2 2

Python RPM dependency generators and LegacyVersion, time to error?
by Miro Hrončok 11 Nov '21

11 Nov '21

Hello Pythonistas. After some recent improvements in the Python RPM dependency generators, a regression was discovered [0]. Turns out the error happened when the upstream metadata contained a requirement with a PEP 440 [1] incompatible version. A fix is pending and only one package in Fedora was affected. However, this brings me to a question: What shall we do with such incompatible versions in the future? Implementation-wise, the generators use the pypa/packaging library to parse the requirements. When a PEP 440 incompatible version is found, it's parsed as a LegacyVersion [3]. A LegacyVersion has been deprecated for 1 year now and when found it might indicate a packaging problem. I've experimented with a patched version of the generators that would error out right away when a LegacyVersion is parsed. I found the following categories of problems: 1) A requirement uses invalid/legacy version (2 packages): python-celery: pytz>dev This is a pip-compatible shortcut for 0.dev.0, which is only required as a hack for PEP 440 incompatible versions of pytz from before 2013 (2012a, 2012b, 2012c...). python-pvc: pyvmomi>=5.5.0-2014.1.1 This is an existing PEP 440 incompatible version of pyvmomi that has been since replaced with PEP 440 compatible version on PyPI. 2) The package itself has an invalid/legacy version (3 packages): python-haversion: main python-ipmi: unknown python-lacrosse: unknown This is most likely a packaging problem introduced by the Fedora packager by using a GitHub tarball whereas the version information is only stored in sdist. I believe the generators should error out immediately when this happens to prevent packaging errors, but there might be a need to allow it to proceed if we want to package e.g. pyvmomi 5.5.0-2014.1.1 or pytz 2012d. tl;dr: - there might be cases where legacy versions are needed - but in most cases, they should be avoided Hence, I propose we do the following in Rawhide: We turn LegacyVersions to failures, but we provide a stop-gap measure, such as (%global python_dependency_allow_legacy_version_provides 1 / %global python_dependency_allow_legacy_version_requires 1 ) for packages that need to override this. When pypa/packaging actually drops LegacyVersion, this stop-gap measure will no longer work. [0] https://bugzilla.redhat.com/show_bug.cgi?id=2019954 [1] https://www.python.org/dev/peps/pep-0440/ [2] https://packaging.pypa.io/ [3] https://packaging.pypa.io/en/latest/version.html#packaging.version.LegacyVe… -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 3

HEADS UP: Slightly more robust Python Requires generated by RPM
by Miro Hrončok 01 Nov '21

01 Nov '21

Hello Pythonistas, just a heads up that in rawhide, the automatically generated requirements on Python packages, e.g. python3.Xdist(DISTNAME), ar enow generated in more strict form. If a package required "PyYAML < 6", it was previously generated as: python3.10dist(pyyaml) < 6 Which was satisfied by PyYAML 6.0b1 (python3.10dist(pyyaml) 6~b1). From now on, it will be generated as: python3.10dist(pyyaml) < 6~ Which will not be satisfied by any alpha/beta/rc release of PyYAML 6. There are other related differences as well neither of which should release break anything except for when dealing with pre-releases, it is now more strict. Details: https://bugzilla.redhat.com/show_bug.cgi?id=2014616 Any new requires will start happening as the packages are rebuilt, so as always, please check your packages for installability when you build them and let me know if something behaves in a weird way. Identical changes to %pyproject_buildrequires will land soon (on all releases, as maintaining bit-by-bit requires compatibility is not necessary there). Thanks to Gordon for improving the generators. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

5 11

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel November 2021