python-devel June 2020

python-devel@lists.fedoraproject.org

17 participants
23 discussions

Fwd: Fedora 33 Self-Contained Change proposal: Automatic RPM dependencies on Python Extras
by Miro Hrončok 26 Jun '20

26 Jun '20

-------- Forwarded Message -------- Subject: Fedora 33 Self-Contained Change proposal: Automatic RPM dependencies on Python Extras Date: Tue, 23 Jun 2020 15:22:29 -0400 From: Ben Cotton <bcotton(a)redhat.com> Reply-To: Development discussions related to Fedora <devel(a)lists.fedoraproject.org> To: devel-announce(a)lists.fedoraproject.org, Development discussions related to Fedora <devel(a)lists.fedoraproject.org> https://fedoraproject.org/wiki/Changes/PythonExtras == Summary == The Python RPM dependency generator (that generates <code>python3.Xdist(foo)</code> requirements) will be adapted to also generate requirements on [https://www.python.org/dev/peps/pep-0508/#extras Python extras] (e.g. <code>python3.Xdist(foo[bar])</code>) whenever upstream metadata indicate such dependency. An easy opt out mechanism will exist. A supported way of adding metapackages that provide such Python extras (e.g. <code>python3.Xdist(foo[bar])</code>) will be introduced. Change owners will add the missing metapackages that would otherwise cause broken dependencies (in non-modular packages). == Owner == * Name: [[User:Torsava|Tomáš Orsava]] * Name: [[User:Churchyard|Miro Hrončok]] * Email: <python-maint(a)redhat.com> == Detailed Description == === The problem === [https://www.python.org/dev/peps/pep-0508/#extras Python extras] are a way for a Python package (called "distribution" or "distribution package" upstream) to declare that extra dependencies are required for additional functionality. For example Python package <code>requests</code> has several standard dependencies (e.g. <code>urllib3</code>). But it also declares an extra named <code>requests[security]</code> which lists additional dependencies (e.g. <code>pyOpenSSL</code>) if you want to use this additional functionality. The Python package code handles the missing optional dependency gracefully -- e.g. it won't crash but might instruct the user to install <code>requests[security]</code> if needed by a warning or an actionable error message. Python packages included in Fedora as RPMs automatically create a special Provides in the format <code>python3.Xdist(foo)</code> (and <code>python3dist(foo)</code>) where <code>foo</code> is the upstream Python package distribution name and X is the Python minor version. That way you can require any Python package without knowing under which name it was packaged in Fedora. And these tags are also automatically used by the Python dependency generator, which reads upstream Python metadata and creates dependencies on these Provides. However, Python extras are not yet handled by the Provides tags which leads to imperfections and problems in declared dependencies. === Status quo === Currently in Fedora (before this change), no package provides <code>python3.Xdist(foo[bar])</code> for the <code>foo[bar]</code> Python extra. As a direct result of this, no package can require it. The automatic RPM Python dist dependency generator only generates an incomplete requirement on the base package (<code>python3.Xdist(foo)</code>) in such cases. The transitive extra dependencies were often needed to be hardcoded manually. I.e. when <code>foo</code> requires <code>bar[baz]</code>, package <code>bar</code> does not require the additional dependencies for the <code>bar[baz]</code> extra. Thus <code>foo</code> needs to hardcode those dependencies manually. For example: [https://src.fedoraproject.org/rpms/poetry/c/97fa3d908]. This leads to possibly missing, broken and/or outdated superfluous dependencies. === Extras metapackages === In this change proposal, we propose to solve the problem using metapackages. The following metapackage represents the <code>setuptools_scm[toml]</code> extra for the <code>python3-setuptools_scm</code> RPM package (<code>python-setuptools_scm</code> source package): %package -n python3-setuptools_scm+toml Summary: Metapackage for python3-setuptools_scm: toml extra Requires: python3-setuptools_scm = %{?epoch:%{epoch}:}%{version}-%{release} %description -n python3-setuptools_scm+toml This is a metapackage bringing in toml extra requires for python3-setuptools_scm. It contains no code, just makes sure the dependencies are installed. %files -n python3-setuptools_scm+toml %ghost %{python3_sitelib}/*.egg-info Notice several things: * The package has a hard dependency on <code>python3-setuptools_scm = %{?epoch:%{epoch}:}%{version}-%{release}</code>. While this could be in theory generated by the dependency generator, the change owners have decided not to do that to allow certain leeway for experimentation. However, the dependency will created by the macro helper below. Technically, <code>%{?_isa}</code> should also be used for arched packages, but in practice we believe it can be omitted. * The package contains no files except the <code>%ghost</code> metadata. This is needed for the dependency generator to have access to the upstream metadata of this package. The [https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/19 updated RPM Python dist dependency generator] parses the extras name from the subpackage name by splitting it on the <code>+</code> sign. This naming scheme is not new, it is copied from Rust packaging. Five Python packages in Fedora already use the same scheme for similar metapackages representing Python extras. And normalized Python distribution package names (or extras names) don't naturally contain the <code>+</code> sign. (Neither do existing Fedora packages prefixed with <code>python3-</code>, except the 5 components already mentioned.) The metapackage can have additional features if desired. For example: * It can obsolete/provide other names (e.g. obsoleted extras packages) * It can have manual strong or weak dependencies on other (possibly non-Python) packages * It can contain files excluded from the "base" package (if such files only make sense with the extra and the base package does not fail without them) The "base" package (in this case <code>python3-setuptools_scm</code>) can optionally Require/Recommend/Suggest a Python extras metapackage if the packager deems it useful. The change for the RPM Python dist dependency generator is prepared in: * https://github.com/torsava/rpm/pull/2 (PR for upstream RPM will follow after this change is discussed in Fedora) * https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/19 (to be adapted based on feedback and merged in Fedora once the change is approved) === Macro helper === For the most common case, the change owners have prepared a macro helper in https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/59 To generate the example above, it should be used like this: %{?python_extras_subpkg:%python_extras_subpkg -n python3-setuptools_scm -i %{python3_sitelib}/*.egg-info toml} * The <code>%{?python_extras_subpkg:...}</code> way of using this macro ensures the spec file remains valid for older Fedora/EL releases, where this code will do nothing. * The <code>-n</code> option specifies the name of the "base" package. * The <code>-i</code> option specifies the <code>%files %ghost</code> path (glob) to the the metadata directory (the <code>.dist-info</code> or <code>.egg-info</code> directory) * The one or more positional arguments specify the extra(s) name(s) — multiple metapackages are generated when multiple names are provided. Other possible arguments: * The <code>-f</code> option (conflicts with <code>-i</code> and <code>-F</code>) can specify the relative path to the filelist for this metapackage (which should contain the <code>%files %ghost</code> path (glob) to the the metadata directory). This API is prepared for integration with <code>pyproject-rpm-macros</code>. * The <code>-F</code> flag (conflicts with <code>-i</code> and <code>-f</code>) can be used to skip the <code>%files</code> section entirely (if the packager wants to construct it manually). Note that this macro generates all the subpackage definition sections (<code>%package</code> including the Summary and Requires on the base package, <code>%description</code> and <code>%files</code>), and hence it cannot be extended with custom Provides/Obsoletes/Requires/etc. This macro is designed to fit the most common uses. It doesn't currently cover all use cases. Packagers can, however, construct the subpackage manually if they need custom features not covered by <code>%python_extras_subpkg</code>. In the future, the API of the macro can be extended if there is demand. See the [https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/59 linked pull request] for example outputs. Due to technical limitations, the macro helper never generates requirement on the arched <code>BASE_PACKAGE%{?_isa} = %{?epoch:%{epoch}:}%{version}-%{release}</code>. It only adds <code>Requires: BASE_PACKAGE = %{?epoch:%{epoch}:}%{version}-%{release}</code>) because a [https://github.com/rpm-software-management/rpm/issues/689 macro cannot reliably detect if the subpackage is arched or not]. The change owners believe the resolver will do the right thing by default. If there are problems with this approach, an additional flag (such as <code>-a</code>) can be introduced to indicate an arched base package. === Why is there no automatic extras discovery? === [http://lists.rpm.org/pipermail/rpm-ecosystem/2020-February/000730.html RPM is not capable of creating dynamic subpackages] based on the content in <code>%{buildroot}</code> or on the unpacked sources (<code>%{_builddir}</code>) yet. Hence, we require the packager to manually list which Python extras (if any) should be packaged as metapackages. Not all extras are useful for us anyway, as there are often extras representing the build/dev/doc/test dependencies of the project. In the future (once/if RPM supports this), the generators can be extended with auto-discovery of Python extras (with filtering). === Automatic provides generator === To continue with our example, the <code>python3-setuptools_scm+toml</code> subpackage will Provide <code>python3.Xdist(setuptools_scm[toml])</code> (and also <code>python3dist(setuptools_scm[toml])</code>). An attempt to package a nonexsiting extra (e.g. <code>python3-setuptools_scm+nopenopenope</code>) will result in build failure with an human-readable error message. === Automatic requires generator === If a Python package requires <code>setuptools_scm[toml]</code>, the Fedora RPM package will require <code>python3.Xdist(setuptools_scm[toml])</code> and also <code>python3.Xdist(setuptools_scm)</code>. In theory, the second requirement is redundant, but in practice, it makes it easier (and less error prone) to query package dependencies in Fedora (e.g. using <code>dnf repoquery</code>). The packaged extras will also Require additional dependencies listed in their Python metadata, in the case of <code>python3-setuptools_scm+toml</code>, it will require <code>python3.Xdist(toml)</code> (because on the Python level, <code>setuptools_scm[toml]</code> requires <code>toml</code>). Packagers can opt out from automatically generated dependencies on Python extras by defining the <code>%_python_no_extras_requires</code> macro to any value (usually <code>1</code>) in the spec file. This should be only a a temporary measure until the missing extra is packaged. If the upstream dependency information is not accurate, please work with upstream to fix it. === Coordinated effort to avoid breakage === The change owners have [https://copr.fedorainfracloud.org/coprs/g/python/python-extras/ collected data about non-modular packages in Copr]. Note that ~270 packages failed to build for unrelated reasons and hence we miss data for them. However, ~3300 packages built successfully. The following extras metapackages will be added to avoid broken dependencies: autobahn[twisted] cachecontrol[filecache] cairocffi[xcb] cli-helpers[styles] docker[ssh] fonttools[ufo] fonttools[unicode] ipython[notebook] lunr[languages] oauthlib[signedtoken] pyjwt[crypto] raven[flask] requests[security] requests[socks] tabulate[widechars] twisted[tls] vistir[spinner] The following components will be modified: python-autobahn python-CacheControl python-cairocffi python-cli-helpers python-docker fonttools ipython python-lunr python-oauthlib python-jwt python-raven python-requests python-tabulate python-twisted python-vistir When we added the metapackages for these extras in our testing Copr, no new broken requires on Python extras were generated. In other words, these new extras subpackages don't require adding any more extras subpackages. No extras are required by the remaining Python 2 packages in Fedora. Once the change in the dependency generator is deployed in rawhide, the change owners will monitor all newly added requires on missing extras and will add new metapackages as needed. 5 source packages in Fedora already have Python extras meta-subpackages with the proposed naming pattern, but they don't have any listed <code>%files</code>. They will be non-intrusively adapted via pull requests — by adding the <code>%ghost</code> file entry to the metapackage(s). Maintainers can then decide whether to opt for simpler rawhide only specfile with <code>%python_extras_subpkg</code> or to maintain the current compatibility. This concerns the following 18 subpackages: python3-dask+{array,bag,dataframe,delayed} python3-django-storages+{azure,boto,boto3,dropbox,libcloud,sftp} python3-dns-lexicon+{easyname,gratisdns,henet,hetzner,plesk,route53} python3-drf-yasg+validation python3-prometheus_client+twisted ==== Modular packages ==== The change owners are only cable of monitoring and adapting non-modular packages. Due to long standing issues, we are unable to inspect, query (or do a targeted rebuild of) modular content: * https://pagure.io/modularity/issue/160 * https://pagure.io/modularity/issue/163 * https://pagure.io/modularity/issue/165 If there are people available to help with this problem, the change owners will gladly accept their help, we are not excluding modular content because we would like to do it, but because we don't know how to work with it at scale. === How to add Python extras subpackage to my package? === In this section, we'll describe a step-by-step guide of adding the Python extras subpackage to your package. Imagine you maintain <code>python-requests</code> and a maintainer of a dependent package contacts you: "I would like you to add a subpackage for <code>requests[security]</code>, because my package requires it." # Locate the <code>%files</code> section for <code>python3-requests</code> package in <code>python-requests.spec</code>. # Find the entry for <code>.egg-info</code> or <code>.dist-info</code> metadata directory. If the entry is generalized with globs like <code>%{python3_sitelib}/*</code>, please make the <code>%files</code> section more explicit while at it. Copy the line with the metadata directory. In this guide we assume it is <code>%{python3_sitelib}/*.egg-info</code>. # Locate the <code>%description</code> of the <code>python3-requests</code> package. # After the description, add: <code>%{?python_extras_subpkg:%python_extras_subpkg -n python3-requests -i %{python3_sitelib}/*.egg-info security}</code> on a separate line. # Build the package (e.g. in local mock). # Verify the <code>python3-requests+security</code> package is build and provides <code>python3dist(requests[security])</code>. # See if the new extras package doesn't have dependencies on packages missing from Fedora (extras or "basic") and proceed with adding those if needed. # Ship the change in Fedora 33+. It should do nothing in Fedora 31/32 or current EPELs. === Packaging guidelines === The change owners will describe this concept in the Python packaging guidelines and will propose the following rules for the Fedora Packaging Committee to approve: * Packagers MAY add Python extras metapackages as needed. * The Python extras metapackages MUST require the base package (exact NEVR). * Packagers MAY add strong or weak dependencies on the extras metapackages from the base package as they see fit. * Packagers SHOULD NOT add Python extras metapackages with dependencies only useful for maintaining the package (usually extras called dev/test/doc/build/...). ** Optional: Packagers MAY package tests separately into the <code>[test]</code> or <code>[testing]</code> extras subpackage. * If a Fedora package requires a Python extra of a different package, the extras metapackage MUST be added to that package to avoid broken dependencies. * Packagers MAY temporarily disable the automatic requires on extras subpackages (by defining <code>%_python_no_extras_requires</code>) until the missing metapackage is introduced, but they SHOULD notify the maintainer of the package they depend on about the situation. * If upstream drops an extra, even though it is discouraged by upstream documentation ([https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extra… see final paragraph]), the metapackage SHOULD be Obsoleted from the base package or, if there is continuity, from another extras metapackage. * If the upstream Python package name contains <code>+</code>, it MUST be replaced with <code>-</code> in package names (in accordance with the upstream [https://www.python.org/dev/peps/pep-0503/#normalized-names Python package names normalization]). == Feedback == This has been briefly discussed in general terms [https://github.com/rpm-software-management/rpm/issues/1061 upstream]. People tend to agree that some solution is needed. The concrete proposal contained in this Fedora Change is based on the discussion, but has received no feedback yet. More feedback will be documented here once the change proposal is announced and discussed in Fedora. == Benefit to Fedora == * Packages will have more accurate automatic dependencies, and the hard-to-maintain and error prone manual transitive (and other) dependencies can be dropped. * There will be less missing and redundant dependencies. * Python packagers will have less manual dependencies to worry about and less problems to workaround. * The handling of Python extras will be standardized. * Overall, the Python ecosystem in Fedora will be closer to upstream. == Scope == * Proposal owners: ** Polish and merge the code changes for <code>python-rpm-generators</code> and <code>python-rpm-macros</code> linked above. ** Add the 17 missing extras metapackages listed in this change to avoid broken dependencies (using pull requests or provenpackager powers if need be). ** Adapt the 5 existing Python extras subpackages listed in this change to work with the dependency generator (using pull requests, or provenpackager powers if need be). ** Monitor new dependencies on Python extras subpackages, add extras subpackages where needed (using pull requests, or provenpackager powers if need be). ** Propose the updated Python packaging guidelines to FPC for approval. ** Provide help and guidance for packagers. ** Optional: Prepare <code>pyproject-rpm-macros</code> integration of this change. * Other developers: ** No immediate action necessary. ** They can opt in for more metapackages with extras. ** They can review and merge pull requests. ** They should follow the updated Python packaging guidelines if the changes are approved by FPC. * Release engineering: No releng impact anticipated. The new dependencies will be primarily generated by the mass rebuild, but if the mass rebuild is missed, the package maintainers or change owners can rebuild the packages that will gain the new automatic Requires is on Python extras. * Policies and guidelines: Yes, see detailed description. * Trademark approval: Not needed for this Change. == Upgrade/compatibility impact == No impact anticipated. == How To Test == Check that there are packages that Require <code>python3.9dist(basename[extrasname])</code>. You can use the following repoquery: dnf repoquery --repo=rawhide --whatrequires 'python3.9dist(*\[*\])' Check that there are Python extras metapackages with the correct Provides, for example by installing the packages returned by the above query, or manually via queries like: dnf repoquery --repo=rawhide --whatprovides 'python3.9dist(requests\[security\])' To query all existing Python extras metapackages, you can use: dnf repoquery --repo=rawhide --provides -a | grep -E 'python(3\.9|2\.7)dist\(\S+\[\S+\]\)' And lastly, to query all required Python extras metapackages: dnf repoquery --repo=rawhide --requires -a | grep -E 'python(3\.9|2\.7)dist\(\S+\[\S+\]\)' == User Experience == When installing Python RPM packages, the dependencies are more likely to fulfill user expectations, as they will more closely adhere to the behavior of pip (the Python package installer). == Dependencies == Nothing. == Contingency Plan == * Contingency mechanism: (What to do? Who will do it?) ** Soft: The change owners will disable the requirements generator by default and rebuild (or untag if FTBFS) packages with broken dependencies caused by the change. ** Hard: The change owners will revert everything and rebuild (or untag if FTBFS) packages with new requirements/provides caused by the change. * Contingency deadline: Beta freeze * Blocks release? No == Documentation == The packaging guidelines will be the documentation if approved. If not, this Fedora Change shall serve as the documentation. -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

1 0

Re: Please BuildRequire python3-setuptools explicitly
by Gabriel L. Somlo 25 Jun '20

25 Jun '20

On Tue, Jun 23, 2020 at 06:26:23PM +0200, Tomas Hrnciar wrote: > somlo python-logzero python-migen Fixed in rawhide. Thanks, --Gabriel

1 0

New dockerhub organization
by Lumir Balhar 25 Jun '20

25 Jun '20

Hello. Because the fedora-python-tox container image is getting popular, I've decided to move it from my own account to more official-looking space. Therefore I've created a new organization fedorapython (no dashes allowed there :( ) on Docker hub and cloned the repository there: https://hub.docker.com/repository/docker/fedorapython/fedora-python-tox Let me know if you want to migrate some of your projects there. Have a nice day. Lumír

1 0

Deprecating python-pytoml?
by Miro Hrončok 25 Jun '20

25 Jun '20

Hello. Upstream pytoml is deprecated in favor of toml: https://pypi.org/project/pytoml/ Would you mind if I submit a self contained change to deprecate python-pytoml in Fedora 33? Similar to https://fedoraproject.org/wiki/Changes/DeprecateNose -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

Re: Please BuildRequire python3-setuptools explicitly
by Miro Hrončok 24 Jun '20

24 Jun '20

On 23. 06. 20 18:36, Adam Williamson wrote: > IMBW, but I think I recall the Python packaging guidelines specifically > said that you could or should (I forget which) just BR python-devel and > not BR python-setuptools at some point. At this point there seems to be > no explicit mention, but the sample spec file looks a lot like a > project that uses setuptools, but does not explicitly BuildRequire > it... I'll work towards adding the explicit BR there. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

Orphaned python-sphinxcontrib-issuetracker
by Orion Poplawski 23 Jun '20

23 Jun '20

I've orphaned python-sphinxcontrib-issuetracker as I no longer need it. It does not appear to be used by anything in the distro and so probably should be retired, but I figured I would do this step first to see if anyone wants to take it over. Orion -- Orion Poplawski Manager of NWRA Technical Systems 720-772-5637 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane orion(a)nwra.com Boulder, CO 80301 https://www.nwra.com/

1 0

python-virtualenv 20.0.23 in rawhide
by Lumir Balhar 21 Jun '20

21 Jun '20

Hello. Virtualenv has been completely rewritten so we were waiting for it to settle a little bit and it wasn't simple to adapt our downstream modification (to use our own wheels) but the latest version 20.0.23 just landed in rawhide. We tested it heavily but it might still bring some incompatibilities so feel free to let us know if you spot any troubles. Have a nice day. Lumír

1 0

Updating tox to 3.15.x in Fedora 32
by Miro Hrončok 21 Jun '20

21 Jun '20

Hello. I'd like to update tox to 3.15.x in Fedora 32, because it has a feature I'd like to use (it works with pyproject.toml+setup.cfg (sans setup.py) projects). https://tox.readthedocs.io/en/latest/changelog.html#v3-15-0-2020-05-02 The changelog doesn't reveal any breakages, but I'd rather ask here if anybody has a reason not to. (The updated package builds, installs and works reliably for me.) Thanks, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

Monthly highlights from the Red Hat's Python-maint team, May 2020
by Tomas Orsava 11 Jun '20

11 Jun '20

Hi, we're the Python-maint team from Red Hat, and we'd like to experiment with sharing with you what we are working on from month to month. There are some things we should not disclose (such as embargoed CVEs), but the majority of what we do is out in public. Please let us know if you find this useful and whether you'd like us to continue. So here are the highlights of what we've been working on this May, 2020: *Fedora:* * Python updated to Python 3.9 in rawhide (Fedora 33) * ~3000 source packages rebuilt * ~100 reamain problematic (fail to build) * python3X packages renamed to python3.X in rawhide * Python RPM depndency generators: * Backported proposed upstream changes, added new test suite to Fedora CI * Working on handling of proposed Python extras subpackages * pipenv continually being rebased to new versions, currently 2020.5.28 * Test in copr:https://copr.fedorainfracloud.org/coprs/g/python/pipenv/ * Provided feedback and fixes upstream * pytest updated to 5 on rawhide (compat package with pytest 4 introduced) * pip updated to version 20.1, changes in Python RPM macros were needed to workaround problems with new PEP 610 * https://www.python.org/dev/peps/pep-0610/ * New RPM macros for easier packaging: %py_provides, %pytest and %python3_shebang_fix * %py_provides:https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/52 * %python3_shebang_fixhttps://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/58 * %pytesthttps://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/55 * Improved %pypi_source macro * Removes tildes (`~`) from %version so that it works with alpha/beta/dev releases out of the box * https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/57 * Python 3.8.3 updated across Fedora releases * pypy and pypy 3 updated to versions 7.3.1 * provided feedback upstream for new Cython alpha release *CPython:* * CPython 3.9b1 was released: seehttps://docs.python.org/dev/whatsnew/3.9.html * Fixing a CPython type reference count issue found by PySide * Python 3.9 is now FIPS compatible upstream. * Proof-of-concept of subinterpreters running in parellel with one GIL per interpreter: "Subinterpreters 4x faster than sequential execution or threads on CPU-bound workload"https://mail.python.org/archives/list/python-dev@python.org/thread/S5GZZCER… * Progress on subinterpreter isolation work: most free lists are now per-interpreter.https://bugs.python.org/issue40521 * Progress on hiding implementation details from the C API: Py_TYPE(), Py_REFCNT() and Py_SIZE() can no longer be used as l-value. Use Py_SET_TYPE(), Py_SET_REFCNT() and Py_SET_SIZE() instead.https://mail.python.org/archives/list/capi-sig@python.org/thread/HY… * New --hardlink-dupes option in compileall: "If two .pyc files with different optimization level have the same content, use hard links to consolidate duplicate files."https://docs.python.org/dev/library/compileall.html#cmdoption-compileall-ha… * Helped to get features merged before the 3.9 feature freeze: removeprefix/removesuffix (PEP 616), new zoneinfo module (PEP 615). Fixed buildbot failures and a few reference leaks. * Python Steering Council Community Address with our team member Victor Stinner:https://www.youtube.com/watch?v=xX8fGuh4T_o * * *Red Hat Enterprise Linux:* * Overhauling the FIPS patch for python3 in RHEL 7 * Created an Insights rule candidate that verifies Python alternatives * Backporting fix for Python's CVE-2019-16056 to older RHEL 7 versions * For RHEL9 preparations -- see the Fedora section *Red Hat Software Collections:* * Python 3.8 from RHSCL 3.5 added for s2i-python-container upstream *Misc:* * Releasing a GitHub action to run Tox on Fedora:https://github.com/marketplace/actions/python-tox-on-fedora All the best, Python-maint

1 0

Python 3.9 (not so) mini mass rebuilds and when they will/can happen
by Miro Hrončok 11 Jun '20

11 Jun '20

Hello. Kevin and Smooge, you are CCed in case you have some concerns regarding the data centre move. Jitka and Zuzka, you are CCed because you plan mini mass rebuilds of your own (Perl and Node.js). Ben, you are CCed because I considered it a good idea. Python 3.9.0b1 was released earlier this week and we are running test rebuilds in Copr. Things are going suspiciously well so far, except for some critical stuff -- but we have that covered for now. Hopefully nobody decides to upgrade something major right now :P If everything goes extremely well, I expect us to start rebuilding the initial bootstrap sequence (438 srpms) in Koji side tag this week, the remaining packages (2733) the next week. We expect some packages to fail (~200-300) and merge the side tag, so it's not open too long. We can help firefight things if this blocks anybody. There are possible followup rebuilds in play for two particular reasons: 1) The ABI can change between beta releases of Python and all arched Python packages (661) will need a rebuild. This can happen (but doesn't have to) at: - beta 2: 2020-06-08+ - beta 3: 2020-06-29+ - beta 4: 2020-07-20+ Source: https://www.python.org/dev/peps/pep-0596/#schedule Anytime during the release cycle, the format (magic number) of bytecode cache (.pyc files) can change. The problems with that are not rawhide-critical, but we don't want to ship wrong bytecode cache in Fedora 33 GA. Hence we'll check the numbers prior beta freeze (2020-08-25) and rebuild what's needed (can be anything between 0 and 3000). -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel June 2020