python-devel November 2019

python-devel@lists.fedoraproject.org

8 participants
8 discussions

Re: Pagure for EL8 (EPEL8)
by Neal Gompa 25 Mar '20

25 Mar '20

On Mon, Nov 18, 2019 at 11:18 AM Kevin Fenzi <kevin(a)scrye.com> wrote: > > On Sat, Nov 16, 2019 at 05:37:16PM -0500, Neal Gompa wrote: > > I've done an early build locally to determine what's needed to make > > this possible. The following report from DNF indicates the missing > > packages that need to be added to EPEL 8 before I can introduce Pagure > > into EPEL8: > > > > Problem 1: conflicting requests > > - nothing provides python3-jenkins needed by pagure-ci-5.8-1.el8.noarch > > Problem 2: conflicting requests > > - nothing provides gitolite3 needed by pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(binaryornot) needed by > > pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(celery) needed by pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(flask-wtf) needed by pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(redis) needed by pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(straight.plugin) needed by > > pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(wtforms) needed by pagure-5.8-1.el8.noarch > > - nothing provides python3.6dist(pygit2) >= 0.26.0 needed by > > pagure-5.8-1.el8.noarch > > Problem 3: conflicting requests > > - nothing provides python3-trololio needed by pagure-ev-5.8-1.el8.noarch > > > > One of the reasons I'd like to have this done sooner rather than later > > is so that we can drop Python 2 support from Pagure with version 6.0. > > I think it's quite reasonable to say that version 6.0 isn't going to > > happen until we can get our Pagure servers running on EL8 using Python > > 3. > > > > So now, I need some help making this happen. I already own trololio, > > and I'm going to make that available in EPEL 8 ASAP. Can anyone help > > with some of the other dependencies here? > > Can you give a list with maintainers? I'm not sure off hand how many of > those are maintained by me/infra-sig, but any I can I would be happy to > add in. There's a few that are in testing I think already... > Sure, here's a list so far (package: maintainer): * gitolite3: limb * python-jenkins: cottsay * python-binaryornot: pingou * python-celery: bowlofeggs * python-flask-wtf: pingou * python-redis: kevin * python-straight-plugin: pingou * python-wtforms: kumarpraveen * python-pygit2: pwalter * python-trololio: ngompa I've already got trololio going: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-6c443851dd -- 真実はいつも一つ！/ Always, there's only one truth!

1 3

Python 3.9 package review request
by Miro Hrončok 20 Nov '19

20 Nov '19

Python 3.9.0a1 is out: https://www.python.org/downloads/release/python-390a1/ Here is the package review request for python39: https://bugzilla.redhat.com/show_bug.cgi?id=1774417 -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 1

Pagure for EL8 (EPEL8)
by Neal Gompa 16 Nov '19

16 Nov '19

Hey all, In the interest of helping to modernize the infrastructure Fedora runs on, I'm working on introducing Pagure into EPEL8. This will hopefully allow us to upgrade our Pagure instances to use RHEL 8 instead of RHEL 7, and notably, make the transition (mostly) complete for moving all Python software Fedora runs to Python 3. I've done an early build locally to determine what's needed to make this possible. The following report from DNF indicates the missing packages that need to be added to EPEL 8 before I can introduce Pagure into EPEL8: Problem 1: conflicting requests - nothing provides python3-jenkins needed by pagure-ci-5.8-1.el8.noarch Problem 2: conflicting requests - nothing provides gitolite3 needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(binaryornot) needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(celery) needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(flask-wtf) needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(redis) needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(straight.plugin) needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(wtforms) needed by pagure-5.8-1.el8.noarch - nothing provides python3.6dist(pygit2) >= 0.26.0 needed by pagure-5.8-1.el8.noarch Problem 3: conflicting requests - nothing provides python3-trololio needed by pagure-ev-5.8-1.el8.noarch One of the reasons I'd like to have this done sooner rather than later is so that we can drop Python 2 support from Pagure with version 6.0. I think it's quite reasonable to say that version 6.0 isn't going to happen until we can get our Pagure servers running on EL8 using Python 3. So now, I need some help making this happen. I already own trololio, and I'm going to make that available in EPEL 8 ASAP. Can anyone help with some of the other dependencies here? Thanks in advance, Neal -- 真実はいつも一つ！/ Always, there's only one truth!

2 1

Re: Python bindings for protobuf on RHEL/CentOS/EPEL 8
by Petr Viktorin 14 Nov '19

14 Nov '19

On 2019-11-14 16:13, Denis Arnaud wrote: > > > Thanks Petr for your answer! > > Let's give the COPR short-term solution a try, then :) > > What is frustrating is that we cannot get our hands on the package > source for Protobuf for RHEL/CentOS, and we cannot therefore know who > maintains it at RedHat. The maintainer is (usually, effectively) also the Bugzilla assignee. > For Boost for instance, which is also a RedHat > core package, I know the RedHat maintainer (Jon Wakely) and can > co-maintain Boost with him and have an influence on (non-EPEL) > RHEL/CentOS Boost. In my personal opinion, that's the way it should be; it benefits both Red Hat and the community. > At least, we get visibility on when, why and how > Boost is rebuilt on RHEL/CentOS. It would be nice to achieve the same > with Protobuf. But I have no clue on how to identify who at RedHat is > (are) the main maintainer(s) of Protobuf. I am sure they would be > interested if I/we can build protobuf with Python bindings for > RHEL/CentOS 8 (at least on EPEL). > And it seems strange to me that RHEL/CentOS misses Python bindings for > Protobuf (which is the cornerstone to many frameworks (e.g., gRPC)). I > wonder how many applications work at all. Or maybe those Python bindings > are build-time only in the new modularization framework? > It would help to have a way, for Fedora/EPEL volunteers, to be able to > reach out to RedHat core package maintainers, at least for > cross-awareness, knowledge sharing and potential collaboration. > > Kind regards > > Denis > > > 14 Nov. 2019 15:27, Petr Viktorin <pviktori(a)redhat.com > <mailto:pviktori@redhat.com>> wrote: > > On 2019-11-11 11:45, Denis Arnaud wrote: > > Hi, > > > > the Python (3) bindings are missing on RHEL/CentOS/EPEL 8 for > the > > protobuf package (https://src.fedoraproject.org/rpms/protobuf). > > A bug request has been created on Bugzilla > > (https://bugzilla.redhat.com/show_bug.cgi?id=1765844), but as > no status > > has been given, I was wondering whether someone could shed > some light on > > the context. > > > > Since protobuf is a RedHat core package (maintained by RedHat > and > > therefore not managed by Fedora/EPEL), it appears as a kind > of black box > > from Fedora perspective. On Fedora (Rawhide, 31), the Python (3) > > bindings are generated/packaged (see for instance > > > https://bodhi.fedoraproject.org/updates/FEDORA-2019-e3a662fe8b and > > https://koji.fedoraproject.org/koji/rpminfo?rpmID=19440119), > but for > > some reason, those Python bindings are not generated by > RedHat for > > RHEL/CentOS 8. > > > > 1. Would anyone from RedHat be able to provide some heads up > on why > > those Python 3 bindings are missing for Protobuf, and/or an > approximate > > timeline for when it would be generated? > > Hi! > We just talked about this among Red Hat python-maint. (Sorry for > the > delay!) We don't have influence over this package; unfortunately we > can't help directly. > > > 2. Would RedHat need help with packaging protobuf on > RHEL/CentOS/EPEL 8? > > At this level "Red Hat" is not a single entity. You're probably not > reaching the right people on this list. > If the packager isn't active in Fedora or on Bugzilla, then > unfortunately, the way to get them to respond is to file a customer > support ticket. > Another way to approach this is change EPEL rules somehow to > make it > possible to build subpackages like this. From what I understand, > it's > being discussed, but it'll take time to get sorted out. > > > 3. Would you recommend another way for Fedora packagers/users > to get > > their hands on the python3-protobuf/protobuf-python3 package? > For > > instance, through COPR, or some module we may have missed. > > Building a replacement protobuf package in COPR is the best > short-term > solution. > > Sorry I couldn't be more helpful. >

1 0

Provisional pyproject RPM macros: Dynamic BuildRequires for Python packages
by Miro Hrončok 14 Nov '19

14 Nov '19

Hello fellow Python packagers. This is an announcement about a new set of RPM macros you can use to build PEP 517/518 enabled packages, that is Python packages that have the pyproject.toml file. https://www.python.org/dev/peps/pep-0517/ https://www.python.org/dev/peps/pep-0518/ https://snarky.ca/clarifying-pep-518/ The set of macros is designed for modern packaging with dynamic buildrequires in mind. The macros are in the pyproject-rpm-macros package and you can use them like this: BuildRequires: pyproject-rpm-macros %prep ... %generate_buildrequires %pyproject_buildrequires -t %build %pyproject_wheel %install %pyproject_install %check %tox See the full documentation of the macros: https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/master/f/READM… See example spec files: https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/master/f/tests (These use setuptools (setup.py), flit and poetry for build backends, but you cannot tell that from the specfiles - BuildRequires are generated dynamically from upstream metadata.) The macros are **provisional**, i.e. their API may be changed upon feedback received from you. We are not (yet) interested in a general "update all the Python packages" hunt, but rather in early adopters. If you have questions, ask here. We'll gladly extend the docs if something is not clear. If you find bugs, report them in bugzilla or here. Likewise for RFEs. Happy hacking, Python Maintenance

3 2

Python bindings for protobuf on RHEL/CentOS/EPEL 8
by Denis Arnaud 13 Nov '19

13 Nov '19

Hi, the Python (3) bindings are missing on RHEL/CentOS/EPEL 8 for the protobuf package (https://src.fedoraproject.org/rpms/protobuf). A bug request has been created on Bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1765844), but as no status has been given, I was wondering whether someone could shed some light on the context. Since protobuf is a RedHat core package (maintained by RedHat and therefore not managed by Fedora/EPEL), it appears as a kind of black box from Fedora perspective. On Fedora (Rawhide, 31), the Python (3) bindings are generated/packaged (see for instance https://bodhi.fedoraproject.org/updates/FEDORA-2019-e3a662fe8b and https://koji.fedoraproject.org/koji/rpminfo?rpmID=19440119), but for some reason, those Python bindings are not generated by RedHat for RHEL/CentOS 8. 1. Would anyone from RedHat be able to provide some heads up on why those Python 3 bindings are missing for Protobuf, and/or an approximate timeline for when it would be generated? 2. Would RedHat need help with packaging protobuf on RHEL/CentOS/EPEL 8? 3. Would you recommend another way for Fedora packagers/users to get their hands on the python3-protobuf/protobuf-python3 package? For instance, through COPR, or some module we may have missed. Kind regards Denis

1 0

Python Annual Release Cycle adjusted to match odd-numbered Fedora releases
by Miro Hrončok 07 Nov '19

07 Nov '19

Hello, I'd like to inform you that [PEP 602] "Annual Release Cycle for Python" has been approved and [PEP 596] "Python 3.9 Release Schedule" is pending approval: tl;dr New Python 3.X versions will be released annually, with RC period adjusted to make it possible to update Python in odd-numbered Fedora releases. We plan to submit the Python 3.9 change proposal for Fedora 33 after the first Python 3.9 alpha (expected in ~2 weeks). The Python 3.8 update was postponed from Fedora 31 to Fedora 32 because the schedule was too tight. With 3.9 and the recent adjustments, we expect that Fedora 33 is a reasonable target release, but we are prepared to reevaluate that after branching. Like with 3.8, we plan to rebuild Fedora packages with Python 3.9 pre-releases, file bugs, and coordinate fixes both to Python and the software that uses it. We hope to make that process smoother, but even more effective at finding and resolving incompatibilities. With the shorter cycle, we anticipate less incompatibilities. Quotes from PEP 602: (Rationale and Goals) "This change provides the following advantages: ... allows for synchronizing the schedule of Python release management with external distributors like Fedora who've been historically very helpful in finding regressions early not only in core Python but also in third-party libraries, helping moving the community forward to support the latest version of Python from Day 1" (Rejected Ideas) "While [keeping 4 betas over 4 months and a final month for the release candidate] would make the release calendar a bit cleaner, it would make it very hard for external distributors like Fedora to release the newest version of Python as soon as possible. We are adjusting Python's calendar here in the hope that this will enable Fedora to integrate the newest version of Python with the newest version of Fedora as both are being developed which makes both projects better." Quotes from the [python-dev] mailing list: "...in order to give the community enough time to provide feedback in the betas while having enough time to thoroughly test the RC and to prep for the final release so the delay from Python's final release to any new project releases is minimal. It should also fit into the release schedule of Linux distributions like Fedora better than previously proposed so the distributions can test the RC when they start preparing for their own October releases." -- Brett Cannon, Python Steering Council member "[2 months for RCs instead of 1] allows for synchronizing the schedule of Python release management with Fedora. They've been historically very helpful in early finding regressions not only in core Python but also in third-party libraries, helping moving the community forward. It seems like a bargain to make a slight adjustment of our schedule to help Fedora help us make 3.9 and beyond better releases." -- Łukasz Langa, Python 3.8 and 3.9 Release Manager [PEP 596] https://www.python.org/dev/peps/pep-0596/ [PEP 602] https://www.python.org/dev/peps/pep-0602/ [python-dev] https://mail.python.org/archives/list/python-dev@python.org/thread/KE7OS4PZ… -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

Building the python executable statically with libpython*.a for better performance
by Miro Hrončok 04 Nov '19

04 Nov '19

Hello, we've been recently approached by a colleague from Red Hat working on performance (CCed). According to their testing, Fedora Python performance could be improved by ~15% by building /usr/bin/python* statically with libpython*.a. That sounds like a worthy thing to do. https://bugzilla.redhat.com/show_bug.cgi?id=1749479 Since Python 3.8 Python extension modules are no longer linked to libpython.so and we can do the following: * build /usr/bin/python3(.8) statically with libpython*.a * build and ship libpython3.8.so.1.0 for packages that "embed" Python The change in the python3 package is trivial: https://src.fedoraproject.org/rpms/python3/pull-request/133 However it can have serious impact on Python extension modules that are linked to libpython3.8.so.1.0 by various "nonstandard" build mechanisms or by compiling code for Python extension module and code that embeds Python into one file. We will likely propose a Fedora 32 Change for this, however I'm opening this topic for discussion before we do so. Testing the proposed Pull Request with your code is also helpful. Let me know how can we make that easier (e.g. if you want a Copr or a Fedora 30/31 python38 package with this change). WDYT? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 6

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel November 2019