Hello Pythonistas.
(I've CC'ed the devel list for further exposure. But let's discuss this on python-devel list please to avoid noise.)
We would like ro rename the "python3" component (SRPM) to "python39" to make maintaining various Python versions in various Fedora versions easier. The names of binary RPMs would be unchanged; you still do `dnf install python3`.
The change should only affect packagers of Python itself; not users or packagers of software written in Python. The only user-visible or general-packager-visible change is a different component (SRPM) name.
Details follow.
Currently, we build the Python interpreters this way:
- the "main" Python 3 interpreter component is called python3 - it builds a couple of subpackages, like python3-libs or python3-devel - the "alternate" Python 3 versions components are called python3X (python37, python39...)
For example, Fedora 31 has:
- python36 - python3 (the "main" Python, which is 3.7.x) - python38 - python39
(python37 is retired in f31, but present in Fedora versions where python3 != 3.7)
This makes updating the "main" Python fairly complicated. To upgrade from 3.7 to 3.8 in f32, we had to:
1) retire python38 2) upgrade python3 from 3.7 to 3.8 (this makes python3 obsolete python38) 3) re-introduce (unretire) python37
Such an update comes every few years: - https://fedoraproject.org/wiki/Changes/Python3.7 (f29) - https://fedoraproject.org/wiki/Changes/Python3.8 (f32) - https://fedoraproject.org/wiki/Changes/Python3.9 (f33)
What's even more complicated is backporting patches across Fedora releases. Assume there is an important upgrade/fix for Python 3.7 we wish to apply for all Fedoras including stable. That currently means we need to apply the fix in:
- master branch of python37 - f32 branch of python37 - f31 branch of python3 (!) - f30 branch of python3
While cherry-picking or merging patches between branches of one git repo is quite straight-forward (the only major obstacle is the release and %chaneglog), doing this across multiple git repos (and spec filenames) is very tedious and error-prone (albeit possible).
With the 3.9 update approaching in Fedora 33, we would end up with the following scheme when updating Python 3.8:
- master branch of python38 - f32 branch of python3 (!) - f31 branch of python38 (!) - f30 branch of python38
Apart from technological obstacles, this causes nontrivial cognitive overhead. So far, I've been carrying that overhead myself, but others (who don't work on the Fedora Python packages daily) always need my assistance with this, because it's overly complicated and hard to understand without drawing pictures.
For this to be easier for us, while keeping the change minimal for others, we have decided to retire the python3 component and only continue maintaining the interpreters in "python3X" components. The names of "binary" packages remain unchanged.
Before:
SRPM: python3 builds: - python3 - python-unversioned-command - python3-{libs,devel,tkinter,idle,test,debug} - python3-debug{info,source}
After:
SRPM: python39 builds: - python3 (unchanged) - python-unversioned-command (unchanged) - python3-{libs,devel,tkinter,idle,test,debug} (unchanged) - python39-debug{info,source} (usually not installed by name)
To avoid further confusion, the "python3-libs" etc. packages will provide "python39-libs" as well.
We plan to do this together with the Python 3.9 upgrade. I'll update the change page after some feedback happens here.
The initial implementation is in: https://src.fedoraproject.org/rpms/python39/pull-request/32
Here are other approaches we have considered:
Why not rename the packages to python39-libs etc. (and only provide python3-libs)?
This would be inconsistent with the way other Python 3 packages are called (e.g. python39-setuptools). We consider that confusing.
The goal here is to make our work easier while introducing minimal changes to our users and contributors.
Why not rename all the Python packages to python39-setuptools etc.?
This would create a mess upon distro boundary Python upgrades. All python39-foo packages would need to obsolete all python38-foo packages. Even if we add the obsoletes metadata to all the packages, it would prevent users from having the two parallel stacks of Python installed on the same OS. (We don't have such stacks in Fedora, but we intend to continue to make this possible in third party repos, downstreams of Fedora or other systems built on top or Fedora).
See also the "minimal changes" point above.
Why not package the alternate stacks as modular streams and keep everything called just python3- (or even python-)?
1) The Python stacks are naturally parallel-installable. This would not be possible if they were different streams of one module. 2) Vital system tools (dnf, anaconda, packaging tools) use Python, so the "main" Python can't be in a module. (We could solve this by a separate "platform Python" just for these system tools, but that makes everything more complex and harder to work with -- the opposite of what this change is about.)
See also the "minimal changes" point above.
Why are you breaking the world once again?
Sorry if this breaks anything for you. Could you be as specific as possible? We anticipate no changes in Fedora packages (other than python3/python39).
On Tue, Mar 24, 2020 at 01:40:24PM +0100, Miro Hrončok wrote:
Hello Pythonistas.
(I've CC'ed the devel list for further exposure. But let's discuss this on python-devel list please to avoid noise.)
We would like ro rename the "python3" component (SRPM) to "python39" to make maintaining various Python versions in various Fedora versions easier. The names of binary RPMs would be unchanged; you still do `dnf install python3`.
+1, sounds very reasonable.
Zbyszek
On Tuesday, 24 March 2020 12.40.24 WET Miro Hrončok wrote:
We would like ro rename the "python3" component (SRPM) to "python39" to make maintaining various Python versions in various Fedora versions easier. The names of binary RPMs would be unchanged; you still do `dnf install python3`.
The change should only affect packagers of Python itself; not users or packagers of software written in Python. The only user-visible or general-packager-visible change is a different component (SRPM) name.
That seems like a good idea. What about bugzilla? Will it follow the renaming? That means that we need to know the python version (not difficult I admit) to know the component where to submit the bug report.
What about the upgrade path? Will it be preserved. Case: assume that I install python39 for F33. Then in F35 will it upgrade all the python39* to python310*? Or will the python39 remain installed?
There is some work to be done here although IMHO I agree that it is worth. :-)
Regards,
On Tue, Mar 24, 2020 at 04:12:29PM +0000, José Abílio Matos wrote:
On Tuesday, 24 March 2020 12.40.24 WET Miro Hrončok wrote:
We would like ro rename the "python3" component (SRPM) to "python39" to make maintaining various Python versions in various Fedora versions easier. The names of binary RPMs would be unchanged; you still do `dnf install python3`.
The change should only affect packagers of Python itself; not users or packagers of software written in Python. The only user-visible or general-packager-visible change is a different component (SRPM) name.
That seems like a good idea. What about bugzilla? Will it follow the renaming? That means that we need to know the python version (not difficult I admit) to know the component where to submit the bug report.
That's a good point.
Maybe it'd be useful to set up a periodic job to move any new bugs from python to pythonNN as appropriate for any given release.
What about the upgrade path? Will it be preserved. Case: assume that I install python39 for F33. Then in F35 will it upgrade all the python39* to python310*? Or will the python39 remain installed?
The binary path only cares about binary package names, so it is unaffected by this change.
There is some work to be done here although IMHO I agree that it is worth. :-)
Zbyszek
On Tuesday, 24 March 2020 16.38.38 WET Zbigniew Jędrzejewski-Szmek wrote:
The binary path only cares about binary package names, so it is unaffected by this change.
My concern here is not about the path of binaries.
The issue is about the upgrade path. But after looking into the pull request I see that the upgrade path is preserved. So it is not an issue. :-)
On 24. 03. 20 17:12, José Abílio Matos wrote:
What about bugzilla? Will it follow the renaming? That means that we need to know the python version (not difficult I admit) to know the component where to submit the bug report.
We will still own the python3 component and triage the bugs as usual. If it proves to be problematic, we can automate this, but IMHO it's not worth it currently.
On 24. 03. 20 13:40, Miro Hrončok wrote:
Hello Pythonistas.
(I've CC'ed the devel list for further exposure. But let's discuss this on python-devel list please to avoid noise.)
We would like ro rename the "python3" component (SRPM) to "python39" to make maintaining various Python versions in various Fedora versions easier. The names of binary RPMs would be unchanged; you still do `dnf install python3`.
The change should only affect packagers of Python itself; not users or packagers of software written in Python. The only user-visible or general-packager-visible change is a different component (SRPM) name.
We'll do this. The feedback so far was only positive.
On Tue, Mar 24, 2020 at 8:40 AM Miro Hrončok mhroncok@redhat.com wrote:
Hello Pythonistas.
(I've CC'ed the devel list for further exposure. But let's discuss this on python-devel list please to avoid noise.)
We would like ro rename the "python3" component (SRPM) to "python39" to make maintaining various Python versions in various Fedora versions easier. The names of binary RPMs would be unchanged; you still do `dnf install python3`.
The change should only affect packagers of Python itself; not users or packagers of software written in Python. The only user-visible or general-packager-visible change is a different component (SRPM) name.
Details follow.
Currently, we build the Python interpreters this way:
- the "main" Python 3 interpreter component is called python3
- it builds a couple of subpackages, like python3-libs or python3-devel
- the "alternate" Python 3 versions components are called python3X (python37,
python39...)
For example, Fedora 31 has:
- python36
- python3 (the "main" Python, which is 3.7.x)
- python38
- python39
(python37 is retired in f31, but present in Fedora versions where python3 != 3.7)
This makes updating the "main" Python fairly complicated. To upgrade from 3.7 to 3.8 in f32, we had to:
- retire python38
- upgrade python3 from 3.7 to 3.8 (this makes python3 obsolete python38)
- re-introduce (unretire) python37
Such an update comes every few years:
- https://fedoraproject.org/wiki/Changes/Python3.7 (f29)
- https://fedoraproject.org/wiki/Changes/Python3.8 (f32)
- https://fedoraproject.org/wiki/Changes/Python3.9 (f33)
What's even more complicated is backporting patches across Fedora releases. Assume there is an important upgrade/fix for Python 3.7 we wish to apply for all Fedoras including stable. That currently means we need to apply the fix in:
- master branch of python37
- f32 branch of python37
- f31 branch of python3 (!)
- f30 branch of python3
While cherry-picking or merging patches between branches of one git repo is quite straight-forward (the only major obstacle is the release and %chaneglog), doing this across multiple git repos (and spec filenames) is very tedious and error-prone (albeit possible).
With the 3.9 update approaching in Fedora 33, we would end up with the following scheme when updating Python 3.8:
- master branch of python38
- f32 branch of python3 (!)
- f31 branch of python38 (!)
- f30 branch of python38
Apart from technological obstacles, this causes nontrivial cognitive overhead. So far, I've been carrying that overhead myself, but others (who don't work on the Fedora Python packages daily) always need my assistance with this, because it's overly complicated and hard to understand without drawing pictures.
For this to be easier for us, while keeping the change minimal for others, we have decided to retire the python3 component and only continue maintaining the interpreters in "python3X" components. The names of "binary" packages remain unchanged.
Before:
SRPM: python3 builds:
- python3
- python-unversioned-command
- python3-{libs,devel,tkinter,idle,test,debug}
- python3-debug{info,source}
After:
SRPM: python39 builds:
- python3 (unchanged)
- python-unversioned-command (unchanged)
- python3-{libs,devel,tkinter,idle,test,debug} (unchanged)
- python39-debug{info,source} (usually not installed by name)
To avoid further confusion, the "python3-libs" etc. packages will provide "python39-libs" as well.
We plan to do this together with the Python 3.9 upgrade. I'll update the change page after some feedback happens here.
The initial implementation is in: https://src.fedoraproject.org/rpms/python39/pull-request/32
Here are other approaches we have considered:
Why not rename the packages to python39-libs etc. (and only provide python3-libs)?
This would be inconsistent with the way other Python 3 packages are called (e.g. python39-setuptools). We consider that confusing.
The goal here is to make our work easier while introducing minimal changes to our users and contributors.
Why not rename all the Python packages to python39-setuptools etc.?
This would create a mess upon distro boundary Python upgrades. All python39-foo packages would need to obsolete all python38-foo packages. Even if we add the obsoletes metadata to all the packages, it would prevent users from having the two parallel stacks of Python installed on the same OS. (We don't have such stacks in Fedora, but we intend to continue to make this possible in third party repos, downstreams of Fedora or other systems built on top or Fedora).
See also the "minimal changes" point above.
Why not package the alternate stacks as modular streams and keep everything called just python3- (or even python-)?
- The Python stacks are naturally parallel-installable. This would not be
possible if they were different streams of one module. 2) Vital system tools (dnf, anaconda, packaging tools) use Python, so the "main" Python can't be in a module. (We could solve this by a separate "platform Python" just for these system tools, but that makes everything more complex and harder to work with -- the opposite of what this change is about.)
See also the "minimal changes" point above.
Why are you breaking the world once again?
Sorry if this breaks anything for you. Could you be as specific as possible? We anticipate no changes in Fedora packages (other than python3/python39).
As long as there's a flag for doing fully versioned not-flat packages, I think I'd be good with this. That is: python3Y-* packages.
On 06. 04. 20 13:56, Neal Gompa wrote:
As long as there's a flag for doing fully versioned not-flat packages, I think I'd be good with this. That is: python3Y-* packages.
--without flatpackage
python-devel@lists.fedoraproject.org
-
José Abílio Matos -
Miro Hrončok -
Neal Gompa -
Zbigniew Jędrzejewski-Szmek