With reference to the guidelines for Python packaging found here:
https://fedoraproject.org/wiki/Packaging:Python
Specifically the material concerning executables in /usr/bin.
In the "Example common spec file" section is this comment.
# Must do the python2 install first because the scripts in /usr/bin are # overwritten with every setup.py install, and in general we want the # python3 version to be the default.
But this does not work if you only install the python2-XXX package instead of both the python2-XXX and python3-XXX packages. Here is why.
distutils will adjust the shbang interpreter line in installed scripts to be interpreter specific, e.g. you get one of:
#!/usr/bin/python2
or
#!/usr/bin/python3
But the guidelines require the py3 version of the script in the py2 package. Thus if you install just the python2-XXX package you'll end up with a script that runs the /usr/bin/python3 interpreter whose sys.path only includes library locations for py3! Thus the script cannot execute because it's missing its libraries (because the libraries were installed in the py2 locations).
How is this supposed to be addressed?
One possible workaround is to add a requirement on the py3 package in the py2 package. But that is silly because it completely negates everything in the py2 package (everything will run as if it were py3 after installing the py2 package and the py2 files will never be referenced).
Perhaps the issue is better rephrased as "How does one package Python when there are no version specific differences/requirements?" If this is covered in the guidelines it's not obvious and needs clarification. I assume this means using the generic %{_python} macros instead of dual use of the version specific macros so that everything corresponds to the system default version of Python. Correct?
Hi,
those are very good questions to which you should be able to find answers on the Python RPM Porting Guide [0]. You are right that this should be better covered in the packaging guidelines, sadly the process of changing them is rather problematic so no one yet had the time to update them.
[0] http://python-rpm-porting.readthedocs.io/en/latest/
Kind regards, Tomas
On 05/26/2016 12:18 AM, John Dennis wrote:
With reference to the guidelines for Python packaging found here:
https://fedoraproject.org/wiki/Packaging:Python
Specifically the material concerning executables in /usr/bin.
In the "Example common spec file" section is this comment.
# Must do the python2 install first because the scripts in /usr/bin are # overwritten with every setup.py install, and in general we want the # python3 version to be the default.
But this does not work if you only install the python2-XXX package instead of both the python2-XXX and python3-XXX packages. Here is why.
distutils will adjust the shbang interpreter line in installed scripts to be interpreter specific, e.g. you get one of:
#!/usr/bin/python2
or
#!/usr/bin/python3
But the guidelines require the py3 version of the script in the py2 package. Thus if you install just the python2-XXX package you'll end up with a script that runs the /usr/bin/python3 interpreter whose sys.path only includes library locations for py3! Thus the script cannot execute because it's missing its libraries (because the libraries were installed in the py2 locations).
How is this supposed to be addressed?
One possible workaround is to add a requirement on the py3 package in the py2 package. But that is silly because it completely negates everything in the py2 package (everything will run as if it were py3 after installing the py2 package and the py2 files will never be referenced).
Perhaps the issue is better rephrased as "How does one package Python when there are no version specific differences/requirements?" If this is covered in the guidelines it's not obvious and needs clarification. I assume this means using the generic %{_python} macros instead of dual use of the version specific macros so that everything corresponds to the system default version of Python. Correct?
On 05/26/2016 08:24 AM, Tomas Orsava wrote:
Hi,
those are very good questions to which you should be able to find answers on the Python RPM Porting Guide [0]. You are right that this should be better covered in the packaging guidelines, sadly the process of changing them is rather problematic so no one yet had the time to update them.
Thanks Tomas, I was aware of this document as well. However I believe both documents contain the same mistake.
Here is the problem:
Only the python3-XXX package installs the (Py3) script. Unless the python2-XXX package requires the script neither the script nor the Py3 modules/packages required by the (Py3) script will be installed when you install either the python-XXX or python2-XXX package (assuming the virtual provides of python-XXX points to python2-XXX as is currently the case).
I think the python2-XXX package in the examples is missing something like this:
Requires: %{_bindir}/sample-exec
Make sense?
Hi!
On 05/26/2016 07:38 PM, John Dennis wrote:
On 05/26/2016 08:24 AM, Tomas Orsava wrote:
Hi,
those are very good questions to which you should be able to find answers on the Python RPM Porting Guide [0]. You are right that this should be better covered in the packaging guidelines, sadly the process of changing them is rather problematic so no one yet had the time to update them.
Thanks Tomas, I was aware of this document as well. However I believe both documents contain the same mistake.
Here is the problem:
Only the python3-XXX package installs the (Py3) script. Unless the python2-XXX package requires the script neither the script nor the Py3 modules/packages required by the (Py3) script will be installed when you install either the python-XXX or python2-XXX package (assuming the virtual provides of python-XXX points to python2-XXX as is currently the case).
I think the python2-XXX package in the examples is missing something like this:
Requires: %{_bindir}/sample-exec
Make sense?
I believe there is a misunderstanding. In your first message you said "But the guidelines require the py3 version of the script in the py2 package." That is incorrect.
The guidelines only require you to package the Python 3 version of the script. They are not very clear on where you should put it, but the only logical place is of course the Python 3 subpackage (or a new subpackage with only the executable script that relies on the Python 3 subpackage).
You should never put the Python 3 executable in the Python 2 subpackage.
Kind regards, Tomas
On 05/27/2016 10:10 AM, Tomas Orsava wrote:
I think the python2-XXX package in the examples is missing something like this:
Requires: %{_bindir}/sample-exec
Make sense?
I believe there is a misunderstanding. In your first message you said "But the guidelines require the py3 version of the script in the py2 package." That is incorrect.
The guidelines only require you to package the Python 3 version of the script. They are not very clear on where you should put it, but the only logical place is of course the Python 3 subpackage (or a new subpackage with only the executable script that relies on the Python 3 subpackage).
You should never put the Python 3 executable in the Python 2 subpackage.
That's fine. But unless one of two things are done the guidelines as they currently stand will leave you with a broken python2-XXX package. You either have to copy the Py2 version of the script to %{_bindir}/sample-exec-2.7 after the %py2_install runs instead of deleting the scripts as is currently recommended. -OR- you need to add a Requires on the %{_bindir}/sample-exec to the python2-XXX package as I suggested above.
If one of these 2 things are not done then someone installing just the python2-XXX version of the package (what I expect might be a common user action) will not have a script that can execute.
Also neither set of guidelines include examples of setting up links if you elect to use versioned scripts (e.g. %{_bindir}/sample-exec-2.7, %{_bindir}/sample-exec-3.5). The guidelines seem to suggest it's best to avoid versioned scripts which I would agree with, if that is the case then of the two options discussed above only the Requires fix will work.
Hi!
On 05/27/2016 07:44 PM, John Dennis wrote:
On 05/27/2016 10:10 AM, Tomas Orsava wrote:
I think the python2-XXX package in the examples is missing something like this:
Requires: %{_bindir}/sample-exec
Make sense?
I believe there is a misunderstanding. In your first message you said "But the guidelines require the py3 version of the script in the py2 package." That is incorrect.
The guidelines only require you to package the Python 3 version of the script. They are not very clear on where you should put it, but the only logical place is of course the Python 3 subpackage (or a new subpackage with only the executable script that relies on the Python 3 subpackage).
You should never put the Python 3 executable in the Python 2 subpackage.
That's fine. But unless one of two things are done the guidelines as they currently stand will leave you with a broken python2-XXX package. You either have to copy the Py2 version of the script to %{_bindir}/sample-exec-2.7 after the %py2_install runs instead of deleting the scripts as is currently recommended. -OR- you need to add a Requires on the %{_bindir}/sample-exec to the python2-XXX package as I suggested above.
If one of these 2 things are not done then someone installing just the python2-XXX version of the package (what I expect might be a common user action) will not have a script that can execute.
That is as it should be. The application should be provided in only one subpackage, not both, unless it behaves differently on Python 2 and Python 3 (example would be e.g. pip that does behave differently). If the user wants the application, they will have to install the Python 3 subpackage.
Also neither set of guidelines include examples of setting up links if you elect to use versioned scripts (e.g. %{_bindir}/sample-exec-2.7, %{_bindir}/sample-exec-3.5). The guidelines seem to suggest it's best to avoid versioned scripts which I would agree with, if that is the case then of the two options discussed above only the Requires fix will work.
The guidelines are indeed flawed. But the Python RPM Packaging Guide (not part of the guidelines) does include examples of this [0]. However, it's only explained in the 4th chapter—chapter for packages where the executable behaves differently depending on the Python version (2 or 3). In all other cases you should not use versioned executables as the executable shall be included in only one version.
[0] http://python-rpm-porting.readthedocs.io/en/latest/tools.html
python-devel@lists.fedoraproject.org