python-devel February 2017

python-devel@lists.fedoraproject.org

11 participants
7 discussions

Alternative Python build tools issues
by Michal Cyprian 28 Feb '17

28 Feb '17

After application of patches described in Making sudo pip safe Fedora Change [0], unexpected issues in packages using non-standard build methods/scripts appeared. We received bug reports for DNF [1] and Anaconda [2]. We decided to temporarily revert the main patch last Friday (2017-02-24) to prevent build failures and anaconda crashes while we investigate what will need to change. To simplify, the main reason for the build failures is that some packages use /usr/bin/python3 (or $PATH/python3) instead of %{__python3} when building RPMs. It's not always trivial to fix this in build systems that hardcode assumptions about where the python3 binary is. I collected some data to have an idea how many packages are affected. The total number of Python 3 packages in Fedora according Fedora Porting Database [3] is 1837. The most frequently used alternative build tools for Python packages are CMake (28 packages) and Autotools (133 packages). 8 out of 28 packages using CMake fail to build in Copr [4] as well as 12 out of 133 packages using Autotools [5]. These build tools need to be configured to use %{__python3} during rpm build. There are also some custom build scripts not following best practices and hardcoding the path to the python executable or to python site-packages. Specific fixes will have to be done in these cases. We'll meet with Anaconda developers to discuss the problem. [0] https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe?rd=User:Torsava… [1] https://bugzilla.redhat.com/show_bug.cgi?id=1426435 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1425827 [3] http://fedora.portingdb.xyz/ [4] https://copr.fedorainfracloud.org/coprs/mcyprian/cmake-pkgs/builds/ [5] https://copr.fedorainfracloud.org/coprs/mcyprian/autotools-pkgs/builds/ Michal Cyprian

1 0

Python 3.6, Fedora, and the "C" locale
by Nick Coghlan 24 Feb '17

24 Feb '17

Hi folks, One of the minor irritations with running Python 3 applications in Fedora containers is that those containers still default to the "C" local by default - you have to force "C.UTF-8" or "en_US.UTF-8" in order to tell Python that it should use UTF-8 for communicating with system interfaces. However, since 3.5, the CPython start-up sequence has treated claims from the OS that it should use ASCII more skeptically, and enabled the "surrogateescape" error handling by default on the standard streams. Along similar lines, what do folks think of the idea of patching Python 3.6 in Fedora to assume UTF-8 if it's told that it should use ASCII to communicate with the OS? That changes the failure mode from "interface problems happen any time you fail to configure your locale correctly" to "interface problems happen any time you fail to configure your locale correctly *and* your default locale uses an encoding other than UTF-8". That latter case then further breaks down into the following two situations: - you use an ASCII-incompatible encoding like Shift-JIS or GB-18030, in which case the old POSIX default of ASCII was already broken, and the surrogateescape workaround in 3.5 probably didn't help much - you use an ASCII-compatible encoding like koi8-r, which means the surrogateescape workaround in 3.5 probably helped a bit, but libraries like `click` would still have complained and refused to run So I think doing this would be a nice usability improvement for users of the system Python 3 installation on Fedora. I also have an upstream motive for suggesting this, though: if we do this in the more constrained environment of "Fedora users" and it doesn't break the world, then that will help build a case for making it the default upstream behaviour in Python 3.7. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 19

Warning before sudo pip'ing?
by Tomas Orsava 22 Feb '17

22 Feb '17

Hi! On the last FESCo meeting while discussing the sudo pip Fedora [Change], maxamillion proposed that it might be useful to issue a warning when a user tries to run pip with root privileges--as in most cases it's not what they should be doing (`pip install --user` is usually more appropriate). What do you think? [Change] https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe Regards, Tomas Orsava

4 5

Mass-rebuild hiccup
by Tomas Orsava 15 Feb '17

15 Feb '17

Hi! It appears that there's been a bug in rpm during the time of Fedora 26 mass rebuild. The [bug] resulted in some packages failing to be rebuilt and some being built without .pyc files inside __pycache__ directories. [bug]https://bugzilla.redhat.com/show_bug.cgi?id=1411588 Together with ignatenkobrain and sgallagh who alerted us to the issue, we've managed to narrow down affected packages to Python 3–only packages that don't drag Python 2 into the buildroot and don't use standard Python build/install macros. In the end the following 22 packages were rebuilt: antimony (antimony-0.9.3-0.3.20161128git41a770.fc26.src.rpm) btest (btest-0.54-7.fc26.src.rpm) coccinelle (coccinelle-1.0.6-6.fc26.src.rpm) devscripts (devscripts-2.17.1-3.fc26.src.rpm) dnf-plugin-spacewalk (dnf-plugin-spacewalk-2.6.1-3.fc26.src.rpm) dnfdragora (dnfdragora-0.0.0-0.111.git20170207.783aede.fc26.src.rpm) gausssum (gausssum-3.0.1.1-4.fc26.src.rpm) liborcus-python3 (liborcus-0.12.1-3.fc26.src.rpm) loook (loook-0.8.1-7.fc26.src.rpm) mock (mock-1.3.3-2.fc26.src.rpm) mock-lvm (mock-1.3.3-2.fc26.src.rpm) mock-scm (mock-1.3.3-2.fc26.src.rpm) policycoreutils-devel (policycoreutils-2.5-21.fc26.src.rpm) python3-fuckit (python-fuckit-4.8.0-8.fc26.src.rpm) python3-louis (liblouis-2.6.2-8.fc26.src.rpm) python3-lxc (lxc-2.0.7-1.fc26.1.src.rpm) python3-meh-gui (python-meh-0.44-4.fc26.src.rpm) python3-pillow-qt (python-pillow-4.0.0-2.fc26.src.rpm) python3-pylint-gui (pylint-1.6.4-3.fc26.src.rpm) python3-qt5-devel (python-qt5-5.7.1-4.fc26.src.rpm) python3-uwsgidecorators (uwsgi-2.0.14-8.fc26.src.rpm) speedtest-cli (speedtest-cli-0.3.2-7.fc26.src.rpm) Regards, Tomas Orsava

1 0

Unable to install packages in a Python 3 virtual environment created with venv module with --system-site-packages option
by Tadej Janež 10 Feb '17

10 Feb '17

Hi! If I create a Python 2 virtual environment with virtualenv and the -- system-site-packages option, installation of any subsequent package works as expected: [vagrant@tadej-zbook ~]$ rpm -q python2-virtualenv python2-virtualenv-15.0.3-2.fc25.noarch [vagrant@tadej-zbook ~]$ virtualenv --system-site-packages myvenv2 New python executable in /home/vagrant/myvenv2/bin/python2 Also creating executable in /home/vagrant/myvenv2/bin/python Installing setuptools, pip, wheel...done. [vagrant@tadej-zbook ~]$ source myvenv2/bin/activate (myvenv2) [vagrant@tadej-zbook ~]$ pip install Django Collecting Django Downloading Django-1.10.5-py2.py3-none-any.whl (6.8MB) 100% |████████████████████████████████| 6.8MB 214kB/s Installing collected packages: Django Successfully installed Django-1.10.5 On the other hand, if I create a Python 3 virtual environment with venv and use the --system-site-packages option, installation of any subsequent package fails: [vagrant@tadej-zbook ~]$ rpm -q python3-libs python3-libs-3.5.2-4.fc25.x86_64 [vagrant@tadej-zbook ~]$ python3 -m venv --system-site-packages myvenv3 [vagrant@tadej-zbook ~]$ source myvenv3/bin/activate (myvenv3) [vagrant@tadej-zbook ~]$ pip3 install Django Collecting Django Using cached Django-1.10.5-py2.py3-none-any.whl Installing collected packages: Django Exception: Traceback (most recent call last): File "/usr/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/usr/lib/python3.5/site-packages/pip/commands/install.py", line 326, in run strip_file_prefix=options.strip_file_prefix, File "/usr/lib/python3.5/site-packages/pip/req/req_set.py", line 742, in install **kwargs File "/usr/lib/python3.5/site-packages/pip/req/req_install.py", line 834, in install strip_file_prefix=strip_file_prefix File "/usr/lib/python3.5/site-packages/pip/req/req_install.py", line 1037, in move_wheel_files strip_file_prefix=strip_file_prefix, File "/usr/lib/python3.5/site-packages/pip/wheel.py", line 346, in move_wheel_files clobber(source, lib_dir, True) File "/usr/lib/python3.5/site-packages/pip/wheel.py", line 317, in clobber ensure_dir(destdir) File "/usr/lib/python3.5/site-packages/pip/utils/__init__.py", line 83, in ensure_dir os.makedirs(path) File "/usr/lib64/python3.5/os.py", line 241, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.5/site-packages/Django-1.10.5.dist-info' Apparently pip inside the virtual environment tried to install Django to the system-wide location (/usr/lib/python3.5/site-packages/) and failed (as expected). But why did it try to install Django in the system-wide location instead of virtual environment's site-packages directory? Taking a closer look to the created Python 3 virtual environment reveals that it doesn't contain it's own pip3 command: (myvenv3) [vagrant@tadej-zbook ~]$ ls -al myvenv3/bin/ total 20 drwxrwxr-x. 2 vagrant vagrant 4096 Feb 2 14:49 . drwxrwxr-x. 5 vagrant vagrant 4096 Feb 2 14:49 .. -rw-r--r--. 1 vagrant vagrant 2143 Feb 2 14:49 activate -rw-r--r--. 1 vagrant vagrant 1259 Feb 2 14:49 activate.csh -rw-r--r--. 1 vagrant vagrant 2397 Feb 2 14:49 activate.fish lrwxrwxrwx. 1 vagrant vagrant 7 Feb 2 14:49 python -> python3 lrwxrwxrwx. 1 vagrant vagrant 16 Feb 2 14:49 python3 -> /usr/bin/python3 Which in turn means 'pip3 install Django' actually used the system-wide pip3 command which tried to install Django to the system-wide location and failed. Is there a reason this behavior has changed compared to Python 2 and virtualenv command or is it simply a bug that I should report? Thanks for help and regards, Tadej

5 10

Feature EasierPythonDebugging ?
by Sérgio Basto 03 Feb '17

03 Feb '17

Hello , I'd like use this gdb python [1] , but can't put it work on Fedora 24, last problems mentioned are about Fedora 13 , can someone help me on pointing how I could use New_gdb_commands [2] ? Thanks , [1] https://fedoraproject.org/wiki/Features/EasierPythonDebugging#Current_state… [2] https://fedoraproject.org/wiki/Features/EasierPythonDebugging#New_gdb_comma… https://wiki.python.org/moin/DebuggingWithGdb -- Sérgio M. B.

2 1

Feature EasierPythonDebugging ?
by Sérgio Basto 03 Feb '17

03 Feb '17

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel February 2017