Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
Please note, that I don't want to start a flamewar with this, but a serious discussion. I want to know what python SIG thinks about taking this step before taking it to wider public on fedora-devel.
On 08/03/2012 04:57 PM, Bohuslav Kabrda wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :(
Just to clarify Ubuntu's Python 3 plans: they're not quite as crazy as Arch.
On 12.10, /usr/bin/python will still point to python2.7. The big change they're aiming for is that their *installation CDs* will not include Python 2, only Python 3, which means any Python utilities they include with the base OS must be ported to Python 3.
We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
I believe Fedora uses Python for more core OS infrastructure than Ubuntu does, so it's a bigger migration challenge. Does anaconda run on Python 3? Does yum?
It's that delta of Python applications that Fedora ships as required components in the base OS, but Ubuntu does not, that will potentially cause problems.
Cheers, Nick.
----- Original Message -----
On 08/03/2012 04:57 PM, Bohuslav Kabrda wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :(
Just to clarify Ubuntu's Python 3 plans: they're not quite as crazy as Arch.
On 12.10, /usr/bin/python will still point to python2.7. The big change they're aiming for is that their *installation CDs* will not include Python 2, only Python 3, which means any Python utilities they include with the base OS must be ported to Python 3.
We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
I believe Fedora uses Python for more core OS infrastructure than Ubuntu does, so it's a bigger migration challenge. Does anaconda run on Python 3? Does yum?
Well, I believe they don't. But if noone pushes them, they never will. I think that a decision needs to be made when the transition will happen, so that these projects have a time frame, in which they need to adapt to Python 3.
It's that delta of Python applications that Fedora ships as required components in the base OS, but Ubuntu does not, that will potentially cause problems.
Sure, it will cause problems. But it will cause them eventually. Fedora 19 may be too soon (personally I think it's not), but we should really start pushing the various projects to actually start doing something about Python 3.
Cheers, Nick.
-- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
On Fri, Aug 03, 2012 at 03:55:26AM -0400, Bohuslav Kabrda wrote:
----- Original Message ----- (From Nick)
I believe Fedora uses Python for more core OS infrastructure than Ubuntu does, so it's a bigger migration challenge. Does anaconda run on Python 3? Does yum?
Well, I believe they don't. But if noone pushes them, they never will. I think that a decision needs to be made when the transition will happen, so that these projects have a time frame, in which they need to adapt to Python 3.
I suggest that you start by asking nicely what things need to be done to port those things to python2 and then go to work on doing them. Trying to force a change via the Feature process is reversing the order of things. Features don't exist to force other people to do work. Features exist to showcase and coordinate the work that you are doing.
For the package management stack, I know that one big blocker is that pycurl doesn't have a python3 port. There's a patch out there to add it (complete with reference counting leaks and other bugs :-( but pycurl upstream seems dead. So one of the very first steps would be for someone to take over pycurl upstream maintainance and add python3 support. (And then start fixing reference leaks and other bugs in both the python2 and python3 bindings).
It's that delta of Python applications that Fedora ships as required components in the base OS, but Ubuntu does not, that will potentially cause problems.
Sure, it will cause problems. But it will cause them eventually. Fedora 19 may be too soon (personally I think it's not), but we should really start pushing the various projects to actually start doing something about Python 3.
The best way to push is to work with projects to port their code. I think talking to the projects that require python2 that are in the LiveCDs and the install DVDs and porting the libraries that they require to be buildable as both python2 and python3 modules would be a sensible first step. Then work on porting the applications themselves. Along the way, you can figure out some important milestones to call out in Features. "The 100 python3 library port challenge -- Fedora Contributors helped port 100 python libraries to python3", "50% python3 in distro -- half of all python modules that are packaged are now python3 capable", "python3 package management stack -- our package manager is now running on python3 instead of python2"
One thing to watch out for is that once you start porting applications, the live images and install dvds will start requiring python3. If you aren't careful, some releases may end up requiring both python2 and python3. This is something that would need to be coordinated with the spins as well as the applications you're porting to figure out what is okay as a size constraint. You may end up having to maintain a parallel python3 branch for many applications until all of the applications on an image are ported to python3.
-Toshio
----- Original Message -----
On Fri, Aug 03, 2012 at 03:55:26AM -0400, Bohuslav Kabrda wrote:
----- Original Message ----- (From Nick)
I believe Fedora uses Python for more core OS infrastructure than Ubuntu does, so it's a bigger migration challenge. Does anaconda run on Python 3? Does yum?
Well, I believe they don't. But if noone pushes them, they never will. I think that a decision needs to be made when the transition will happen, so that these projects have a time frame, in which they need to adapt to Python 3.
I suggest that you start by asking nicely what things need to be done to port those things to python2 and then go to work on doing them. Trying to force a change via the Feature process is reversing the order of things. Features don't exist to force other people to do work. Features exist to showcase and coordinate the work that you are doing.
Ok, I probably didn't make myself clear. What I meant was making Python 3 the default, not dropping Python 2 support. Therefore all the projects can still use python2 in shebangs/whatever. What I mean to achieve by this is saying "hey, we're switching to Python 3, it's default from now on and you should start porting your code and stop writing python3-incompatible code" - and I will be very glad to help them.
For the package management stack, I know that one big blocker is that pycurl doesn't have a python3 port. There's a patch out there to add it (complete with reference counting leaks and other bugs :-( but pycurl upstream seems dead. So one of the very first steps would be for someone to take over pycurl upstream maintainance and add python3 support. (And then start fixing reference leaks and other bugs in both the python2 and python3 bindings).
It's that delta of Python applications that Fedora ships as required components in the base OS, but Ubuntu does not, that will potentially cause problems.
Sure, it will cause problems. But it will cause them eventually. Fedora 19 may be too soon (personally I think it's not), but we should really start pushing the various projects to actually start doing something about Python 3.
The best way to push is to work with projects to port their code. I think talking to the projects that require python2 that are in the LiveCDs and the install DVDs and porting the libraries that they require to be buildable as both python2 and python3 modules would be a sensible first step. Then work on porting the applications themselves. Along the way, you can figure out some important milestones to call out in Features. "The 100 python3 library port challenge -- Fedora Contributors helped port 100 python libraries to python3", "50% python3 in distro -- half of all python modules that are packaged are now python3 capable", "python3 package management stack -- our package manager is now running on python3 instead of python2"
One thing to watch out for is that once you start porting applications, the live images and install dvds will start requiring python3. If you aren't careful, some releases may end up requiring both python2 and python3. This is something that would need to be coordinated with the spins as well as the applications you're porting to figure out what is okay as a size constraint. You may end up having to maintain a parallel python3 branch for many applications until all of the applications on an image are ported to python3.
-Toshio
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
On Sat, Aug 4, 2012 at 9:46 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
----- Original Message -----
On Fri, Aug 03, 2012 at 03:55:26AM -0400, Bohuslav Kabrda wrote:
----- Original Message ----- (From Nick)
I believe Fedora uses Python for more core OS infrastructure than Ubuntu does, so it's a bigger migration challenge. Does anaconda run on Python 3? Does yum?
Well, I believe they don't. But if noone pushes them, they never will. I think that a decision needs to be made when the transition will happen, so that these projects have a time frame, in which they need to adapt to Python 3.
I suggest that you start by asking nicely what things need to be done to port those things to python2 and then go to work on doing them. Trying to force a change via the Feature process is reversing the order of things. Features don't exist to force other people to do work. Features exist to showcase and coordinate the work that you are doing.
Ok, I probably didn't make myself clear. What I meant was making Python 3 the default, not dropping Python 2 support. Therefore all the projects can still use python2 in shebangs/whatever. What I mean to achieve by this is saying "hey, we're switching to Python 3, it's default from now on and you should start porting your code and stop writing python3-incompatible code"
- and I will be very glad to help them.
For the package management stack, I know that one big blocker is that pycurl doesn't have a python3 port. There's a patch out there to add it (complete with reference counting leaks and other bugs :-( but pycurl upstream seems dead. So one of the very first steps would be for someone to take over pycurl upstream maintainance and add python3 support. (And then start fixing reference leaks and other bugs in both the python2 and python3 bindings).
It's that delta of Python applications that Fedora ships as required components in the base OS, but Ubuntu does not, that will potentially cause problems.
Sure, it will cause problems. But it will cause them eventually. Fedora 19 may be too soon (personally I think it's not), but we should really start pushing the various projects to actually start doing something about Python 3.
The best way to push is to work with projects to port their code. I think talking to the projects that require python2 that are in the LiveCDs and the install DVDs and porting the libraries that they require to be buildable as both python2 and python3 modules would be a sensible first step. Then work on porting the applications themselves. Along the way, you can figure out some important milestones to call out in Features. "The 100 python3 library port challenge -- Fedora Contributors helped port 100 python libraries to python3", "50% python3 in distro -- half of all python modules that are packaged are now python3 capable", "python3 package management stack -- our package manager is now running on python3 instead of python2"
One thing to watch out for is that once you start porting applications, the live images and install dvds will start requiring python3. If you aren't careful, some releases may end up requiring both python2 and python3. This is something that would need to be coordinated with the spins as well as the applications you're porting to figure out what is okay as a size constraint. You may end up having to maintain a parallel python3 branch for many applications until all of the applications on an image are ported to python3.
-Toshio
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
-- Regards, Bohuslav "Slavek" Kabrda. _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
I think you miss the point, you just don't change the default /usr/bin/python to python 3 and forces 90% of the python apps in Fedora to change there shebangs to /ust/bin/python2 that is not the way to do it, when 90% of the python apps in Fedora is python3 then we can start to talk about. The transition to python3 starts upstream of the python apps and before the big majority running python3, changing default python is nonsense. IMO
Tim
On 08/04/2012 02:19 AM, Toshio Kuratomi wrote:
I suggest that you start by asking nicely what things need to be done to port those things to python2 and then go to work on doing them. Trying to force a change via the Feature process is reversing the order of things. Features don't exist to force other people to do work. Features exist to showcase and coordinate the work that you are doing.
Right. Since I know Barry Warsaw, I got to see a bit of the way the Ubuntu/Canonical folks went about this, and one of the big things they did was to work with upstream projects to add Python 3 support. (e.g. the dbus-python patches for Python 3 compatibility were mostly Barry's work: http://www.wefearchange.org/2011/12/lessons-in-porting-to-python-3.html)
One thing to watch out for is that once you start porting applications, the live images and install dvds will start requiring python3. If you aren't careful, some releases may end up requiring both python2 and python3. This is something that would need to be coordinated with the spins as well as the applications you're porting to figure out what is okay as a size constraint. You may end up having to maintain a parallel python3 branch for many applications until all of the applications on an image are ported to python3.
Wherever possible, it's *much* easier to go with a single codebase that runs on both Python 2 and Python 3. The common subset of the two variants ended up being much larger than originally expected. For pure Python code, Armin Ronacher's "python-modernize" can help flush out legacy constructs that will fail under Python 3.
Cheers, Nick.
On Fri, Aug 3, 2012 at 3:43 AM, Nick Coghlan ncoghlan@redhat.com wrote:
On 08/03/2012 04:57 PM, Bohuslav Kabrda wrote:
We always take pride in being close to upstream and having the bleeding
edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
I believe Fedora uses Python for more core OS infrastructure than Ubuntu does, so it's a bigger migration challenge. Does anaconda run on Python 3? Does yum?
I don't think there are RPM bindings for python3 yet, although it looks like it is getting closer: https://bugzilla.redhat.com/show_bug.cgi?id=531543
On Fri, 2012-08-03 at 04:53 -0400, Andrew Parker wrote: [...snip...]
I don't think there are RPM bindings for python3 yet, although it looks like it is getting closer: https://bugzilla.redhat.com/show_bug.cgi?id=531543
The bindings are done; they're just waiting on review of that change to the rpm specfile.
On Fri, Aug 3, 2012 at 8:57 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
In which a way do you want to switch to python3? For me, that means any mention of "python" is synonym to "python3", which leads to the question, what is inside a python-foo package? The module foo, built for python3?
It would be best to rename all python-foo packages, to python2-foo or python3-foo and don't allow subpackaging of modules, which provide multiple python versions. And when that's consistent, we can switch /usr/bin/python to python3.
So how about a "Renaming of all python packages" feature for F19, and when that's done, switch to python3. When we are fast enough, maybe also for F19, but F20 or F21 would be more realistic....
Greetings, Tom
----- Original Message -----
On Fri, Aug 3, 2012 at 8:57 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
In which a way do you want to switch to python3? For me, that means any mention of "python" is synonym to "python3", which leads to the question, what is inside a python-foo package? The module foo, built for python3?
As you say, switching means /usr/bin/python -> /usr/bin/python3. And yes, that is a very valid point about the naming, I think.
It would be best to rename all python-foo packages, to python2-foo or python3-foo and don't allow subpackaging of modules, which provide multiple python versions. And when that's consistent, we can switch /usr/bin/python to python3.
I agree about the renaming. Having said that, I don't like that the users will need to install python2-foo or python3-foo, not just python-foo. On the other hand, the normal users will just get stuff and update it. The developers using various versions of python might actually appreciate the knowledge of what they are installing.
So how about a "Renaming of all python packages" feature for F19, and when that's done, switch to python3. When we are fast enough, maybe also for F19, but F20 or F21 would be more realistic....
Sounds good to me.
Greetings, Tom _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
On Fri, Aug 03, 2012 at 04:42:02AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On Fri, Aug 3, 2012 at 8:57 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
In which a way do you want to switch to python3? For me, that means any mention of "python" is synonym to "python3", which leads to the question, what is inside a python-foo package? The module foo, built for python3?
As you say, switching means /usr/bin/python -> /usr/bin/python3. And yes, that is a very valid point about the naming, I think.
Porting, taking those ports upstream, and even *tiny shudder* carrying those python3 patches locally for a loooong time, I could support.
Switching /usr/bin/python to point to /usr/bin/python3 I'd be very much against (at least for several years): http://www.python.org/dev/peps/pep-0394/
has a section that says it may someday be updated to recommend changing a /usr/bin/python symlink to point at python3. I would wait to make the link change until that PEP is updated (or many other unix distributions are also ready to make the switch). Switching the link is a largely symbolic gesture that creates more work for package maintainers, more work for end users, and more work for developers (who all have to find uses of /usr/bin/python and change them to /usr/bin/python2).
Switching /usr/bin/python to /usr/bin/python3 sacrifices practicality to purity which is unpythonic :-)
It would be best to rename all python-foo packages, to python2-foo or python3-foo and don't allow subpackaging of modules, which provide multiple python versions. And when that's consistent, we can switch /usr/bin/python to python3.
I agree about the renaming. Having said that, I don't like that the users will need to install python2-foo or python3-foo, not just python-foo. On the other hand, the normal users will just get stuff and update it. The developers using various versions of python might actually appreciate the knowledge of what they are installing.
One of the things that tomspur and I talkd about relating to this... There's a large potntial for confusion (in, for instance, bugzilla) if we simply rename python-foo to python2-foo if the python-foo package was building a python3-foo subpackage. If you find a bug in python3-foo, you'd need to file it against python2-foo in bugzilla... very non-intuitive.
A way to deal with this would be to stop shipping python3-foo as subpackages. Modules for python3-foo would have to be from separate srpms.
Another way would be to have empty python-foo main packages that generates python2-foo and python3-foo subpackages. That's kinda ugly though.
Regarding users needing to install python2-foo vs python3-foo: I'd propose virtual provides *for backwards compatibility*. For several years, yum install python-foo would match a Provides: python-foo = %{version}-%{release}
in the python2-foo package. At some point we'd decide these were no longer worthwhile and get rid of them. Note that this would only be for backwards compat. We'd never change them to install python3-foo. It's just to wean users off of python-foo meaning python2-foo.
-Toshio
----- Original Message -----
On Fri, Aug 03, 2012 at 04:42:02AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On Fri, Aug 3, 2012 at 8:57 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
In which a way do you want to switch to python3? For me, that means any mention of "python" is synonym to "python3", which leads to the question, what is inside a python-foo package? The module foo, built for python3?
As you say, switching means /usr/bin/python -> /usr/bin/python3. And yes, that is a very valid point about the naming, I think.
Porting, taking those ports upstream, and even *tiny shudder* carrying those python3 patches locally for a loooong time, I could support.
Switching /usr/bin/python to point to /usr/bin/python3 I'd be very much against (at least for several years): http://www.python.org/dev/peps/pep-0394/
has a section that says it may someday be updated to recommend changing a /usr/bin/python symlink to point at python3. I would wait to make the link change until that PEP is updated (or many other unix distributions are also ready to make the switch). Switching the link is a largely symbolic gesture that creates more work for package maintainers, more work for end users, and more work for developers (who all have to find uses of /usr/bin/python and change them to /usr/bin/python2).
Well, my opinion is different on this. The PEP says that bleeding edge distributions may have python3 as a default. And I like to think of Fedora as bleeding edge.
Switching /usr/bin/python to /usr/bin/python3 sacrifices practicality to purity which is unpythonic :-)
It would be best to rename all python-foo packages, to python2-foo or python3-foo and don't allow subpackaging of modules, which provide multiple python versions. And when that's consistent, we can switch /usr/bin/python to python3.
I agree about the renaming. Having said that, I don't like that the users will need to install python2-foo or python3-foo, not just python-foo. On the other hand, the normal users will just get stuff and update it. The developers using various versions of python might actually appreciate the knowledge of what they are installing.
One of the things that tomspur and I talkd about relating to this... There's a large potntial for confusion (in, for instance, bugzilla) if we simply rename python-foo to python2-foo if the python-foo package was building a python3-foo subpackage. If you find a bug in python3-foo, you'd need to file it against python2-foo in bugzilla... very non-intuitive.
A way to deal with this would be to stop shipping python3-foo as subpackages. Modules for python3-foo would have to be from separate srpms.
Another way would be to have empty python-foo main packages that generates python2-foo and python3-foo subpackages. That's kinda ugly though.
Regarding users needing to install python2-foo vs python3-foo: I'd propose virtual provides *for backwards compatibility*. For several years, yum install python-foo would match a Provides: python-foo = %{version}-%{release}
in the python2-foo package. At some point we'd decide these were no longer worthwhile and get rid of them. Note that this would only be for backwards compat. We'd never change them to install python3-foo. It's just to wean users off of python-foo meaning python2-foo.
Yep, shipping python2 and python3 packages separately and having the virtual provides both sound very good to me, I thought about something similar. (BTW I'm sorry to interrupt this discussion from my side, as I'm leaving for a vacation. I'll be glad to continue when I return :) Have a good time everyone!)
-Toshio
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
On Sat, Aug 04, 2012 at 03:50:56AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On Fri, Aug 03, 2012 at 04:42:02AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On Fri, Aug 3, 2012 at 8:57 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
In which a way do you want to switch to python3? For me, that means any mention of "python" is synonym to "python3", which leads to the question, what is inside a python-foo package? The module foo, built for python3?
As you say, switching means /usr/bin/python -> /usr/bin/python3. And yes, that is a very valid point about the naming, I think.
Porting, taking those ports upstream, and even *tiny shudder* carrying those python3 patches locally for a loooong time, I could support.
Switching /usr/bin/python to point to /usr/bin/python3 I'd be very much against (at least for several years): http://www.python.org/dev/peps/pep-0394/
has a section that says it may someday be updated to recommend changing a /usr/bin/python symlink to point at python3. I would wait to make the link change until that PEP is updated (or many other unix distributions are also ready to make the switch). Switching the link is a largely symbolic gesture that creates more work for package maintainers, more work for end users, and more work for developers (who all have to find uses of /usr/bin/python and change them to /usr/bin/python2).
Well, my opinion is different on this. The PEP says that bleeding edge distributions may have python3 as a default. And I like to think of Fedora as bleeding edge.
The only distribution that has switched is arch. When they did there was a big uproar about how arch was doing something wrong which eventually resulted in that PEP.
/usr/bin/python is a program that end users depend on being a certain thing -- it shouldn't be used to set policy. That would be like us deciding we wanted to switch the default shell from /bin/bash to /bin/zsh and to implement that policy we were going to symlink /bin/bash to /bin/zsh. Users wanting /bin/bash would need to update their scripts, tools, and practices to reference /bin/gnu-bash.
-Toshio
On 08/06/2012 04:22 AM, Toshio Kuratomi wrote:
The only distribution that has switched is arch. When they did there was a big uproar about how arch was doing something wrong which eventually resulted in that PEP.
Yeah, we mainly wrote PEP 394 in order to nudge *everyone else* into providing a /usr/bin/python2 symlink to help deal with Arch making their bold leap into the unknown (as well as going on record that we think switching it *right now* is still a bad idea). There's "bleeding edge" and then there's "tap dancing on razor blades in your bare feet" :P
To be honest, I expect that the long term outcome will be that "/usr/bin/python" becomes solely the preserve of the OS, with all cross-platform scripts and applications using "/usr/bin/pythonX", software collections, or language level virtual environments.
From an end user perspective, having things mostly compatible with both
2 and 3 should come *before* that symlink gets flipped rather than after.
Cheers, Nick.
----- Original Message -----
On 08/06/2012 04:22 AM, Toshio Kuratomi wrote:
The only distribution that has switched is arch. When they did there was a big uproar about how arch was doing something wrong which eventually resulted in that PEP.
Yeah, we mainly wrote PEP 394 in order to nudge *everyone else* into providing a /usr/bin/python2 symlink to help deal with Arch making their bold leap into the unknown (as well as going on record that we think switching it *right now* is still a bad idea). There's "bleeding edge" and then there's "tap dancing on razor blades in your bare feet" :P
To be honest, I expect that the long term outcome will be that "/usr/bin/python" becomes solely the preserve of the OS, with all cross-platform scripts and applications using "/usr/bin/pythonX", software collections, or language level virtual environments.
From an end user perspective, having things mostly compatible with both 2 and 3 should come *before* that symlink gets flipped rather than after.
Cheers, Nick.
Ok, then I would suggest using Tom Spura's idea about making only python2- and python3- packages (maybe with the virtual python- provides for python2- packages, as Toshio has mentioned). We could target this for F19 and we could also start helping various upstreams with switching to python 3 and see where this will take us. Does that sound good?
-- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
On 08/13/2012 05:09 PM, Bohuslav Kabrda wrote:
Ok, then I would suggest using Tom Spura's idea about making only python2- and python3- packages (maybe with the virtual python- provides for python2- packages, as Toshio has mentioned). We could target this for F19 and we could also start helping various upstreams with switching to python 3 and see where this will take us. Does that sound good?
That sounds pretty sensible to me. One sticking point may be upstreams that still want to support RHEL 5, since supporting both 2.4 and 3.x out of the same code base is fairly painful (whereas 2.4-2.7 is pretty easy if you don't mind missing out on the shiny new features in later versions and 2.6+ and 3.2+ is fairly straightforward with the aid of the six project and/or some compatibility modules).
However, that problem can be addressed on a case-by-case basis as it comes up.
Cheers, Nick.
On Mon, 2012-08-13 at 03:09 -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On 08/06/2012 04:22 AM, Toshio Kuratomi wrote:
The only distribution that has switched is arch. When they did there was a big uproar about how arch was doing something wrong which eventually resulted in that PEP.
Yeah, we mainly wrote PEP 394 in order to nudge *everyone else* into providing a /usr/bin/python2 symlink to help deal with Arch making their bold leap into the unknown (as well as going on record that we think switching it *right now* is still a bad idea). There's "bleeding edge" and then there's "tap dancing on razor blades in your bare feet" :P
To be honest, I expect that the long term outcome will be that "/usr/bin/python" becomes solely the preserve of the OS, with all cross-platform scripts and applications using "/usr/bin/pythonX", software collections, or language level virtual environments.
From an end user perspective, having things mostly compatible with both 2 and 3 should come *before* that symlink gets flipped rather than after.
Cheers, Nick.
Ok, then I would suggest using Tom Spura's idea about making only python2- and python3- packages (maybe with the virtual python- provides for python2- packages, as Toshio has mentioned). We could target this for F19 and we could also start helping various upstreams with switching to python 3 and see where this will take us. Does that sound good?
If we're looking at this kind of change (python2-foo and python3-foo), how about some other prefixes: * python2-debug-foo (or somesuch) for a build of the package against the --with-pydebug python2 package * python3-debug-foo * pypy-foo for the package built against PyPy
FWIW an old idea I had for revamping how we maintain Python packages can be seen here: http://lists.fedoraproject.org/pipermail/python-devel/2010-March/000213.html with some further ideas here: https://fedoraproject.org/wiki/DaveMalcolm/PythonIdeas (you can tell that page is old, it still mentions Unladen Swallow and Pynie...)
I still like the basic idea: introduce a tool that captures the info on python runtimes for a given release in one place, and use it programmatically within specfiles to avoid copy&paste. Although the proposal isn't complete (how to handle exclusions?), I'm still fond of it.
Another idea would be to use the Software Collections tool seen here: https://fedorahosted.org/SoftwareCollections/ I known that people within Red Hat have been looking at using that in the RHEL context, and some of that is likely to be appropriate for EPEL too - though AIUI it's a shift in model from one build per src.rpm to multiple builds per src.rpm
Hope this is helpful Dave
On Mon, 2012-08-13 at 20:53 -0400, David Malcolm wrote:
On Mon, 2012-08-13 at 03:09 -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On 08/06/2012 04:22 AM, Toshio Kuratomi wrote:
The only distribution that has switched is arch. When they did there was a big uproar about how arch was doing something wrong which eventually resulted in that PEP.
Yeah, we mainly wrote PEP 394 in order to nudge *everyone else* into providing a /usr/bin/python2 symlink to help deal with Arch making their bold leap into the unknown (as well as going on record that we think switching it *right now* is still a bad idea). There's "bleeding edge" and then there's "tap dancing on razor blades in your bare feet" :P
To be honest, I expect that the long term outcome will be that "/usr/bin/python" becomes solely the preserve of the OS, with all cross-platform scripts and applications using "/usr/bin/pythonX", software collections, or language level virtual environments.
From an end user perspective, having things mostly compatible with both 2 and 3 should come *before* that symlink gets flipped rather than after.
Cheers, Nick.
Ok, then I would suggest using Tom Spura's idea about making only python2- and python3- packages (maybe with the virtual python- provides for python2- packages, as Toshio has mentioned). We could target this for F19 and we could also start helping various upstreams with switching to python 3 and see where this will take us. Does that sound good?
If we're looking at this kind of change (python2-foo and python3-foo), how about some other prefixes:
- python2-debug-foo (or somesuch) for a build of the package against
the --with-pydebug python2 package
- python3-debug-foo
- pypy-foo for the package built against PyPy
FWIW an old idea I had for revamping how we maintain Python packages can be seen here: http://lists.fedoraproject.org/pipermail/python-devel/2010-March/000213.html
The "gitweb" URLs in that mail are stale as fedorapeople.org is now using cgit; see: http://fedorapeople.org/cgit/dmalcolm/public_git/rpm-pyconfig.git/tree/
----- Original Message -----
On Mon, 2012-08-13 at 03:09 -0400, Bohuslav Kabrda wrote:
----- Original Message -----
On 08/06/2012 04:22 AM, Toshio Kuratomi wrote:
The only distribution that has switched is arch. When they did there was a big uproar about how arch was doing something wrong which eventually resulted in that PEP.
Yeah, we mainly wrote PEP 394 in order to nudge *everyone else* into providing a /usr/bin/python2 symlink to help deal with Arch making their bold leap into the unknown (as well as going on record that we think switching it *right now* is still a bad idea). There's "bleeding edge" and then there's "tap dancing on razor blades in your bare feet" :P
To be honest, I expect that the long term outcome will be that "/usr/bin/python" becomes solely the preserve of the OS, with all cross-platform scripts and applications using "/usr/bin/pythonX", software collections, or language level virtual environments.
From an end user perspective, having things mostly compatible with both 2 and 3 should come *before* that symlink gets flipped rather than after.
Cheers, Nick.
Ok, then I would suggest using Tom Spura's idea about making only python2- and python3- packages (maybe with the virtual python- provides for python2- packages, as Toshio has mentioned). We could target this for F19 and we could also start helping various upstreams with switching to python 3 and see where this will take us. Does that sound good?
If we're looking at this kind of change (python2-foo and python3-foo), how about some other prefixes:
- python2-debug-foo (or somesuch) for a build of the package
against the --with-pydebug python2 package
- python3-debug-foo
- pypy-foo for the package built against PyPy
FWIW an old idea I had for revamping how we maintain Python packages can be seen here: http://lists.fedoraproject.org/pipermail/python-devel/2010-March/000213.html with some further ideas here: https://fedoraproject.org/wiki/DaveMalcolm/PythonIdeas (you can tell that page is old, it still mentions Unladen Swallow and Pynie...)
I still like the basic idea: introduce a tool that captures the info on python runtimes for a given release in one place, and use it programmatically within specfiles to avoid copy&paste. Although the proposal isn't complete (how to handle exclusions?), I'm still fond of it.
Personally, I like the idea. As you say, there are still some issues, but I guess we could solve those easily. There is IMHO one downside of this approach - the complexity. I fear that rpm-pyconfig can be easily taken to such extremes, that we will end up with completely unintelligible specfiles. As for supporting more runtimes (e.g. Pypy), I would be very careful about that, as it might get us overloaded by tons of issues of different packages with different runtimes. Here is a wild idea: Would it be possible to create a common site-packages directory for all runtimes? I don't know much about how different runtimes handle bytecode generated by another runtime etc, but it may be worth looking into, if we want to support more runtimes with our packages.
Another idea would be to use the Software Collections tool seen here: https://fedorahosted.org/SoftwareCollections/ I known that people within Red Hat have been looking at using that in the RHEL context, and some of that is likely to be appropriate for EPEL too - though AIUI it's a shift in model from one build per src.rpm to multiple builds per src.rpm
I've already done my bit of SCL packaging, so here are my 50 cents :) - How exactly would you apply SCLs in our case? I like the SCL concept (obviously :) ), but if you package e.g. Python 3 as an SCL, then you're going to have a hard time convincing someone to switch their application to it. Maybe packaging all the -debug builds for a single runtime in an SCL might be a good idea? E.g. have one srpm, that would produce optimized build if built out of SCL and debug build if built with SCL. - SCLs are currently not allowed in Fedora [1] (I'm not sure why, maybe Toshio can explain why FPC chose to forbid them). - If anyone is interested in an example how things could work out, drop me a line, I can create a simple example.
Hope this is helpful Dave
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
FWIW an old idea I had for revamping how we maintain Python packages can be seen here: http://lists.fedoraproject.org/pipermail/python-devel/2010-March/000213.html with some further ideas here: https://fedoraproject.org/wiki/DaveMalcolm/PythonIdeas (you can tell that page is old, it still mentions Unladen Swallow and Pynie...)
I still like the basic idea: introduce a tool that captures the info on python runtimes for a given release in one place, and use it programmatically within specfiles to avoid copy&paste. Although the proposal isn't complete (how to handle exclusions?), I'm still fond of it.
Personally, I like the idea. As you say, there are still some issues, but I guess we could solve those easily. There is IMHO one downside of this approach - the complexity. I fear that rpm-pyconfig can be easily taken to such extremes, that we will end up with completely unintelligible specfiles. As for supporting more runtimes (e.g. Pypy), I would be very careful about that, as it might get us overloaded by tons of issues of different packages with different runtimes. Here is a wild idea: Would it be possible to create a common site-packages directory for all runtimes? I don't know much about how different runtimes handle bytecode generated by another runtime etc, but it may be worth looking into, if we want to support more runtimes with our packages.
While this is pretty nice idea it's not feasible unless all runtimes support http://www.python.org/dev/peps/pep-3147/ and http://www.python.org/dev/peps/pep-3149/ which will require some backporting in 2.x series.
On 08/14/2012 05:12 PM, Konstantin Zemlyak wrote:
[Bohuslav Kabrda]
Here is a wild idea: Would it be possible to create a common site-packages directory for all runtimes? I don't know much about how different runtimes handle bytecode generated by another runtime etc, but it may be worth looking into, if we want to support more runtimes with our packages.
While this is pretty nice idea it's not feasible unless all runtimes support http://www.python.org/dev/peps/pep-3147/ and http://www.python.org/dev/peps/pep-3149/ which will require some backporting in 2.x series.
As Konstantin noted, this isn't really feasible except in Python 3.2+. Enabling that kind of thing is *why* those PEPs exist (although the binary one is also there due to the way Debian/Ubuntu handle multiarch support), but that doesn't help when we also want to deal with 2.7.
Backporting is a very dubious idea, as it changes the APIs visible at the Python level (you need at least the "source_to_cache" and "cache_to_source" APIs), and application code won't expect those to be present when the version is less than 3.2. Even worse, the behavioural change is known to break some programs that assume the source to cache mapping is just "source_fname + ('c' if __debug__ else 'o')". While that assumption is technically relying on an implementation detail, breaking it in an distro patch would not be a popular decision (to say the least).
However, I think it's enough to place a clear upper limit on the number of runtimes to be supported (where 'x' is the relevant minor version packaged in the Fedora repos): CPython 2.x, PyPy 1.x, Python 3.x (with shared site-packages)
Cheers, Nick.
On Fri, Aug 3, 2012 at 8:57 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
Hi, I'd like to start a discussion about the release where we should switch to Python 3. As I have learned recently, Ubuntu 12.10 will have Python 3 as default [1], which makes me a sad panda :( We always take pride in being close to upstream and having the bleeding edge. Python 3 is stable and more and more libraries support it. So I'd like to propose an idea to switch to Python 3 for Fedora 19.
Please note, that I don't want to start a flamewar with this, but a serious discussion. I want to know what python SIG thinks about taking this step before taking it to wider public on fedora-devel.
-- Regards, Bohuslav "Slavek" Kabrda.
[1] https://wiki.ubuntu.com/QuantalQuetzal/TechnicalOverview/Alpha3#Python_3.0 _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
What is the benefit of using python3 as primary version in Fedora, Other than generate at lot of extra work for developers. Python3 is just like another language, so you will put extra maintenance work on developers who make system tools there is used on long life releases like RHEL, it make it hard to backport bug fixes. Python 3 is great and if you make something new in python, it is a good choice if then need libraries exists to Python 3, but converting an application to be used on both python 2.x and 3.x is a lot of extra work without big benefits.
Tim
On 08/03/2012 11:15 PM, tim.lauridsen@gmail.com wrote:
What is the benefit of using python3 as primary version in Fedora, Other than generate at lot of extra work for developers. Python3 is just like another language, so you will put extra maintenance work on developers who make system tools there is used on long life releases like RHEL, it make it hard to backport bug fixes. Python 3 is great and if you make something new in python, it is a good choice if then need libraries exists to Python 3, but converting an application to be used on both python 2.x and 3.x is a lot of extra work without big benefits.
The single biggest practical benefit of Python 3.2 over Python 2.x is chained exceptions (i.e. if an error occurs in an exception handler, Python 3 will report *both* exceptions by default, where Python 2 will only report the broken error handler. Better hope you can fix the error handler *and* subsequently reproduce the original fault that triggered it!). I mostly work in 2.6 (for deployment on RHEL 6), and they're the one thing I genuinely miss from 3.2 that isn't also available in 2.7 or on PyPI (the changes needed to the exception infrastructure to support chained exceptions are technically backwards incompatible due to a change in the scope of the name bindings for caught exceptions, thus they were not backported before the 2.x series went into maintenance mode).
The 3.x series also tries to minimise unnecessary copying, with many interfaces returning iterators or views over the original data rather than copies.
The practical benefits of Python 3.3 are even more obvious, especially the new Unicode implementation (full Unicode compatibility without paying the price of making even pure ASCII strings 4 times the size), the substantially improved exception hierarchy for the OS APIs, the reduced memory consumption for object-oriented code, the faulthandler module, access to new POSIX features (http://docs.python.org/dev/whatsnew/3.3.html#os), caching of filesystem details in the import machinery to minimise stat call overhead (drastically increasing the speed of Python imports from network shares), etc.
Converting applications to work on both Python 2 and 3 often *isn't* a lot of work. In many cases, Armin Ronacher's python-modernize will (with the aid of the stdlib's "lib2to3" module and Benjamin Peterson's "six" compatibility module) automatically convert an existing code base into one that will run on 2.6, 2.7 and 3.2+ (or, 3.3+ if you decide to rely on 3.3's reintroduction of support for explicit unicode literals)
Not *all* applications are going to be easy to port (especially those with intricate dependencies on the Python C API or the broken Python 2 text model), but many should fit fairly easily into the common Python 2/3 language subset.
Cheers, Nick.
python-devel@lists.fedoraproject.org
-
Andrew Parker -
Bohuslav Kabrda -
David Malcolm -
Konstantin Zemlyak -
Nick Coghlan -
Thomas Spura -
Tim Lauridsen -
Toshio Kuratomi