Hello all, I created a patch to update python-bugzilla to work on both python 2 and python 3. The unit tests all pass under both py2.7 and py3.2. I have tested the login and query commands by hand on py2.6, py2.7, and py3.2.
http://threebean.org/patches/python-bugzilla-Python3-support.patch
The patch is against the master branch (7ca8a60). If I should recreate it against another branch, please let me know.
I'm also willing to update and maintain a spec file supporting both versions if and when that time comes.
Cheers- -Ralph
On Tue, Jun 26, 2012 at 12:32:38AM -0400, Ralph Bean wrote:
Hello all, I created a patch to update python-bugzilla to work on both python 2 and python 3. The unit tests all pass under both py2.7 and py3.2. I have tested the login and query commands by hand on py2.6, py2.7, and py3.2.
http://threebean.org/patches/python-bugzilla-Python3-support.patch
The patch is against the master branch (7ca8a60). If I should recreate it against another branch, please let me know.
I'm also willing to update and maintain a spec file supporting both versions if and when that time comes.
Dependency notes (and note, these are just notes -- I don't know if the project has defintions of what is acceptable)
* Needs python-2.6 or greater (as python-2.5 doesn't have from __future__ import print_function, the new format of "except Foo as bar:", and relative imports) * New dependency on python-six * There are places that str(foo) is used that haven't been converted to a six.XXX() method. These might fail in some circumstances.
* Note: the to_encoding() function looks to handle edge cases wrong. This would become more apparent in python3 than in python2. It looks like it's intended to take a unicode string or a byte string and return a byte string. However, if the parameter passed in is neither a unicode nor a byte string, it will return an empty unicode string. This will likely trigger tracebacks in python3 where bytes and unicode cannot be combined implictly. * Note2: Some of the lines that use to_encoding() look like they can still throw exceptions. they look like they can mix unicode and bytes in some circumstances. Running with python-unicodenazi installed and enabled (see that package's documentation) might find some of these cases.
* This might better be rewritten: to_remove = [k for k in query.keys() if query[k] is None] for key in to_remove: del query[key] as this: query = dict(((k, v) for k, v in six.iteritems(query) if v is not None))
which condenses two loops into one (Note that the generator expression there can be made into a list comprehension if compatibility with older python's is desired).
-Toshio
On Mon, Jun 25, 2012 at 10:39:48PM -0700, Toshio Kuratomi wrote:
- Needs python-2.6 or greater (as python-2.5 doesn't have from __future__ import print_function, the new format of "except Foo as bar:", and relative imports)
This is correct. Can anyone comment on what python versions python-bugzilla is required to support? EPEL6 is on python-2.6 now.
- New dependency on python-six
Correct.
- There are places that str(foo) is used that haven't been converted to a six.XXX() method. These might fail in some circumstances.
I took on the remainder of these in one of the new patches listed below.
- Note: the to_encoding() function looks to handle edge cases wrong. This would become more apparent in python3 than in python2. It looks like it's intended to take a unicode string or a byte string and return a byte string. However, if the parameter passed in is neither a unicode nor a byte string, it will return an empty unicode string. This will likely trigger tracebacks in python3 where bytes and unicode cannot be combined implictly.
Nice catch! I fixed to_encoding so it always returns byte strings now.
- Note2: Some of the lines that use to_encoding() look like they can still throw exceptions. they look like they can mix unicode and bytes in some circumstances. Running with python-unicodenazi installed and enabled (see that package's documentation) might find some of these cases.
I went through all the uses of it and couldn't find anywhere where byte or unicode strings are implicitly coerced. I checked the fields in query results and in every case I tested, they were unicode strings (thus making it all consistent). unicode-nazi only complained about some stuff in stdlib.
- This might better be rewritten: to_remove = [k for k in query.keys() if query[k] is None] for key in to_remove: del query[key] as this: query = dict(((k, v) for k, v in six.iteritems(query) if v is not None))
Done! (The *original* loop modified the dict during iteration; illegal in Python 3).
Here's my list of updated patches (including a rename of my original). I kept them separated out instead of rebasing just to preserve history and make them easier to cherry-pick.
http://threebean.org/patches/python-bugzilla/0001-Python3-support.patch http://threebean.org/patches/python-bugzilla/0002-Replace-str-foo-with-six.t... http://threebean.org/patches/python-bugzilla/0003-Rewrite-two-loops-as-one-g... http://threebean.org/patches/python-bugzilla/0004-Regression-fix.patch http://threebean.org/patches/python-bugzilla/0005-Always-return-bytestrings-...
-Ralph
On Tue, Jun 26, 2012 at 01:08:02PM -0400, Ralph Bean wrote:
On Mon, Jun 25, 2012 at 10:39:48PM -0700, Toshio Kuratomi wrote:
Here's my list of updated patches (including a rename of my original). I kept them separated out instead of rebasing just to preserve history and make them easier to cherry-pick.
http://threebean.org/patches/python-bugzilla/0001-Python3-support.patch http://threebean.org/patches/python-bugzilla/0002-Replace-str-foo-with-six.t... http://threebean.org/patches/python-bugzilla/0003-Rewrite-two-loops-as-one-g... http://threebean.org/patches/python-bugzilla/0004-Regression-fix.patch http://threebean.org/patches/python-bugzilla/0005-Always-return-bytestrings-...
These look good to me.
- Needs python-2.6 or greater (as python-2.5 doesn't have from __future__ import print_function, the new format of "except Foo as bar:", and relative imports)
This is correct. Can anyone comment on what python versions python-bugzilla is required to support? EPEL6 is on python-2.6 now.
So this question and whether adding the dep on python-six is okay are the questions that need answering.
-Toshio
On Tue, Jun 26, 2012 at 11:20:30AM -0700, Toshio Kuratomi wrote:
On Tue, Jun 26, 2012 at 01:08:02PM -0400, Ralph Bean wrote:
On Mon, Jun 25, 2012 at 10:39:48PM -0700, Toshio Kuratomi wrote:
Here's my list of updated patches (including a rename of my original). I kept them separated out instead of rebasing just to preserve history and make them easier to cherry-pick.
http://threebean.org/patches/python-bugzilla/0001-Python3-support.patch http://threebean.org/patches/python-bugzilla/0002-Replace-str-foo-with-six.t... http://threebean.org/patches/python-bugzilla/0003-Rewrite-two-loops-as-one-g... http://threebean.org/patches/python-bugzilla/0004-Regression-fix.patch http://threebean.org/patches/python-bugzilla/0005-Always-return-bytestrings-...
These look good to me.
- Needs python-2.6 or greater (as python-2.5 doesn't have from __future__ import print_function, the new format of "except Foo as bar:", and relative imports)
This is correct. Can anyone comment on what python versions python-bugzilla is required to support? EPEL6 is on python-2.6 now.
So this question and whether adding the dep on python-six is okay are the questions that need answering.
It has been over a month, so I'm bumping this discussion.
I understand in retrospect that I introduced a patch that could potentially destabilize python-bugzilla thus giving everybody nightmares after the BZ upgrade earlier in the Summer.
I think this work maintains all the backwards compatibility it can, with exceptions to what Toshio has raised: no more python-2.5 and a new dependency on python-six.
Someday, python2 will be unmaintained. We won't be able to start porting things to python3 that depend on python-bugzilla until it itself is ported. Better to get a head start.
On Tue, Jun 26, 2012 at 11:20:30AM -0700, Toshio Kuratomi wrote:
On Tue, Jun 26, 2012 at 01:08:02PM -0400, Ralph Bean wrote:
On Mon, Jun 25, 2012 at 10:39:48PM -0700, Toshio Kuratomi wrote:
Here's my list of updated patches (including a rename of my original). I kept them separated out instead of rebasing just to preserve history and make them easier to cherry-pick.
http://threebean.org/patches/python-bugzilla/0001-Python3-support.patch http://threebean.org/patches/python-bugzilla/0002-Replace-str-foo-with-six.t... http://threebean.org/patches/python-bugzilla/0003-Rewrite-two-loops-as-one-g... http://threebean.org/patches/python-bugzilla/0004-Regression-fix.patch http://threebean.org/patches/python-bugzilla/0005-Always-return-bytestrings-...
These look good to me.
- Needs python-2.6 or greater (as python-2.5 doesn't have from __future__ import print_function, the new format of "except Foo as bar:", and relative imports)
This is correct. Can anyone comment on what python versions python-bugzilla is required to support? EPEL6 is on python-2.6 now.
So this question and whether adding the dep on python-six is okay are the questions that need answering.
It has been over a month, so I'm bumping this discussion.
I understand in retrospect that I introduced a patch that could potentially destabilize python-bugzilla thus giving everybody nightmares after the BZ upgrade earlier in the Summer.
I think this work maintains all the backwards compatibility it can, with exceptions to what Toshio has raised: no more python-2.5 and a new dependency on python-six.
Someday, python2 will be unmaintained. We won't be able to start porting things to python3 that depend on python-bugzilla until it itself is ported. Better to get a head start.
On 08/01/2012 08:55 AM, Ralph Bean wrote:
On Tue, Jun 26, 2012 at 11:20:30AM -0700, Toshio Kuratomi wrote:
On Tue, Jun 26, 2012 at 01:08:02PM -0400, Ralph Bean wrote:
On Mon, Jun 25, 2012 at 10:39:48PM -0700, Toshio Kuratomi wrote:
Here's my list of updated patches (including a rename of my original). I kept them separated out instead of rebasing just to preserve history and make them easier to cherry-pick.
http://threebean.org/patches/python-bugzilla/0001-Python3-support.patch http://threebean.org/patches/python-bugzilla/0002-Replace-str-foo-with-six.t... http://threebean.org/patches/python-bugzilla/0003-Rewrite-two-loops-as-one-g... http://threebean.org/patches/python-bugzilla/0004-Regression-fix.patch http://threebean.org/patches/python-bugzilla/0005-Always-return-bytestrings-...
These look good to me.
- Needs python-2.6 or greater (as python-2.5 doesn't have from __future__ import print_function, the new format of "except Foo as bar:", and relative imports)
This is correct. Can anyone comment on what python versions python-bugzilla is required to support? EPEL6 is on python-2.6 now.
So this question and whether adding the dep on python-six is okay are the questions that need answering.
It has been over a month, so I'm bumping this discussion.
I understand in retrospect that I introduced a patch that could potentially destabilize python-bugzilla thus giving everybody nightmares after the BZ upgrade earlier in the Summer.
I think this work maintains all the backwards compatibility it can, with exceptions to what Toshio has raised: no more python-2.5 and a new dependency on python-six.
Someday, python2 will be unmaintained. We won't be able to start porting things to python3 that depend on python-bugzilla until it itself is ported. Better to get a head start.
Hey Ralph, sorry for the general lack of response (excluding Toshio being awesome).
Originally my thought was to cut another python-bugzilla release or two, with a whole bunch more unit tests and squashing most of the reports in bugzilla. That way we could push a more solid version out to epel 5, commit your patches the next day, and then declare that we aren't ever rebasing again in epel 5.
And then I didn't find the time/energy/motivation to do anything with python-bugzilla for months :)
During that time, epel5 /usr/bin/bugzilla has been completely busted due to a python2.5-ism:
https://bugzilla.redhat.com/show_bug.cgi?id=854979
It took 3 months for anyone to file a bug which leads me to believe that there aren't enough epel5 users to hold up taking this patch.
Ralph and/or Toshio, can you clarify a couple things for me?
1) How can I test future changes to make sure they work on both python2 and python3?
2) If packaging this new version for epel6, I'd need to add a dep on python26 and python-six. How to ensure that python-bugzilla is only used with python26 in that case? (or does python26 basically replace the stock RHEL6 python version?)
Also cc-ing wwoods, the main man in python-bugzilla land. Will, any issues with taking this patch soon-ish?
Thanks, Cole
On Thu, Oct 18, 2012 at 05:40:05PM -0400, Cole Robinson wrote:
During that time, epel5 /usr/bin/bugzilla has been completely busted due to a python2.5-ism:
https://bugzilla.redhat.com/show_bug.cgi?id=854979
It took 3 months for anyone to file a bug which leads me to believe that there aren't enough epel5 users to hold up taking this patch.
Ralph and/or Toshio, can you clarify a couple things for me?
- How can I test future changes to make sure they work on both python2 and
python3?
I'll let ralph answer this one.
- If packaging this new version for epel6, I'd need to add a dep on python26
and python-six. How to ensure that python-bugzilla is only used with python26 in that case? (or does python26 basically replace the stock RHEL6 python version?)
For epel6, the system python is at version 2.6 already so no worries there.
For epel5, you'd need to package a separate python26-bugzilla package. That package will build and install into the python2.6/site-packages directory and you'd change the requires to use python2.6 instead of python.
There's some information here: https://fedoraproject.org/wiki/Python26 and I'd probably also take a look at some of the packages that are already submitted.
Also cc-ing wwoods, the main man in python-bugzilla land. Will, any issues with taking this patch soon-ish?
Thanks, Cole
Cole, thanks for looking into this.
On Thu, Oct 18, 2012 at 05:40:05PM -0400, Cole Robinson wrote:
- How can I test future changes to make sure they work on both python2 and
python3?
While developing, I'll have two terminals open each with a different virtualenv[1] activated: one for python-2.7 and the other for python-3.2. I'll use nosy.py[2] in each terminal to re-run the test suite each time I change a file. This helps catch errors early.
A popular tool for testing a library against multiple python versions is tox[3]. I have used it on other people's projects but haven't yet taken the time to set it up on any of my own; it looks more robust than my nosy.py setup, but I can't really vouch for it yet.
Lastly, I rely pretty heavily on external continuous integration systems like travis-ci[4]. If python-bugzilla has %check enabled for both python and python3-bugzilla, then koji can provide some safeguards. I think there is some activity in Fedora Infra to set up a jenkins instance in our hosted cloud -- that can probably help with cross-python testing soon, too.
-Ralph
[1] http://www.virtualenv.org [2] http://github.com/ralphbean/threebean-dot-files/blob/master/content/bin/nosy... [3] http://tox.testrun.org [4] http://travis-ci.org
On 10/29/2012 09:49 AM, Ralph Bean wrote:
Cole, thanks for looking into this.
On Thu, Oct 18, 2012 at 05:40:05PM -0400, Cole Robinson wrote:
- How can I test future changes to make sure they work on both python2 and
python3?
While developing, I'll have two terminals open each with a different virtualenv[1] activated: one for python-2.7 and the other for python-3.2. I'll use nosy.py[2] in each terminal to re-run the test suite each time I change a file. This helps catch errors early.
A popular tool for testing a library against multiple python versions is tox[3]. I have used it on other people's projects but haven't yet taken the time to set it up on any of my own; it looks more robust than my nosy.py setup, but I can't really vouch for it yet.
Lastly, I rely pretty heavily on external continuous integration systems like travis-ci[4]. If python-bugzilla has %check enabled for both python and python3-bugzilla, then koji can provide some safeguards. I think there is some activity in Fedora Infra to set up a jenkins instance in our hosted cloud -- that can probably help with cross-python testing soon, too.
-Ralph
[1] http://www.virtualenv.org [2] http://github.com/ralphbean/threebean-dot-files/blob/master/content/bin/nosy... [3] http://tox.testrun.org [4] http://travis-ci.org
Thanks for the info Ralph. I still haven't forgotten about this.
However given that there's an RH bugzilla upgrade coming soon that will again break the majority of python-bugzilla usage, I'm holding off on this work until after the next release, since I still want to get an update into epel for RHEL5.
Hopefully this is the last time an rhbz upgrade causes these types of problems, since we are dropping most of the RH specific code from python-bugzilla.
- Cole
However given that there's an RH bugzilla upgrade coming soon that will again break the majority of python-bugzilla usage, I'm holding off on this work until after the next release, since I still want to get an update into epel for RHEL5.
No problem. Thanks for keeping me updated! :)
On 01/02/2013 08:16 AM, Cole Robinson wrote:
On 10/29/2012 09:49 AM, Ralph Bean wrote:
Cole, thanks for looking into this.
On Thu, Oct 18, 2012 at 05:40:05PM -0400, Cole Robinson wrote:
- How can I test future changes to make sure they work on both python2 and
python3?
While developing, I'll have two terminals open each with a different virtualenv[1] activated: one for python-2.7 and the other for python-3.2. I'll use nosy.py[2] in each terminal to re-run the test suite each time I change a file. This helps catch errors early.
A popular tool for testing a library against multiple python versions is tox[3]. I have used it on other people's projects but haven't yet taken the time to set it up on any of my own; it looks more robust than my nosy.py setup, but I can't really vouch for it yet.
Lastly, I rely pretty heavily on external continuous integration systems like travis-ci[4]. If python-bugzilla has %check enabled for both python and python3-bugzilla, then koji can provide some safeguards. I think there is some activity in Fedora Infra to set up a jenkins instance in our hosted cloud -- that can probably help with cross-python testing soon, too.
-Ralph
[1] http://www.virtualenv.org [2] http://github.com/ralphbean/threebean-dot-files/blob/master/content/bin/nosy... [3] http://tox.testrun.org [4] http://travis-ci.org
Thanks for the info Ralph. I still haven't forgotten about this.
However given that there's an RH bugzilla upgrade coming soon that will again break the majority of python-bugzilla usage, I'm holding off on this work until after the next release, since I still want to get an update into epel for RHEL5.
Hopefully this is the last time an rhbz upgrade causes these types of problems, since we are dropping most of the RH specific code from python-bugzilla.
I think python-bugzilla is in pretty good shape after the current release. Any future issues shouldn't be world breaking, so I'm fine with never rebasing epel5 again. This opens the door for bumping the dep to python-2.7 and committing python3 support.
TBH it's not a priority for me so I don't intend to actively work on it, but if you revive the patches I'll be happy to review + push + handle the packaging and testing side of things. Unfortunately the code has changed a lot, so it may be easier to just reimplement the whole thing than to try and get the old patch to apply.
Thanks, Cole
On Tue, Jun 25, 2013 at 10:42:05AM -0400, Cole Robinson wrote:
I think python-bugzilla is in pretty good shape after the current release. Any future issues shouldn't be world breaking, so I'm fine with never rebasing epel5 again. This opens the door for bumping the dep to python-2.7 and committing python3 support.
TBH it's not a priority for me so I don't intend to actively work on it, but if you revive the patches I'll be happy to review + push + handle the packaging and testing side of things. Unfortunately the code has changed a lot, so it may be easier to just reimplement the whole thing than to try and get the old patch to apply.
Cool. It's not really a priority here either, just a side project. I'll revive them some point in the next year and send them back to the list. Thanks again!
python-bugzilla@lists.stg.fedorahosted.org
-
Cole Robinson -
Ralph Bean -
Ralph Bean -
Toshio Kuratomi