Hi,
I have started work on porting the python-bugzilla for python 3. The work is being done at [1], this maintains backward compatibility. Currently it is stable, however I have only performed very limited testing. Hope this is something the project can use.
The changes on the branch include: - deprecating pycurl in favor of python-requests - a fair few pep8 fixes - generic python 2 to 3 change handling
It'd be great if someone more experienced with the code base can take a look to see if I've missed anything in particular.
Additionally, if any one has any feedback or performed additional tests, let me know.
Regards, Arun
[1] https://github.com/abn/python-bugzilla/tree/py3port
On Sun, Sep 22, 2013 at 08:48:31PM -0400, Arun Neelicattu wrote:
Hi,
I have started work on porting the python-bugzilla for python 3. The work is being done at [1], this maintains backward compatibility. Currently it is stable, however I have only performed very limited testing. Hope this is something the project can use.
The changes on the branch include: - deprecating pycurl in favor of python-requests - a fair few pep8 fixes - generic python 2 to 3 change handling
It'd be great if someone more experienced with the code base can take a look to see if I've missed anything in particular.
Additionally, if any one has any feedback or performed additional tests, let me know.
Hooray! Thank you. :)
A little work went into this last year[1], but it is way out of date now. Thanks for taking it on!
- Ralph
[1] - https://lists.fedorahosted.org/pipermail/python-bugzilla/2012-June/000010.ht...
No problems Ralph. Let me know if you see any issues with changes.
If nothing else comes up, I will post a patch.
-arun
----- Original Message -----
From: "Ralph Bean" rbean@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Sent: Monday, September 23, 2013 10:03:12 PM Subject: Re: [python-bugzilla] Python 3 support
On Sun, Sep 22, 2013 at 08:48:31PM -0400, Arun Neelicattu wrote:
Hi,
I have started work on porting the python-bugzilla for python 3. The work is being done at [1], this maintains backward compatibility. Currently it is stable, however I have only performed very limited testing. Hope this is something the project can use.
The changes on the branch include: - deprecating pycurl in favor of python-requests - a fair few pep8 fixes - generic python 2 to 3 change handling
It'd be great if someone more experienced with the code base can take a look to see if I've missed anything in particular.
Additionally, if any one has any feedback or performed additional tests, let me know.
Hooray! Thank you. :)
A little work went into this last year[1], but it is way out of date now. Thanks for taking it on!
- Ralph
[1] - https://lists.fedorahosted.org/pipermail/python-bugzilla/2012-June/000010.ht...
python-bugzilla mailing list python-bugzilla@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/python-bugzilla
On 09/22/2013 08:48 PM, Arun Neelicattu wrote:
Hi,
I have started work on porting the python-bugzilla for python 3. The work is being done at [1], this maintains backward compatibility. Currently it is stable, however I have only performed very limited testing. Hope this is something the project can use.
The changes on the branch include: - deprecating pycurl in favor of python-requests - a fair few pep8 fixes - generic python 2 to 3 change handling
It'd be great if someone more experienced with the code base can take a look to see if I've missed anything in particular.
Additionally, if any one has any feedback or performed additional tests, let me know.
Regards, Arun
Cool, thanks for this. I've looked over the changes briefly. Admittedly I'm pretty ignorant about python2 vs. python3 so forgive me if I say dumb things :)
We already run pep8 on the code, see 'python setup.py pylint', but it turns off some of the checks. Stuff like indentation fixes I'm neutral on, though 'Too many break lines' check is not to my liking: I like to use two line breaks to separate logical blocks of methods, especially in large classes like Bugzilla.
Regardless, if you want to submit those changes, please break them out separately and submit them first, bonus if you can supply an autopep8 command that reproduces the changes.
Any other change which doesn't have a dep on python2.7+ I'm happy to evaluate first. Ideally I'd like to boil the py3 patch down to the smallest amount necessary.
The main things we want to check:
- Is python-requests in epel6? I'm fine with dropping RHEL5 support, but I'd like to keep new releases working on RHEL6
- Can python-requests be made to reject bad ssl certs by default?
- Is --nosslverify correctly wired up?
- Does python-requests seamlessly use cookies generated by pycurl python-bugzilla? Don't want users to lose their creds on update!
- Is there some relatively easy way to at least run the unit test suite against python2 and python3? Instructions work, better if we can make python setup.py test just do-the-right-thing provided all the system components are vailable. Basically if it isn't easy to test both versions, I _will_ inevitably break one of them, so this is a blocker for me.
If those answers line up, and we get the minor pieces separated out and committed separately, hopefully we can get this done in short order. Though unfortunately I don't have much time to devote here besides reviewing + applying patches.
Thanks, Cole
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Cc: "Arun Neelicattu" abn@redhat.com Sent: Tuesday, September 24, 2013 8:29:07 AM Subject: Re: [python-bugzilla] Python 3 support
Cool, thanks for this. I've looked over the changes briefly. Admittedly I'm pretty ignorant about python2 vs. python3 so forgive me if I say dumb things :)
We already run pep8 on the code, see 'python setup.py pylint', but it turns off some of the checks. Stuff like indentation fixes I'm neutral on, though 'Too many break lines' check is not to my liking: I like to use two line breaks to separate logical blocks of methods, especially in large classes like Bugzilla.
Personally, I reckon following the the standard as-is where possible is the best approach for community projects. Just because this helps with consistency. But that is my personal point of view :)
I am happy to isolate those changes into a separate commit without the line break changes.
Regardless, if you want to submit those changes, please break them out separately and submit them first, bonus if you can supply an autopep8 command that reproduces the changes.
I will see what can be done here.
Any other change which doesn't have a dep on python2.7+ I'm happy to evaluate first. Ideally I'd like to boil the py3 patch down to the smallest amount necessary.
ack.
The main things we want to check:
- Is python-requests in epel6? I'm fine with dropping RHEL5 support, but I'd
like to keep new releases working on RHEL6
Yes.
fedora:18/python-requests-1.1.0-3.fc18 fedora:19/python-requests-1.1.0-4.fc19 fedora:epel:5/python26-requests-0.13.1-1.el5 fedora:epel:6/python-requests-1.1.0-4.el6
- Can python-requests be made to reject bad ssl certs by default?
IIUC this is the default behavior, and is controlled by sslverify. We retain the same signature as of _CURLTransport.
http://www.python-requests.org/en/latest/user/advanced/#ssl-cert-verificatio...
- Is --nosslverify correctly wired up?
Yes this is controlled by the sslverify argument. Will ensure this is tested before final patch.
- Does python-requests seamlessly use cookies generated by pycurl
python-bugzilla? Don't want users to lose their creds on update!
Hadn't considered this a requirement, Will need to check this one, it uses the same cookie jar, but not sure if there are any differences in the cookies themselves, I would assume not.
- Is there some relatively easy way to at least run the unit test suite
against python2 and python3? Instructions work, better if we can make python setup.py test just do-the-right-thing provided all the system components are vailable. Basically if it isn't easy to test both versions, I _will_ inevitably break one of them, so this is a blocker for me.
I am sure this is possible, but it might require some automagic scripts. Will provide them as I get the time.
At the moment, I am just using a virtual environment with -p=/usr/bin/python3
If those answers line up, and we get the minor pieces separated out and committed separately, hopefully we can get this done in short order. Though unfortunately I don't have much time to devote here besides reviewing + applying patches.
Will do the required changes soon, hopefully sometime next week and submit the patches.
-arun
On Mon, Sep 23, 2013 at 07:14:17PM -0400, Arun Neelicattu wrote:
- Is there some relatively easy way to at least run the unit test suite
against python2 and python3? Instructions work, better if we can make python setup.py test just do-the-right-thing provided all the system components are vailable. Basically if it isn't easy to test both versions, I _will_ inevitably break one of them, so this is a blocker for me.
I am sure this is possible, but it might require some automagic scripts. Will provide them as I get the time.
At the moment, I am just using a virtual environment with -p=/usr/bin/python3
The standard way to do this in the python community is with the "tox" test runner: http://testrun.org/tox/latest/ It is widely used.
I do not use it for my projects just because I haven't gotten around to absorbing it into my workflow yet. I do depend on travis-ci.org to run my tests against py2.6, py2.7, and py3.3 after I push -- this is quite easy to set up.
----- Original Message -----
From: "Ralph Bean" rbean@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Sent: Tuesday, September 24, 2013 9:48:53 AM Subject: Re: [python-bugzilla] Python 3 support
The standard way to do this in the python community is with the "tox" test runner: http://testrun.org/tox/latest/ It is widely used.
Did not think of that, thanks. I will look into that.
I do not use it for my projects just because I haven't gotten around to absorbing it into my workflow yet. I do depend on travis-ci.org to run my tests against py2.6, py2.7, and py3.3 after I push -- this is quite easy to set up.
+1 for travis or some automated CI infra.
Are there any thoughts on moving the project to github? Might open up to a wider audience and make workflow a bit more easier. But that is for another thread I suppose.
python-bugzilla mailing list python-bugzilla@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/python-bugzilla
On 09/23/2013 08:44 PM, Arun Neelicattu wrote:
----- Original Message -----
From: "Ralph Bean" rbean@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Sent: Tuesday, September 24, 2013 9:48:53 AM Subject: Re: [python-bugzilla] Python 3 support
The standard way to do this in the python community is with the "tox" test runner: http://testrun.org/tox/latest/ It is widely used.
Did not think of that, thanks. I will look into that.
I do not use it for my projects just because I haven't gotten around to absorbing it into my workflow yet. I do depend on travis-ci.org to run my tests against py2.6, py2.7, and py3.3 after I push -- this is quite easy to set up.
+1 for travis or some automated CI infra.
Are there any thoughts on moving the project to github? Might open up to a wider audience and make workflow a bit more easier. But that is for another thread I suppose.
I'm down with moving to github, though truthfully the idea of more contributions or people starting to use python-bugzilla to interact with something other than rhbz scares me :) At this point my time and inclination is pretty limited beyond keeping the lights on.
- Cole
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Cc: "Arun Neelicattu" abn@redhat.com Sent: Tuesday, September 24, 2013 11:07:20 AM Subject: Re: [python-bugzilla] Python 3 support
I'm down with moving to github, though truthfully the idea of more contributions or people starting to use python-bugzilla to interact with something other than rhbz scares me :) At this point my time and inclination is pretty limited beyond keeping the lights on.
- Cole
Clole, I am happy to help where I can.
As for the the usage, longer term this is a useful tool of which more will be demanded one way or the other, might as well be prepared! :)
Alright, finally got back to this.
This should now be complete as requested earlier. All tests are now passing, basic + functional (ro/rw) on python2 (2.7 tested) and python 3 (3.3 tested).
The smallest change set for Python3 support is: https://github.com/abn/python-bugzilla/commit/6f0bd169fa4748536eefd6c0bade1f...
I have also added helper scripts for development/testing and updated HACKING file: https://github.com/abn/python-bugzilla/commit/f9914db082a26a5bb60541e1701b0c...
I have not made any pep8 changes to any existing code.
Any review might be good :)
-arun
On Tue, 2013-09-24 at 06:58 -0400, Arun Neelicattu wrote:
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Cc: "Arun Neelicattu" abn@redhat.com Sent: Tuesday, September 24, 2013 11:07:20 AM Subject: Re: [python-bugzilla] Python 3 support
I'm down with moving to github, though truthfully the idea of more contributions or people starting to use python-bugzilla to interact with something other than rhbz scares me :) At this point my time and inclination is pretty limited beyond keeping the lights on.
- Cole
Clole, I am happy to help where I can.
As for the the usage, longer term this is a useful tool of which more will be demanded one way or the other, might as well be prepared! :) _______________________________________________ python-bugzilla mailing list python-bugzilla@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/python-bugzilla
Submitting patches for this.
On Thu, 2013-10-10 at 18:05 +1000, Arun Babu Neelicattu wrote:
Alright, finally got back to this.
This should now be complete as requested earlier. All tests are now passing, basic + functional (ro/rw) on python2 (2.7 tested) and python 3 (3.3 tested).
The smallest change set for Python3 support is: https://github.com/abn/python-bugzilla/commit/6f0bd169fa4748536eefd6c0bade1f...
I have also added helper scripts for development/testing and updated HACKING file: https://github.com/abn/python-bugzilla/commit/f9914db082a26a5bb60541e1701b0c...
I have not made any pep8 changes to any existing code.
Any review might be good :)
-arun
On Tue, 2013-09-24 at 06:58 -0400, Arun Neelicattu wrote:
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Cc: "Arun Neelicattu" abn@redhat.com Sent: Tuesday, September 24, 2013 11:07:20 AM Subject: Re: [python-bugzilla] Python 3 support
I'm down with moving to github, though truthfully the idea of more contributions or people starting to use python-bugzilla to interact with something other than rhbz scares me :) At this point my time and inclination is pretty limited beyond keeping the lights on.
- Cole
Clole, I am happy to help where I can.
As for the the usage, longer term this is a useful tool of which more will be demanded one way or the other, might as well be prepared! :) _______________________________________________ python-bugzilla mailing list python-bugzilla@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/python-bugzilla
python-bugzilla mailing list python-bugzilla@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/python-bugzilla
On 10/15/2013 04:19 AM, Arun Babu Neelicattu wrote:
Submitting patches for this.
On Thu, 2013-10-10 at 18:05 +1000, Arun Babu Neelicattu wrote:
Alright, finally got back to this.
This should now be complete as requested earlier. All tests are now passing, basic + functional (ro/rw) on python2 (2.7 tested) and python 3 (3.3 tested).
The smallest change set for Python3 support is: https://github.com/abn/python-bugzilla/commit/6f0bd169fa4748536eefd6c0bade1f...
I have also added helper scripts for development/testing and updated HACKING file: https://github.com/abn/python-bugzilla/commit/f9914db082a26a5bb60541e1701b0c...
I have not made any pep8 changes to any existing code.
Any review might be good :)
Thanks for following up.
In the future, please send patches using git send-email, makes reviewing easier.
That said, these patches are difficult to review, they are still conflating fairly trivial sed-like changes with extremely important functional changes (pycurl->requests). I understand that doing this work in one lump is probably easiest for you, but it makes review and maintenance harder. Maybe what I said earlier mislead you here since I asked for to 'boil the py3 patch down to the smallest amount necessary', but I didn't mean it had to be a single commit :)
I'm realize that unwinding a large commit is frustrating and unfun, but ideally this series would be the following distinct patches:
- print "foo" -> print("foo") - sys.exc_info() change - fix imports to work on python2/3 - remaining python3 issues - convert to python-requests - test suite setup for vs py3 (maybe this is earlier in the series) - build system changes
It might make sense to re-arrange that or make it even more fine grained, that's just off the top of my head.
So, for example, if that print() patch were on the list right now and demonstratably didn't cause any test suite/pylint regression, I'd commit and push it since it's trivial. But as is, I can't push this patch until I can get some time to verify there aren't any ssl or cookie regressions with the python-requests support, in addition to the review overhead.
Most of the conversion looks fine, but a couple bits from the patches jumped out at me:
.gitignore changes: stuff like .so and nosetests.xml don't apply to us, so I don't want to track them. Even stuff like *.swp shouldn't be in our current .gitconfig, people should configure a global .gitconfig and ignore bits generated by their tools of choice. Expecting every project to track these bits does not scale. So please limit the gitignore change to only bits that are generated by any build system changes.
setup.py changes: conditionalizing get_requirements on the prescence of pip is a hack: sounds like it will break local RPM build testing, and we can't be sure that pip won't be pulled in by some python-bugzilla depedency for fedora builds. Any way to wire this in with --single-version-externally-managed ? Maybe setuptools does it automatically.
tests/rw_functional.py refactors the priority/severity test for reasons I can't discern.
Thanks, Cole
Hi Cole,
I misunderstood, just assumed you wanted a single py2 to py3 commit. I can do the split over the weekend hopefully.
.gitignore changes: ack
setup.py changes: i think setuptools should handle it, will test further and see what I can do to make it less 'hacky'.
tests/rw_functional.py refactoring: this was since the tests (original and new) raised Fault as the test user did not have privileges to change the priority/severity and this was uncaught. Just figured it was cleaner/extensible with the refactor. Do you want me to leave it as is? Or just add a generic try/except with a generic failure message around that block?
-arun
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Cc: "Arun Neelicattu" abn@redhat.com Sent: Wednesday, October 16, 2013 11:12:47 PM Subject: Re: [python-bugzilla] Python 3 support
Thanks for following up.
In the future, please send patches using git send-email, makes reviewing easier.
That said, these patches are difficult to review, they are still conflating fairly trivial sed-like changes with extremely important functional changes (pycurl->requests). I understand that doing this work in one lump is probably easiest for you, but it makes review and maintenance harder. Maybe what I said earlier mislead you here since I asked for to 'boil the py3 patch down to the smallest amount necessary', but I didn't mean it had to be a single commit :)
I'm realize that unwinding a large commit is frustrating and unfun, but ideally this series would be the following distinct patches:
- print "foo" -> print("foo")
- sys.exc_info() change
- fix imports to work on python2/3
- remaining python3 issues
- convert to python-requests
- test suite setup for vs py3 (maybe this is earlier in the series)
- build system changes
It might make sense to re-arrange that or make it even more fine grained, that's just off the top of my head.
So, for example, if that print() patch were on the list right now and demonstratably didn't cause any test suite/pylint regression, I'd commit and push it since it's trivial. But as is, I can't push this patch until I can get some time to verify there aren't any ssl or cookie regressions with the python-requests support, in addition to the review overhead.
Most of the conversion looks fine, but a couple bits from the patches jumped out at me:
.gitignore changes: stuff like .so and nosetests.xml don't apply to us, so I don't want to track them. Even stuff like *.swp shouldn't be in our current .gitconfig, people should configure a global .gitconfig and ignore bits generated by their tools of choice. Expecting every project to track these bits does not scale. So please limit the gitignore change to only bits that are generated by any build system changes.
setup.py changes: conditionalizing get_requirements on the prescence of pip is a hack: sounds like it will break local RPM build testing, and we can't be sure that pip won't be pulled in by some python-bugzilla depedency for fedora builds. Any way to wire this in with --single-version-externally-managed ? Maybe setuptools does it automatically.
tests/rw_functional.py refactors the priority/severity test for reasons I can't discern.
Thanks, Cole
On 10/16/2013 10:15 AM, Arun Neelicattu wrote:
Hi Cole,
I misunderstood, just assumed you wanted a single py2 to py3 commit. I can do the split over the weekend hopefully.
Great, thanks a lot.
.gitignore changes: ack
setup.py changes: i think setuptools should handle it, will test further and see what I can do to make it less 'hacky'.
tests/rw_functional.py refactoring: this was since the tests (original and new) raised Fault as the test user did not have privileges to change the priority/severity and this was uncaught. Just figured it was cleaner/extensible with the refactor. Do you want me to leave it as is? Or just add a generic try/except with a generic failure message around that block?
Hmm. I'm probably the only one that has ever run most of those tests, but I'm surprised changing severity/priority raised an error and the rest of the suite didn't. Double check that you can still reproduce (with cached login credentials), but if it's still an issue I'm fine with the current code, just add a short comment about it.
Thanks, Cole
python-bugzilla@lists.stg.fedorahosted.org
-
Arun Babu Neelicattu -
Cole Robinson -
Ralph Bean