From: Arun Babu Neelicattu abn@redhat.com
--- .gitignore | 3 +++ HACKING | 47 +++++++++++++++++++++++++++++++------ contrib/activate-dev-env | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ contrib/run-tests | 21 +++++++++++++++++ setup.py | 2 +- 5 files changed, 125 insertions(+), 8 deletions(-) create mode 100644 contrib/activate-dev-env create mode 100755 contrib/run-tests
diff --git a/.gitignore b/.gitignore index 7608296..8aceef6 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,6 @@ MANIFEST dist build .coverage + +#python-bugzilla venvs +dev-env* diff --git a/HACKING b/HACKING index d269b33..54b77c0 100644 --- a/HACKING +++ b/HACKING @@ -1,23 +1,56 @@ -If submitting any patch, please verify that no new pylint or pep8 violations -are introduced: +Hackers Guide +=============
- python setup.py pylint +1. Setting up the environment +----------------------------- +To start development, you can set-up/activate a virtual environment by using the +following command: + source contrib/activate-dev-env [python2|python3]
-And ensure that the basic unit test suite does not regress +Note: Providing no arguments will attempt to use python2.
- python setup.py test +# Manually activating an environment + source dev-env-${NAME}/bin/activate
+2. Running tests +---------------- +Once you have already activated an environment, you can use the following. + +# Basic unit test suite + python setup.py test + +# Functional tests There are more comprehensive tests that are disabled by default. Readonly functional tests that run against several public bugzilla instances. No login account is required:
- python setup.py test --ro-functional + python setup.py test --ro-functional
And read/write functional tests. These currently run against the test bugzilla instance at partner-bugzilla.redhat.com, and requires a valid login there:
- python setup.py test --rw-functional + python setup.py test --rw-functional + +Note: Before running rw-functional tests, make sure you have logged using: + python bugzilla-cli \ + --bugzilla="https://partner-bugzilla.redhat.com/xmlrpc.cgi" \ + --user=$USER login + +3. pylint and pep8 +------------------ +To test for pylint or pep8 violations, you can run: + python setup.py pylint + +Note: This expects that you already have pylint and pep8 (installed when setting +up virtualenv) isntalled. + +4. Patch Submission +------------------- +If you are submitting a patch, ensure the following: + [REQ] verify that no new pylint or pep8 violations + [REQ] run basic unit test suite across all python versions: + bash contrib/run-tests
Running any of the functional tests is not a requirement for patch submission, but please give them a go if you are interested. diff --git a/contrib/activate-dev-env b/contrib/activate-dev-env new file mode 100644 index 0000000..5fcd430 --- /dev/null +++ b/contrib/activate-dev-env @@ -0,0 +1,60 @@ +# Helper script to create (if required) and activate a virtual environment for +# a specified PYTHON_BINARY (python|python2|python3) and install dependencies +# required for testing and development. +# +# Usage: source $0 [PYTHON_BINARY] + +# use env python by default +PYTHON_BIN=python2 +VIRTIALENV_CMD=virtualenv +SETUP_PY="setup.py" +DEV_MODULE_DIR="bugzilla" +DIR_PREFIX="dev-env" + +if [ $# -ne 0 ]; then + PYTHON_BIN=$1 +fi + +function activate() +{ + DIR_NAME=$1 + echo "INFO: Activating virtualenv at ${DIR_NAME}" + source ${DIR_NAME}/bin/activate +} + +function virtualize() +{ + PROMPT=$(basename ${PYTHON_BIN}) + DIR_NAME="${DIR_PREFIX}-${PROMPT}" + + if [ ! -d ${DIR_NAME} ]; then + echo "INFO: Creating virtualenv ${DIR_NAME}" + ${VIRTIALENV_CMD} -p $(which ${PYTHON_BIN}) --prompt "${PROMPT}" ${DIR_NAME} + fi + + if [ -d ${DIR_NAME} ]; then + activate $DIR_NAME + + for REQ in $(find ./ -maxdepth 1 -type f -name "*requirements.txt"); do + pip install -qr "${REQ}" + done + else + echo >&2 "ERROR: Failed to activate virtualenv." + fi +} + + +if ! command -v ${VIRTIALENV_CMD} >/dev/null 2>&1; then + # We require virtuaenv to be available + echo >&2 "ERROR: virtualenv command not found" +elif ! command -v $PYTHON_BIN >/dev/null 2>&1; then + # Ensure PYTHON_BIN is valid + echo >&2 "ERROR: ${PYTHON_BIN} command not found" +elif ! ${PYTHON_BIN} --version 2>&1 | grep -q '^Python'; then + # Ensure PYTHON_BIN is python + echo >&2 "ERROR: '${PYTHON_BIN} --version' returned an invalid response. Is ${PYTHON_BIN} really python?" +elif [ ! -f ${SETUP_PY} ]; then + echo >&2 "ERROR: ${SETUP_PY} not found" +else + virtualize +fi diff --git a/contrib/run-tests b/contrib/run-tests new file mode 100755 index 0000000..644f5d9 --- /dev/null +++ b/contrib/run-tests @@ -0,0 +1,21 @@ +#!/usr/bin/env bash + +# Helper script to run test for each supported python versions +# Defaults to running basic tests. The script activates the correct environment +# as required. + +# Usage: $0 [options] +# Example: $0 --ro-functional + +ARGS="$@" + +for BIN in python2 python3; do + if command -v ${BIN} >/dev/null 2>&1; then + echo "INFO: Running tests for ${BIN}" + . ./contrib/activate-dev-env ${BIN} + python setup.py test ${ARGS} + echo "INFO: Tests completed for ${BIN}" + else + echo "INFO: ${BIN} not found, skipping tests." + fi +done diff --git a/setup.py b/setup.py index 415eee0..6abd9b0 100755 --- a/setup.py +++ b/setup.py @@ -49,7 +49,7 @@ class TestCommand(Command): usecov = False
if usecov: - cov = coverage.coverage(omit=["/*/tests/*", "/usr/*"]) + cov = coverage.coverage(omit=["/*/tests/*", "/usr/*", "*dev-env*"]) cov.erase() cov.start()
From: Arun Babu Neelicattu <abn at redhat.com>
.gitignore | 3 +++ HACKING | 47 +++++++++++++++++++++++++++++++------ contrib/activate-dev-env | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ contrib/run-tests | 21 +++++++++++++++++ setup.py | 2 +- 5 files changed, 125 insertions(+), 8 deletions(-) create mode 100644 contrib/activate-dev-env create mode 100755 contrib/run-tests
Thanks, I've pushed this now. But I'm hitting an issue with contrib/run-tests. The python2 virtualenv setup works, but the python3 one fails:
INFO: Running tests for python3 INFO: Creating virtualenv dev-env-python3 Running virtualenv with interpreter /usr/bin/python3 New python executable in dev-env-python3/bin/python3 Also creating executable in dev-env-python3/bin/python A globally installed setuptools was found (in /usr/lib/python3.3/site-packages) Use the --no-site-packages option to use distribute in the virtualenv. Installing distribute..................................................................................................................................................................................................................................................................................................................................................................................................done. Installing pip.... Complete output from command /home/crobinso/src/p...-python3/bin/python3 /home/crobinso/src/p...on3/bin/easy_install /usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz: /home/crobinso/src/python-bugzilla/dev-env-python3/bin/python3: can't open file '/home/crobinso/src/python-bugzilla/dev-env-python3/bin/easy_install': [Errno 2] No such file or directory ---------------------------------------- ...Installing pip...done. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 1952, in <module> main() File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 810, in main never_download=options.never_download) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 912, in create_environment install_pip(py_executable, search_dirs=search_dirs, never_download=never_download) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 648, in install_pip filter_stdout=_filter_setup) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 878, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /home/crobinso/src/p...-python3/bin/python3 /home/crobinso/src/p...on3/bin/easy_install /usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz failed with error code 2 INFO: Activating virtualenv at dev-env-python3 ./contrib/activate-dev-env: line 24: dev-env-python3/bin/activate: No such file or directory
Any ideas?
- Cole
hmm, hard to say for sure however, I reckon it has got something to do with the pip version available
It seems to be using the source for pip from:
/usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz
Not sure if this is the issue, but it could be that the 1.0.2 version was not python 3 ready?
Also I noted this change in the activate-dev-env script: - pip install -qr "${REQ}" + pip install -qr "${REQ}" || exit 1
That might not be a good idea as we are sourcing the script. This will kill the shell.
-arun
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "Arun Neelicattu" abn@redhat.com, "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Sent: Sunday, November 10, 2013 5:19:16 AM Subject: Re: [python-bugzilla] [PATCH] Add contrib scripts and relevant
From: Arun Babu Neelicattu <abn at redhat.com>
.gitignore | 3 +++ HACKING | 47 +++++++++++++++++++++++++++++++------ contrib/activate-dev-env | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ contrib/run-tests | 21 +++++++++++++++++ setup.py | 2 +- 5 files changed, 125 insertions(+), 8 deletions(-) create mode 100644 contrib/activate-dev-env create mode 100755 contrib/run-tests
Thanks, I've pushed this now. But I'm hitting an issue with contrib/run-tests. The python2 virtualenv setup works, but the python3 one fails:
INFO: Running tests for python3 INFO: Creating virtualenv dev-env-python3 Running virtualenv with interpreter /usr/bin/python3 New python executable in dev-env-python3/bin/python3 Also creating executable in dev-env-python3/bin/python A globally installed setuptools was found (in /usr/lib/python3.3/site-packages) Use the --no-site-packages option to use distribute in the virtualenv. Installing distribute..................................................................................................................................................................................................................................................................................................................................................................................................done. Installing pip.... Complete output from command /home/crobinso/src/p...-python3/bin/python3 /home/crobinso/src/p...on3/bin/easy_install /usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz: /home/crobinso/src/python-bugzilla/dev-env-python3/bin/python3: can't open file '/home/crobinso/src/python-bugzilla/dev-env-python3/bin/easy_install': [Errno 2] No such file or directory
...Installing pip...done. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 1952, in <module> main() File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 810, in main never_download=options.never_download) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 912, in create_environment install_pip(py_executable, search_dirs=search_dirs, never_download=never_download) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 648, in install_pip filter_stdout=_filter_setup) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 878, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /home/crobinso/src/p...-python3/bin/python3 /home/crobinso/src/p...on3/bin/easy_install /usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz failed with error code 2 INFO: Activating virtualenv at dev-env-python3 ./contrib/activate-dev-env: line 24: dev-env-python3/bin/activate: No such file or directory
Any ideas?
- Cole
On 11/10/2013 06:44 PM, Arun Neelicattu wrote:
hmm, hard to say for sure however, I reckon it has got something to do with the pip version available
It seems to be using the source for pip from:
/usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz
Not sure if this is the issue, but it could be that the 1.0.2 version was not python 3 ready?
Ah, good call. I had some really old cache there, you can tell I don't use pip/virtualenv much :)
Also I noted this change in the activate-dev-env script:
pip install -qr "${REQ}"
pip install -qr "${REQ}" || exit 1
That might not be a good idea as we are sourcing the script. This will kill the shell.
Without that, when the python3 init failed, the script continued on and ran the tests as usual, which all pass since we were still running in the python2 env, which was quite misleading. There may be a better place to stick the 'exit 1' or similar.
Thanks, Cole
----- Original Message -----
From: "Cole Robinson" crobinso@redhat.com To: "Arun Neelicattu" abn@redhat.com, "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Sent: Sunday, November 10, 2013 5:19:16 AM Subject: Re: [python-bugzilla] [PATCH] Add contrib scripts and relevant
From: Arun Babu Neelicattu <abn at redhat.com>
.gitignore | 3 +++ HACKING | 47 +++++++++++++++++++++++++++++++------ contrib/activate-dev-env | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ contrib/run-tests | 21 +++++++++++++++++ setup.py | 2 +- 5 files changed, 125 insertions(+), 8 deletions(-) create mode 100644 contrib/activate-dev-env create mode 100755 contrib/run-tests
Thanks, I've pushed this now. But I'm hitting an issue with contrib/run-tests. The python2 virtualenv setup works, but the python3 one fails:
INFO: Running tests for python3 INFO: Creating virtualenv dev-env-python3 Running virtualenv with interpreter /usr/bin/python3 New python executable in dev-env-python3/bin/python3 Also creating executable in dev-env-python3/bin/python A globally installed setuptools was found (in /usr/lib/python3.3/site-packages) Use the --no-site-packages option to use distribute in the virtualenv. Installing distribute..................................................................................................................................................................................................................................................................................................................................................................................................done. Installing pip.... Complete output from command /home/crobinso/src/p...-python3/bin/python3 /home/crobinso/src/p...on3/bin/easy_install /usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz: /home/crobinso/src/python-bugzilla/dev-env-python3/bin/python3: can't open file '/home/crobinso/src/python-bugzilla/dev-env-python3/bin/easy_install': [Errno 2] No such file or directory
...Installing pip...done. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 1952, in <module> main() File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 810, in main never_download=options.never_download) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 912, in create_environment install_pip(py_executable, search_dirs=search_dirs, never_download=never_download) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 648, in install_pip filter_stdout=_filter_setup) File "/usr/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg/virtualenv.py", line 878, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /home/crobinso/src/p...-python3/bin/python3 /home/crobinso/src/p...on3/bin/easy_install /usr/lib/python2.7/s...ort/pip-1.0.2.tar.gz failed with error code 2 INFO: Activating virtualenv at dev-env-python3 ./contrib/activate-dev-env: line 24: dev-env-python3/bin/activate: No such file or directory
Any ideas?
- Cole
On Tue, 2013-11-12 at 09:33 -0500, Cole Robinson wrote:
Ah, good call. I had some really old cache there, you can tell I don't use pip/virtualenv much :)
:)
Without that, when the python3 init failed, the script continued on and ran the tests as usual, which all pass since we were still running in the python2 env, which was quite misleading. There may be a better place to stick the 'exit 1' or similar.
Hmm, might need to move the check elsewhere, since the run-tests script is running in a sub-shell it should be file, but this will cause issues if a we are activating the environment manually.
Will submit a patch soon. :)
Sent a patch for this one. Tests should skip if the env is not active now.
----- Original Message -----
From: "Arun Babu Neelicattu" abn@redhat.com To: "Cole Robinson" crobinso@redhat.com Cc: "python-bugzilla user/developer list" python-bugzilla@lists.fedorahosted.org Sent: Thursday, November 14, 2013 11:38:17 AM Subject: Re: [python-bugzilla] [PATCH] Add contrib scripts and relevant
On Tue, 2013-11-12 at 09:33 -0500, Cole Robinson wrote:
Ah, good call. I had some really old cache there, you can tell I don't use pip/virtualenv much :)
:)
Without that, when the python3 init failed, the script continued on and ran the tests as usual, which all pass since we were still running in the python2 env, which was quite misleading. There may be a better place to stick the 'exit 1' or similar.
Hmm, might need to move the check elsewhere, since the run-tests script is running in a sub-shell it should be file, but this will cause issues if a we are activating the environment manually.
Will submit a patch soon. :)
python-bugzilla mailing list python-bugzilla@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/python-bugzilla
python-bugzilla@lists.stg.fedorahosted.org