You are not logged in Log in Join
You are here: Home » Members » Stefan's Home » Test Runner Comparison » View Document

Log in
Name

Password

 

Test Runner Comparison

Compares available test runners for Zope 2

 

testrunner.py

test.py

History

Part of the Zope distribution until version 2.7.1. Support was dropped because it would not run Zope's own unit tests anymore. There is an up-to-date version of testrunner.py at the URL below.

Part of the Zope distribution since version 2.7.0. Originally a backport of the Z3 testrunner of the same name.

URL http://zope.org/Members/shh/TestRunner http://cvs.zope.org/Zope/test.py
Installs to $ZOPE_HOME/bin/testrunner.py $ZOPE_HOME/bin/test.py
Usage TestRunner Help See the test.py docstring.
Runs Zope's unit tests no yes
Runs tests in instance homes yes yes
Follows symbolic links yes yes
Reads zope.conf files yes yes
Runs entire test suites yes yes
Runs individual test modules yes yes
Runs individual test methods no yes
Locates tests by

From the given start directory, recursively searches all directories for files matching the shell pattern test*.py. If one is found, testrunner changes to the respective directory and imports the module. If the module has a test_suite function it is executed, and the returned test suite is run immediately. Only then the search for more tests continues.

If instead of a directory a file name is specified, only tests from this module will be imported and run.

From the given start directory, recursively searches for Python packages named tests and inside the package for files matching the shell pattern test*.py. Matching modules are added to the list of testable files. The runner then imports the modules from the list, executes their test_suite functions, and adds the returned test suite to the master suite. Finally, the accumulated test suites are run. [1]

If modulefilter or testfilter arguments have been passed, only tests matching these patterns will be collected and run.

Note that test.py prepends the start directory to the PYTHONPATH. [2]

Ignores
  • Directories build and build-base.

  • Files test_all.py and testrunner.py.

  • All files that do not contain the strings "def test_suite(" or "framework(".
  • Tests that are not in packages named tests. [1]

  • Tests in directories that are not Python packages and/or not importable from the start directory.

  • Tests in modules whose level attribute is lower than the currently active level (1 by default).
Caveats

Due to the way it collects tests, testrunner.py is not well suited for running giant, interdependent test suites; like Zope's.

Adds the start directory to the beginning of sys.path so it can import the test modules. [2]

The option to "remove stale bytecode" is enabled by default. You may want to add the --keepbytecode flag to skip the directory scans and avoid unnecessary thrashing of disk caches.

Examples

Running Zope Unit Tests

cd $ZOPE_HOME
python2.3 bin/test.py -v

Running CMF Unit Tests

cd $INSTANCE_HOME
python2.3 $ZOPE_HOME/bin/testrunner.py -qid Products

cd $INSTANCE_HOME
python2.3 $ZOPE_HOME/bin/test.py -v  \
    --config-file etc/zope.conf      \
    --libdir Products                \
    \!test_all

Running Plone Unit Tests

cd $INSTANCE_HOME
python2.3 $ZOPE_HOME/bin/testrunner.py -qid Products/CMFPlone/tests

cd $INSTANCE_HOME
python2.3 $ZOPE_HOME/bin/test.py -v  \
    --config-file etc/zope.conf      \
    --libdir Products/CMFPlone

[1] If the -f flag is given, test.py searches for packages named ftests.

[2] test.py adds the directory specified by the --libdir option to sys.path.