History for ZopeTestingGuidelines

??changed:
-
Jeremy Hylton in response to MartijnF's query about the testing committee:

  The committee included a bunch of people: Jim, Jeremy, Jeffrey, Fred,
  Evan, Chris, Brian.

  The simple stuff is really 
  simple:   

    - Use the standard unittest module.  

    - Put tests in a subpackage named tests under the package being
      tested.

    - Never write tests that depend on being executed from a
      particular location.

    - If you need to store data in the tests directory, get at it   
      using imports.  Example:                                      
          from ZODB.tests import MinPO
          dir = os.path.dirname(MinPO.__file__)
    - The test runner (person or program) is responsible for making
      sure the Python path is set correctly to find all the needed 
      modules.
              
    - Do not modify sys.path!

    - Be aware that some tests need to work with Zope and
      StandaloneZODB.  If you modify those tests, make sure they work
      with both checkouts.                                           

          packages: ZODB, BTrees, bsddb3Storage, ZEO, ExtensionClass,
                    Persistence, ThreadedAsync, zLOG, zdaemon   

  There was more stuff about the Testing package in Zope/lib/python.
  I'm not sure I understand the details, but you need to import it to
  use it.
  
  There are a bunch of examples that execfile() something called  
  framework.py, but that's deprecated.  I believe Tres has started
  removing it.

  Later:

    I remembered one more rule (and probably still need to remember a few
    more).  Each test module should define a function test_suite() that
    returns a unittest TestSuite object.  The affords some flexibility in
    selecting test cases for a particular environment.  The ZEO tests, for
    example, are different on Win32 than on Unix.

  Commentary from MartijnF:

    Tres is indeed on a hunt to remove 'framework.py'. If you indeed take care   
    to put '/path/to/Zope/lib/python' on your pythonpath (such as through the
    environment variable PYTHONPATH) yourself somehow I've found things work
    fine, and the advantage is that the tests become very straightforward
    to read and maintain.

    One thing I've noticed is that to test many modules (i.e. in ParsedXML) you
    need to import ZODB to make Persistence work, even though the testing
    code does not directly refer to Persistence. That's not a testing
    guideline; perhaps there's a better way, just an observation for folks
    writing unit tests.

  Update on path manipulation

    It turns out sys.path manipulation may be okay under some 
    circumstances.

    Example code that triggered the discussion::

      if __name__=='__main__':
          import sys
          sys.path.insert(0, '../../..')
          sys.path.insert(0, '..')
    
    The question was: "I thought doing things like this were frowned upon now?"
  
    The key is the condition "__name__ == '__main__':  'sys.path' only gets
    munged if that module is run as a standalone script, e.g.::

      $ cd lib/python/Products/FooProduct/tests
      $ python checkSomeGnarlyFeature.py

    For "regression tests", like the multithread tests ChrisM  
    put in a separate module yesterday, assuming that the test
    will be run directly from the command line is fine.  ![In fact, it's
    a feature.]

    If, however, the test is a unit test, then the second path
    insertion is likely to mean that test will fail when run via
    'testrunner', as the parent package won't be in the path
    at that point.  In such a case, the appropriate fix is to
    do imports from fully-qualified package/module names.