History for DoctestIdeas

??changed:
-
<ul>
<li>Review (already checked in to Python)
    <p>
    Current Z3 changes
        <ul>
        <li>cleanup of !DocTestUnit - DONE
        <li>Setup/teardown - DONE
        <li>!DocTestFile (could/should be done better; signature is off) - DONE
        </ul>

<li>New doctest gimmicks >= 2.3 need documentation.

<li>postmortem debugging - DONE
    <p>
    produce

    <pre>
    code
    __compare_output()
    </pre>

    as one code block, so the code is still active if the actual
    output doesn't match.

    <p>
    Scratch that -- too difficult, for "technical reasons".
    Instead a debugging-aware "the output is different" reporter needs access to
    the globals the failing example ran in.  It can exec a statement that raises
    an exception, using those globals as the exec's context.  Then the debugger
    will have access to all the bindings the failing example used.


<li>set_trace - DONE
    <p>
    monkey-patch pdb.set_trace, to first restore stdout

<li>Pass globals to !DocTestSuite - DONE
    <ul>
    <li>globs + extraglobs - DONE
    <li>augment/override flag? 
    <li>make copy of globals (or module globals if globs is None);
        "merge" extraglobs if given - DONE
    <li>The intent of extraglobs is to make a doctest somewhat parameterizable,
        so that the same test can be invoked multiple times with different bindings for
        some of its globals.  For example, a doctest written for a base class may be
        usable for (partially) testing subclasses too, if the test uses a generic
        name for the class being tested and this name is bound to the actual class
        being tested via extraglobs.
    </ul>

<li>Refactor doctest - DONE
    <ul>
    <li>make more pluggable. <p>
           Should this be done with callbacks, or by defining a class that can be subclassed, with methods overridden? <i>-edloper</i>
            <p>DONE, via subclassing.
        <ul>
        <li>comparison - DONE
        <li>output - DONE
        <li>which objects to test
            <p>
            Unclear what this is really about.  There's the now-deprecated "private name" gimmick.
            Filtering a !DocTestFinder list seems better than building in obscure options.
        <li>callbacks for
            <ul>
            <li>comparison failed - DONE
            <li>unexpected exception - DONE
            </ul>
        </ul>
    <li>get rid of umpteen module-level functions with 15 args each - DONE
    <li>use common code to find things to test (why did Jim write his own?) - DONE
    <li>the "private name" idea was/is stupid, and should go away - DONE as far as possible for 2.4, by
        raising !DeprecationWarning; can't go away until 2.5 at earliest
    </ul>

<li>blank line gimmick - DONE, via &lt;BLANKLINE&gt; markup, enabled by default

<li>multiline traceback message - DONE, multi-line exception detail works now
    <p>
    doctest insists on 1-line msg now
    <p>
    (does this mean that I no longer will have my doctests cluttered with "Traceback (most recent call last):"?)
    <ul><li>
    I don't think so -- I'm assuming that this refers to the fact that doctest 
    can't currently deal with things like "raise ValueError('x\ny\nz')", where the
    exception message (value) spans multiple lines.  But if there's demand, we
    might be able to add an additional string to signal an exception (we already
    accept both "most recent call last" and "innermost last" variants of the 
    Traceback line).) <i>-edloper</i>
    </ul>

<li>better diff output when test fails - DONE, context and unified diffs are options now.
    <p>
    esp./only multi-line expected output; it can be very hard to see what's different
    (same thing w/ unittest's assertEqual).

<li>elided parts - DONE, via ELLIPSIS option
    <p>
    generalization of traceback junk-skipping

<li>pretty printer
    <p>
    Unclear there's a strong need for this anymore.  The new NORMALIZE_WHITESPACE comparison option
    goes a long way toward meeting the use cases I had (like showing expected output for
    range(1000) on a 100 lines instead of 1 line).

</ul>

From ManuelG Thu Jul 22 19:30:00 US/Eastern 2004
From: ManuelG
Date: 2004/07/22 19:30 EST
Subject: 
Message-ID: <[email protected]>


<ul>

<li>currently if you assign a shorter name, i.e., Punter = NewImprovedPrinterPunter, NewImprovedPrinterPunter's test get run twice

<li>doctest and unittest work well together, I make passing doctest one of tests in unittest, sometimes the only test is checking doctest strings
    <p>
    when a doctest is starting to look hairy, I usually have a bunch of tests that could efficiently use some ad-hoc framework, and the perfect place for
    that kind of code is in a unittest test

<li>temporary debugging print messages confuse doctest
    <p>
    it would be nice if I could put a prefix to all my temp debug print strings, and doctest would just ignore those lines anywhere they might pop up

</ul>

From Zen Sun Jul 25 18:42:00 US/Eastern 2004
From: Zen
Date: 2004/07/25 18:42 EST
Subject: 
Message-ID: <[email protected]>

- Fix unittest.main to accept a TestSuite as an argument allowing DocTestSuite and TestCase instances to coexist better in the one file.

- Create a DocTestSuite from a text file, allowing us to easily test examples in documentation (currently doing ``def test(): pass; test.__doc__ = open('README.txt').read()``)


From edloper Thu Jul 29 12:54:00 US/Eastern 2004
From: edloper
Date: 2004/07/29 12:54 EST
Subject: Walkthrough for Current Implementation
Message-ID: <[email protected]>

While doing a thourough read-through of the code, I wrote up a walkthrough for doctest.py.  It should be useful for anyone else who wants to start looking at the code, and/or as a reference to the code.  It's at <a href="http://zope.org/Members/edloper/Doctest136ImplementationSummary">Doctest136ImplementationSummary</a>.

From edloper Fri Jul 30 16:11:00 US/Eastern 2004
From: edloper
Date: 2004/07/30 16:11 EST
Subject: SpoofOut vs StringIO
Message-ID: <[email protected]>

Is there a good reason to define a new _SpoofOut class, rather than just using StringIO?

From edloper Fri Jul 30 18:44:00 US/Eastern 2004
From: edloper
Date: 2004/07/30 18:44 EST
Subject: hiding imported objects
Message-ID: <[email protected]>

doctest.py goes to great lengths to make sure that imported modules and objects don't appear in its namespace, or that they're named with leading underscores.  E.g., the "re" module is deleted after it's used; _run_examples_inner does its imports within the function body; and imported objects are renamed (via "as") to begin with a leading underscore.

Is this useful, given that the module also defines __all__?


From edloper Fri Jul 30 20:33:00 US/Eastern 2004
From: edloper
Date: 2004/07/30 20:33 EST
Subject: extraglobs
Message-ID: <[email protected]>

What's the motivation for giving an extra set of globals that should be merged in (extraglobs)?  (Under "Pass globals to DocTestSuite, above.)

From tim_one Mon Aug 2 15:19:00 US/Eastern 2004
From: tim_one
Date: 2004/08/02 15:19 EST
Subject: hiding imported objects
Message-ID: <[email protected]>
In-reply-to: <[email protected]>

doctest existed before __all__ did; that's why it goes to such pain.  As you say, not necessary any more.

From tim_one Mon Aug 2 15:46:00 US/Eastern 2004
From: tim_one
Date: 2004/08/02 15:46 EST
Subject: extraglobs
Message-ID: <[email protected]>
In-reply-to: <[email protected]>

Jim wants a way to "parameterize" doctests.  For example, write a doctest for a base class, and then reuse it verbatim to test any number of subclasses.  The doctest would use a generic name for the actual class getting tested, and extraglobs would map the generic name to the specific class to be tested.

From tim_one Mon Aug 2 15:54:00 US/Eastern 2004
From: tim_one
Date: 2004/08/02 15:54 EST
Subject: SpoofOut vs StringIO
Message-ID: <[email protected]>
In-reply-to: <[email protected]>
[55 more lines...]