Log in
Forgot your password?
History for Doctest 1.36 Implementation Summary
??changed:- ================================================== doctest.py: Implementation Summary (revision 1.36) ================================================== This section gives a summary of the current design. I wrote it while reading through the code, to make sure I understood what all the pieces did (and to give me a reference to check back with); but it might be a useful guide to anyone who's trying to familiarize themselves more with the current code. Entry points ============ Doctest cases are run by either `run_doctest_examples()` or `Tester.runstring()`. (There are other entry points, such as `Tester.rundoc()`, but they all call into either `run_doctest_examples()` or `Tester.runstring()`; see below for more info on other entry points.) Each of these routines calls `_extract_examples()` to pull examples out of the docstring, and then calls `_run_examples()` to run them. They both return a tuple `(num_failures, num_doctest_cases)` (which they get as the return value from `_run_examples()`). `Tester.runstring()` also keeps a running total of `num_failures` and `num_doctest_cases` in instance variables. Since `run_doctest_examples()` is pulls the docstring out of an object, it needs to do a little more work. In particular, it checks for errors & casts to str while pulling out the docstring (is this unicode-unfriendly?); and it gets compiler flags by checking __future__ flags from globals. _extract_examples ================= `_extract_examples(s)` takes a string containing a doctest case, and returns a list of those examples. Each doctest case is encoded as a tripple `(source, outcome, lineno)`, where: - `source` is the source code string. It ends with a newline iff `source` spans multiple lines. - `outcome` is the expected output, if any, or an empty string if no output is expected. `outcome` ends in a newline iff output is expected - `lineno` is the line number of the first line of the doctest case within the input string. This line number is zero-based. `_extract_examples()` is basically implemented as an ad-hoc parser that loops through the lines of the docstring. _run_examples ============= `_run_examples()` is basically a wrapper for `_run_examples_inner()`. Here's what it does: - Redirect stdout to a dummy object (`_SpoofOut`) that captures output. - Make a shallow copy of the gloals. - Call `_run_examples_inner()` to do the real work. After `_run_examples_inner()` returns, `_run_examples()` restores `sys.stdout`; clears the copy of globals (for gc reasons); and returns what `_run_examples_inner()` returned (i.e., a tuple `(num_failures, num_doctest_cases)`). _run_examples_inner =================== This is where the real work happens. It takes a bunch of parameters: - `out`: A function for writing output. Currently, this is set to `sys.stdout.write` by `_run_examples()`. - `fakeout`: The dummy output object (`_SpoofOut`), used to capture the output of each doctest case. - `examples`: A list of `(source, outcome, lineno)` tripples, as returned by `_extract_examples()`. - `globs`: The set of globals used as an environment to evaluate doctest cases. - `verbose`: If true, then print a message to `out` before running each example, indicating what example is being run and what the expected output is. - `name`: The "name" of the collection of examples (typically the name of the object that `_extract_examples()` was run on). This is used in the output to `out`. - `compileflags`: A set of compiler flags for the builtin `compile()` function. This is used to correctly handle `__future__` imports. - `optionflags`: A set of flags for options to doctest itself. Currently, only one flag is supported: `DONT_ACCEPT_TRUE_FOR_1`. `_run_examples_inner()` loops through each doctest example, compiles it, and runs it (with output directed to `fakeout`). It then extracts the output, and comparse it to the expected output. If they differ, it calls `out` with an error message. Exceptions are handled separately: if an exception is raised, then the expected output is compared against a traceback template; if they match, and the exception type/message match, then the test passes; otherwise, it fails. All examples are evaluated in the same globals environment, so variables can be shared between the doctest cases in `examples` (e.g., one doctest case can create a variable, and a later doctest case can use it). Once all examples have been run, `_run_examples_inner()` returns a tuple `(num_failures, num_doctest_cases)`. Other Entry Points ================== The `Tester` class defines a number of entry points: `Tester.rundoc()`, `Tester.rundict()`, and `Tester.run__test__()`. Also, the function `testmod` creates a `Tester` class and uses it to test every object in a module. Finally, there are some entry points to make it easier to interface with the `unittest` module. Tester.rundoc() --------------- `Tester.rundoc(obj)` calls `run_doctest_examples` to extract and process the doctest cases in the given object's docstring. If `obj` is a class, then `Tester.rundoc` puts together a dictionary mapping each attributes name to an object with the right docstrings. (It needs to do some acrobatics to get the right objects when wrappers such as `staticmethod` and `classmethod` are used.) It then calls `Tester.run__test__()` on the dictionary, and merges the return value into the return value from `run_doctest_examples` (both return a `(num_failures, num_doctest_cases)` tuple). Note that running `Tester.rundoc()` on a class will recurse into contained objects; but calling it on a module will *not* recurse into contained objects. Tester.rundict() and Tester.run__test__() ----------------------------------------- Both of these functions basically take a dictionary from names to objects, and calls `rundoc` and/or `runstring` on those objects. There are a number of annoying and useless differences between the two (eg one uses the variable `thisname` for the dictionary key; the other uses `thisname` for the dictionary key *plus* a given prefix). But there are also a couple of real differences: - `rundict` uses `isprivate` to ignore anything with a "private" name; but `run__test__` looks at every dictionary item. - `rundict` excludes any object that is not defined in a given module. (This is to avoid running doctests on imported objects.) - `rundict` ignores everything but functions and classes; but `run__test__` also processes strings. Also, if you give `run__test__` anything *other* than a function, class, or string, then it raises an exception. I don't see any reason why these two have to duplicate code (and do things differently); I think they could be folded into a single function pretty easily, which would reduce the number of places for bugs to hide. testmod ------- `testmod()` is one of the more common entry points for doctest. It basically just creates a Tester object and uses it to run the tests in every docstring in a given module. It takes the following arguments: - `m`: The module to test (use sys.modules['__main__'] if not specified). - `name`: The name of the module (use `m.__name__` if not specified). - `globs`: The set of globals to use (use `m.__dict__` if not specified). - `verbose`: Verbosity level for the `Tester` object. - `isprivate`: Private-detection function for the `Tester` object. - `report`: If true, then print a report at the end. - `optionflags`: flags for the `Tester` object (passed through to `_run_examples_inner`). It calls `Tester.rundict` on the module's `__dict__`; and if the module defines a `__test__` attribute, then it `Tester.run__test__` on it. It also creates a global "master" trainer, which it uses to keep track of the total number of errors that have been generated across multiple calls to `testmod()`. Like most of the other entry points, it returns a tuple `(num_failures, num_doctest_cases)`. Utility Functions and Classes ============================= _SpoofOut --------- :Used by: `_run_examples()` and `_run_examples_inner()` (to replace `sys.stdout` and as a `writer` for `traceback.print_exc`). This is a fairly streight-forward object that acts like a file open for writing, and captures any output into a string. It's used to capture output when running doctest cases, so it can be compared against the expected output. If the captured output doesn't already end with a newline, then `_SpoofOut.get()` adds one. _tag_out -------- [72 more lines...]