diff options
author | Georg Brandl <georg@python.org> | 2007-08-15 14:27:07 (GMT) |
---|---|---|
committer | Georg Brandl <georg@python.org> | 2007-08-15 14:27:07 (GMT) |
commit | 739c01d47b9118d04e5722333f0e6b4d0c8bdd9e (patch) | |
tree | f82b450d291927fc1758b96d981aa0610947b529 /Doc/lib/libunittest.tex | |
parent | 2d1649094402ef393ea2b128ba2c08c3937e6b93 (diff) | |
download | cpython-739c01d47b9118d04e5722333f0e6b4d0c8bdd9e.zip cpython-739c01d47b9118d04e5722333f0e6b4d0c8bdd9e.tar.gz cpython-739c01d47b9118d04e5722333f0e6b4d0c8bdd9e.tar.bz2 |
Delete the LaTeX doc tree.
Diffstat (limited to 'Doc/lib/libunittest.tex')
-rw-r--r-- | Doc/lib/libunittest.tex | 978 |
1 files changed, 0 insertions, 978 deletions
diff --git a/Doc/lib/libunittest.tex b/Doc/lib/libunittest.tex deleted file mode 100644 index fa198c7..0000000 --- a/Doc/lib/libunittest.tex +++ /dev/null @@ -1,978 +0,0 @@ -\section{\module{unittest} --- - Unit testing framework} - -\declaremodule{standard}{unittest} -\modulesynopsis{Unit testing framework for Python.} -\moduleauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com} -\sectionauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com} -\sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org} -\sectionauthor{Raymond Hettinger}{python@rcn.com} - -\versionadded{2.1} - -The Python unit testing framework, sometimes referred to as ``PyUnit,'' is -a Python language version of JUnit, by Kent Beck and Erich Gamma. -JUnit is, in turn, a Java version of Kent's Smalltalk testing -framework. Each is the de facto standard unit testing framework for -its respective language. - -\module{unittest} supports test automation, sharing of setup and shutdown -code for tests, aggregation of tests into collections, and independence of -the tests from the reporting framework. The \module{unittest} module -provides classes that make it easy to support these qualities for a -set of tests. - -To achieve this, \module{unittest} supports some important concepts: - -\begin{definitions} -\term{test fixture} -A \dfn{test fixture} represents the preparation needed to perform one -or more tests, and any associate cleanup actions. This may involve, -for example, creating temporary or proxy databases, directories, or -starting a server process. - -\term{test case} -A \dfn{test case} is the smallest unit of testing. It checks for a -specific response to a particular set of inputs. \module{unittest} -provides a base class, \class{TestCase}, which may be used to create -new test cases. - -\term{test suite} -A \dfn{test suite} is a collection of test cases, test suites, or -both. It is used to aggregate tests that should be executed -together. - -\term{test runner} -A \dfn{test runner} is a component which orchestrates the execution of -tests and provides the outcome to the user. The runner may use a -graphical interface, a textual interface, or return a special value to -indicate the results of executing the tests. -\end{definitions} - - -The test case and test fixture concepts are supported through the -\class{TestCase} and \class{FunctionTestCase} classes; the former -should be used when creating new tests, and the latter can be used when -integrating existing test code with a \module{unittest}-driven framework. -When building test fixtures using \class{TestCase}, the \method{setUp()} -and \method{tearDown()} methods can be overridden to provide -initialization and cleanup for the fixture. With -\class{FunctionTestCase}, existing functions can be passed to the -constructor for these purposes. When the test is run, the -fixture initialization is run first; if it succeeds, the cleanup -method is run after the test has been executed, regardless of the -outcome of the test. Each instance of the \class{TestCase} will only -be used to run a single test method, so a new fixture is created for -each test. - -Test suites are implemented by the \class{TestSuite} class. This -class allows individual tests and test suites to be aggregated; when -the suite is executed, all tests added directly to the suite and in -``child'' test suites are run. - -A test runner is an object that provides a single method, -\method{run()}, which accepts a \class{TestCase} or \class{TestSuite} -object as a parameter, and returns a result object. The class -\class{TestResult} is provided for use as the result object. -\module{unittest} provides the \class{TextTestRunner} as an example -test runner which reports test results on the standard error stream by -default. Alternate runners can be implemented for other environments -(such as graphical environments) without any need to derive from a -specific class. - - -\begin{seealso} - \seemodule{doctest}{Another test-support module with a very - different flavor.} - \seetitle[http://www.XProgramming.com/testfram.htm]{Simple Smalltalk - Testing: With Patterns}{Kent Beck's original paper on - testing frameworks using the pattern shared by - \module{unittest}.} -\end{seealso} - - -\subsection{Basic example \label{unittest-minimal-example}} - -The \module{unittest} module provides a rich set of tools for -constructing and running tests. This section demonstrates that a -small subset of the tools suffice to meet the needs of most users. - -Here is a short script to test three functions from the -\refmodule{random} module: - -\begin{verbatim} -import random -import unittest - -class TestSequenceFunctions(unittest.TestCase): - - def setUp(self): - self.seq = range(10) - - def testshuffle(self): - # make sure the shuffled sequence does not lose any elements - random.shuffle(self.seq) - self.seq.sort() - self.assertEqual(self.seq, range(10)) - - def testchoice(self): - element = random.choice(self.seq) - self.assert_(element in self.seq) - - def testsample(self): - self.assertRaises(ValueError, random.sample, self.seq, 20) - for element in random.sample(self.seq, 5): - self.assert_(element in self.seq) - -if __name__ == '__main__': - unittest.main() -\end{verbatim} - -A testcase is created by subclassing \class{unittest.TestCase}. -The three individual tests are defined with methods whose names start with -the letters \samp{test}. This naming convention informs the test runner -about which methods represent tests. - -The crux of each test is a call to \method{assertEqual()} to -check for an expected result; \method{assert_()} to verify a condition; -or \method{assertRaises()} to verify that an expected exception gets -raised. These methods are used instead of the \keyword{assert} statement -so the test runner can accumulate all test results and produce a report. - -When a \method{setUp()} method is defined, the test runner will run that -method prior to each test. Likewise, if a \method{tearDown()} method is -defined, the test runner will invoke that method after each test. In the -example, \method{setUp()} was used to create a fresh sequence for each test. - -The final block shows a simple way to run the tests. -\function{unittest.main()} provides a command line interface to the -test script. When run from the command line, the above script -produces an output that looks like this: - -\begin{verbatim} -... ----------------------------------------------------------------------- -Ran 3 tests in 0.000s - -OK -\end{verbatim} - -Instead of \function{unittest.main()}, there are other ways to run the tests -with a finer level of control, less terse output, and no requirement to be -run from the command line. For example, the last two lines may be replaced -with: - -\begin{verbatim} -suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions) -unittest.TextTestRunner(verbosity=2).run(suite) -\end{verbatim} - -Running the revised script from the interpreter or another script -produces the following output: - -\begin{verbatim} -testchoice (__main__.TestSequenceFunctions) ... ok -testsample (__main__.TestSequenceFunctions) ... ok -testshuffle (__main__.TestSequenceFunctions) ... ok - ----------------------------------------------------------------------- -Ran 3 tests in 0.110s - -OK -\end{verbatim} - -The above examples show the most commonly used \module{unittest} features -which are sufficient to meet many everyday testing needs. The remainder -of the documentation explores the full feature set from first principles. - - -\subsection{Organizing test code - \label{organizing-tests}} - -The basic building blocks of unit testing are \dfn{test cases} --- -single scenarios that must be set up and checked for correctness. In -\module{unittest}, test cases are represented by instances of -\module{unittest}'s \class{TestCase} class. To make -your own test cases you must write subclasses of \class{TestCase}, or -use \class{FunctionTestCase}. - -An instance of a \class{TestCase}-derived class is an object that can -completely run a single test method, together with optional set-up -and tidy-up code. - -The testing code of a \class{TestCase} instance should be entirely -self contained, such that it can be run either in isolation or in -arbitrary combination with any number of other test cases. - -The simplest \class{TestCase} subclass will simply override the -\method{runTest()} method in order to perform specific testing code: - -\begin{verbatim} -import unittest - -class DefaultWidgetSizeTestCase(unittest.TestCase): - def runTest(self): - widget = Widget('The widget') - self.assertEqual(widget.size(), (50, 50), 'incorrect default size') -\end{verbatim} - -Note that in order to test something, we use the one of the -\method{assert*()} or \method{fail*()} methods provided by the -\class{TestCase} base class. If the test fails, an exception will be -raised, and \module{unittest} will identify the test case as a -\dfn{failure}. Any other exceptions will be treated as \dfn{errors}. -This helps you identify where the problem is: \dfn{failures} are caused by -incorrect results - a 5 where you expected a 6. \dfn{Errors} are caused by -incorrect code - e.g., a \exception{TypeError} caused by an incorrect -function call. - -The way to run a test case will be described later. For now, note -that to construct an instance of such a test case, we call its -constructor without arguments: - -\begin{verbatim} -testCase = DefaultWidgetSizeTestCase() -\end{verbatim} - -Now, such test cases can be numerous, and their set-up can be -repetitive. In the above case, constructing a \class{Widget} in each of -100 Widget test case subclasses would mean unsightly duplication. - -Luckily, we can factor out such set-up code by implementing a method -called \method{setUp()}, which the testing framework will -automatically call for us when we run the test: - -\begin{verbatim} -import unittest - -class SimpleWidgetTestCase(unittest.TestCase): - def setUp(self): - self.widget = Widget('The widget') - -class DefaultWidgetSizeTestCase(SimpleWidgetTestCase): - def runTest(self): - self.failUnless(self.widget.size() == (50,50), - 'incorrect default size') - -class WidgetResizeTestCase(SimpleWidgetTestCase): - def runTest(self): - self.widget.resize(100,150) - self.failUnless(self.widget.size() == (100,150), - 'wrong size after resize') -\end{verbatim} - -If the \method{setUp()} method raises an exception while the test is -running, the framework will consider the test to have suffered an -error, and the \method{runTest()} method will not be executed. - -Similarly, we can provide a \method{tearDown()} method that tidies up -after the \method{runTest()} method has been run: - -\begin{verbatim} -import unittest - -class SimpleWidgetTestCase(unittest.TestCase): - def setUp(self): - self.widget = Widget('The widget') - - def tearDown(self): - self.widget.dispose() - self.widget = None -\end{verbatim} - -If \method{setUp()} succeeded, the \method{tearDown()} method will be -run whether \method{runTest()} succeeded or not. - -Such a working environment for the testing code is called a -\dfn{fixture}. - -Often, many small test cases will use the same fixture. In this case, -we would end up subclassing \class{SimpleWidgetTestCase} into many -small one-method classes such as -\class{DefaultWidgetSizeTestCase}. This is time-consuming and - -discouraging, so in the same vein as JUnit, \module{unittest} provides -a simpler mechanism: - -\begin{verbatim} -import unittest - -class WidgetTestCase(unittest.TestCase): - def setUp(self): - self.widget = Widget('The widget') - - def tearDown(self): - self.widget.dispose() - self.widget = None - - def testDefaultSize(self): - self.failUnless(self.widget.size() == (50,50), - 'incorrect default size') - - def testResize(self): - self.widget.resize(100,150) - self.failUnless(self.widget.size() == (100,150), - 'wrong size after resize') -\end{verbatim} - -Here we have not provided a \method{runTest()} method, but have -instead provided two different test methods. Class instances will now -each run one of the \method{test*()} methods, with \code{self.widget} -created and destroyed separately for each instance. When creating an -instance we must specify the test method it is to run. We do this by -passing the method name in the constructor: - -\begin{verbatim} -defaultSizeTestCase = WidgetTestCase('testDefaultSize') -resizeTestCase = WidgetTestCase('testResize') -\end{verbatim} - -Test case instances are grouped together according to the features -they test. \module{unittest} provides a mechanism for this: the -\dfn{test suite}, represented by \module{unittest}'s \class{TestSuite} -class: - -\begin{verbatim} -widgetTestSuite = unittest.TestSuite() -widgetTestSuite.addTest(WidgetTestCase('testDefaultSize')) -widgetTestSuite.addTest(WidgetTestCase('testResize')) -\end{verbatim} - -For the ease of running tests, as we will see later, it is a good -idea to provide in each test module a callable object that returns a -pre-built test suite: - -\begin{verbatim} -def suite(): - suite = unittest.TestSuite() - suite.addTest(WidgetTestCase('testDefaultSize')) - suite.addTest(WidgetTestCase('testResize')) - return suite -\end{verbatim} - -or even: - -\begin{verbatim} -def suite(): - tests = ['testDefaultSize', 'testResize'] - - return unittest.TestSuite(map(WidgetTestCase, tests)) -\end{verbatim} - -Since it is a common pattern to create a \class{TestCase} subclass -with many similarly named test functions, \module{unittest} provides a -\class{TestLoader} class that can be used to automate the process of -creating a test suite and populating it with individual tests. -For example, - -\begin{verbatim} -suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase) -\end{verbatim} - -will create a test suite that will run -\code{WidgetTestCase.testDefaultSize()} and \code{WidgetTestCase.testResize}. -\class{TestLoader} uses the \code{'test'} method name prefix to identify -test methods automatically. - -Note that the order in which the various test cases will be run is -determined by sorting the test function names with the built-in -\function{cmp()} function. - -Often it is desirable to group suites of test cases together, so as to -run tests for the whole system at once. This is easy, since -\class{TestSuite} instances can be added to a \class{TestSuite} just -as \class{TestCase} instances can be added to a \class{TestSuite}: - -\begin{verbatim} -suite1 = module1.TheTestSuite() -suite2 = module2.TheTestSuite() -alltests = unittest.TestSuite([suite1, suite2]) -\end{verbatim} - -You can place the definitions of test cases and test suites in the -same modules as the code they are to test (such as \file{widget.py}), -but there are several advantages to placing the test code in a -separate module, such as \file{test_widget.py}: - -\begin{itemize} - \item The test module can be run standalone from the command line. - \item The test code can more easily be separated from shipped code. - \item There is less temptation to change test code to fit the code - it tests without a good reason. - \item Test code should be modified much less frequently than the - code it tests. - \item Tested code can be refactored more easily. - \item Tests for modules written in C must be in separate modules - anyway, so why not be consistent? - \item If the testing strategy changes, there is no need to change - the source code. -\end{itemize} - - -\subsection{Re-using old test code - \label{legacy-unit-tests}} - -Some users will find that they have existing test code that they would -like to run from \module{unittest}, without converting every old test -function to a \class{TestCase} subclass. - -For this reason, \module{unittest} provides a \class{FunctionTestCase} -class. This subclass of \class{TestCase} can be used to wrap an existing -test function. Set-up and tear-down functions can also be provided. - -Given the following test function: - -\begin{verbatim} -def testSomething(): - something = makeSomething() - assert something.name is not None - # ... -\end{verbatim} - -one can create an equivalent test case instance as follows: - -\begin{verbatim} -testcase = unittest.FunctionTestCase(testSomething) -\end{verbatim} - -If there are additional set-up and tear-down methods that should be -called as part of the test case's operation, they can also be provided -like so: - -\begin{verbatim} -testcase = unittest.FunctionTestCase(testSomething, - setUp=makeSomethingDB, - tearDown=deleteSomethingDB) -\end{verbatim} - -To make migrating existing test suites easier, \module{unittest} -supports tests raising \exception{AssertionError} to indicate test failure. -However, it is recommended that you use the explicit -\method{TestCase.fail*()} and \method{TestCase.assert*()} methods instead, -as future versions of \module{unittest} may treat \exception{AssertionError} -differently. - -\note{Even though \class{FunctionTestCase} can be used to quickly convert -an existing test base over to a \module{unittest}-based system, this -approach is not recommended. Taking the time to set up proper -\class{TestCase} subclasses will make future test refactorings infinitely -easier.} - - - -\subsection{Classes and functions - \label{unittest-contents}} - -\begin{classdesc}{TestCase}{\optional{methodName}} - Instances of the \class{TestCase} class represent the smallest - testable units in the \module{unittest} universe. This class is - intended to be used as a base class, with specific tests being - implemented by concrete subclasses. This class implements the - interface needed by the test runner to allow it to drive the - test, and methods that the test code can use to check for and - report various kinds of failure. - - Each instance of \class{TestCase} will run a single test method: - the method named \var{methodName}. If you remember, we had an - earlier example that went something like this: - - \begin{verbatim} - def suite(): - suite = unittest.TestSuite() - suite.addTest(WidgetTestCase('testDefaultSize')) - suite.addTest(WidgetTestCase('testResize')) - return suite - \end{verbatim} - - Here, we create two instances of \class{WidgetTestCase}, each of - which runs a single test. - - \var{methodName} defaults to \code{'runTest'}. -\end{classdesc} - -\begin{classdesc}{FunctionTestCase}{testFunc\optional{, - setUp\optional{, tearDown\optional{, description}}}} - This class implements the portion of the \class{TestCase} interface - which allows the test runner to drive the test, but does not provide - the methods which test code can use to check and report errors. - This is used to create test cases using legacy test code, allowing - it to be integrated into a \refmodule{unittest}-based test - framework. -\end{classdesc} - -\begin{classdesc}{TestSuite}{\optional{tests}} - This class represents an aggregation of individual tests cases and - test suites. The class presents the interface needed by the test - runner to allow it to be run as any other test case. Running a - \class{TestSuite} instance is the same as iterating over the suite, - running each test individually. - - If \var{tests} is given, it must be an iterable of individual test cases or - other test suites that will be used to build the suite initially. - Additional methods are provided to add test cases and suites to the - collection later on. -\end{classdesc} - -\begin{classdesc}{TestLoader}{} - This class is responsible for loading tests according to various - criteria and returning them wrapped in a \class{TestSuite}. - It can load all tests within a given module or \class{TestCase} - subclass. -\end{classdesc} - -\begin{classdesc}{TestResult}{} - This class is used to compile information about which tests have succeeded - and which have failed. -\end{classdesc} - -\begin{datadesc}{defaultTestLoader} - Instance of the \class{TestLoader} class intended to be shared. If no - customization of the \class{TestLoader} is needed, this instance can - be used instead of repeatedly creating new instances. -\end{datadesc} - -\begin{classdesc}{TextTestRunner}{\optional{stream\optional{, - descriptions\optional{, verbosity}}}} - A basic test runner implementation which prints results on standard - error. It has a few configurable parameters, but is essentially - very simple. Graphical applications which run test suites should - provide alternate implementations. -\end{classdesc} - -\begin{funcdesc}{main}{\optional{module\optional{, - defaultTest\optional{, argv\optional{, - testRunner\optional{, testLoader}}}}}} - A command-line program that runs a set of tests; this is primarily - for making test modules conveniently executable. The simplest use - for this function is to include the following line at the end of a - test script: - -\begin{verbatim} -if __name__ == '__main__': - unittest.main() -\end{verbatim} - - The \var{testRunner} argument can either be a test runner class or - an already created instance of it. -\end{funcdesc} - -In some cases, the existing tests may have been written using the -\refmodule{doctest} module. If so, that module provides a -\class{DocTestSuite} class that can automatically build -\class{unittest.TestSuite} instances from the existing -\module{doctest}-based tests. -\versionadded{2.3} - - -\subsection{TestCase Objects - \label{testcase-objects}} - -Each \class{TestCase} instance represents a single test, but each -concrete subclass may be used to define multiple tests --- the -concrete class represents a single test fixture. The fixture is -created and cleaned up for each test case. - -\class{TestCase} instances provide three groups of methods: one group -used to run the test, another used by the test implementation to -check conditions and report failures, and some inquiry methods -allowing information about the test itself to be gathered. - -Methods in the first group (running the test) are: - -\begin{methoddesc}[TestCase]{setUp}{} - Method called to prepare the test fixture. This is called - immediately before calling the test method; any exception raised by - this method will be considered an error rather than a test failure. - The default implementation does nothing. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{tearDown}{} - Method called immediately after the test method has been called and - the result recorded. This is called even if the test method raised - an exception, so the implementation in subclasses may need to be - particularly careful about checking internal state. Any exception - raised by this method will be considered an error rather than a test - failure. This method will only be called if the \method{setUp()} - succeeds, regardless of the outcome of the test method. - The default implementation does nothing. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{run}{\optional{result}} - Run the test, collecting the result into the test result object - passed as \var{result}. If \var{result} is omitted or \constant{None}, - a temporary result object is created (by calling the - \method{defaultTestCase()} method) and used; this result object is not - returned to \method{run()}'s caller. - - The same effect may be had by simply calling the \class{TestCase} - instance. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{debug}{} - Run the test without collecting the result. This allows exceptions - raised by the test to be propagated to the caller, and can be used - to support running tests under a debugger. -\end{methoddesc} - - -The test code can use any of the following methods to check for and -report failures. - -\begin{methoddesc}[TestCase]{assert_}{expr\optional{, msg}} -\methodline[TestCase]{failUnless}{expr\optional{, msg}} - Signal a test failure if \var{expr} is false; the explanation for - the error will be \var{msg} if given, otherwise it will be - \constant{None}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{assertEqual}{first, second\optional{, msg}} -\methodline[TestCase]{failUnlessEqual}{first, second\optional{, msg}} - Test that \var{first} and \var{second} are equal. If the values do - not compare equal, the test will fail with the explanation given by - \var{msg}, or \constant{None}. Note that using \method{failUnlessEqual()} - improves upon doing the comparison as the first parameter to - \method{failUnless()}: the default value for \var{msg} can be - computed to include representations of both \var{first} and - \var{second}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{assertNotEqual}{first, second\optional{, msg}} -\methodline[TestCase]{failIfEqual}{first, second\optional{, msg}} - Test that \var{first} and \var{second} are not equal. If the values - do compare equal, the test will fail with the explanation given by - \var{msg}, or \constant{None}. Note that using \method{failIfEqual()} - improves upon doing the comparison as the first parameter to - \method{failUnless()} is that the default value for \var{msg} can be - computed to include representations of both \var{first} and - \var{second}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{assertAlmostEqual}{first, second\optional{, - places\optional{, msg}}} -\methodline[TestCase]{failUnlessAlmostEqual}{first, second\optional{, - places\optional{, msg}}} - Test that \var{first} and \var{second} are approximately equal - by computing the difference, rounding to the given number of \var{places}, - and comparing to zero. Note that comparing a given number of decimal places - is not the same as comparing a given number of significant digits. - If the values do not compare equal, the test will fail with the explanation - given by \var{msg}, or \constant{None}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{assertNotAlmostEqual}{first, second\optional{, - places\optional{, msg}}} -\methodline[TestCase]{failIfAlmostEqual}{first, second\optional{, - places\optional{, msg}}} - Test that \var{first} and \var{second} are not approximately equal - by computing the difference, rounding to the given number of \var{places}, - and comparing to zero. Note that comparing a given number of decimal places - is not the same as comparing a given number of significant digits. - If the values do not compare equal, the test will fail with the explanation - given by \var{msg}, or \constant{None}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{assertRaises}{exception, callable, \moreargs} -\methodline[TestCase]{failUnlessRaises}{exception, callable, \moreargs} - Test that an exception is raised when \var{callable} is called with - any positional or keyword arguments that are also passed to - \method{assertRaises()}. The test passes if \var{exception} is - raised, is an error if another exception is raised, or fails if no - exception is raised. To catch any of a group of exceptions, a tuple - containing the exception classes may be passed as \var{exception}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{failIf}{expr\optional{, msg}} - The inverse of the \method{failUnless()} method is the - \method{failIf()} method. This signals a test failure if \var{expr} - is true, with \var{msg} or \constant{None} for the error message. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{fail}{\optional{msg}} - Signals a test failure unconditionally, with \var{msg} or - \constant{None} for the error message. -\end{methoddesc} - -\begin{memberdesc}[TestCase]{failureException} - This class attribute gives the exception raised by the - \method{test()} method. If a test framework needs to use a - specialized exception, possibly to carry additional information, it - must subclass this exception in order to ``play fair'' with the - framework. The initial value of this attribute is - \exception{AssertionError}. -\end{memberdesc} - - -Testing frameworks can use the following methods to collect -information on the test: - -\begin{methoddesc}[TestCase]{countTestCases}{} - Return the number of tests represented by this test object. For - \class{TestCase} instances, this will always be \code{1}. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{defaultTestResult}{} - Return an instance of the test result class that should be used - for this test case class (if no other result instance is provided - to the \method{run()} method). - - For \class{TestCase} instances, this will always be an instance of - \class{TestResult}; subclasses of \class{TestCase} should - override this as necessary. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{id}{} - Return a string identifying the specific test case. This is usually - the full name of the test method, including the module and class - name. -\end{methoddesc} - -\begin{methoddesc}[TestCase]{shortDescription}{} - Returns a one-line description of the test, or \constant{None} if no - description has been provided. The default implementation of this - method returns the first line of the test method's docstring, if - available, or \constant{None}. -\end{methoddesc} - - -\subsection{TestSuite Objects - \label{testsuite-objects}} - -\class{TestSuite} objects behave much like \class{TestCase} objects, -except they do not actually implement a test. Instead, they are used -to aggregate tests into groups of tests that should be run together. -Some additional methods are available to add tests to \class{TestSuite} -instances: - -\begin{methoddesc}[TestSuite]{addTest}{test} - Add a \class{TestCase} or \class{TestSuite} to the suite. -\end{methoddesc} - -\begin{methoddesc}[TestSuite]{addTests}{tests} - Add all the tests from an iterable of \class{TestCase} and - \class{TestSuite} instances to this test suite. - - This is equivalent to iterating over \var{tests}, calling - \method{addTest()} for each element. -\end{methoddesc} - -\class{TestSuite} shares the following methods with \class{TestCase}: - -\begin{methoddesc}[TestSuite]{run}{result} - Run the tests associated with this suite, collecting the result into - the test result object passed as \var{result}. Note that unlike - \method{TestCase.run()}, \method{TestSuite.run()} requires the - result object to be passed in. -\end{methoddesc} - -\begin{methoddesc}[TestSuite]{debug}{} - Run the tests associated with this suite without collecting the result. - This allows exceptions raised by the test to be propagated to the caller - and can be used to support running tests under a debugger. -\end{methoddesc} - -\begin{methoddesc}[TestSuite]{countTestCases}{} - Return the number of tests represented by this test object, including - all individual tests and sub-suites. -\end{methoddesc} - -In the typical usage of a \class{TestSuite} object, the \method{run()} -method is invoked by a \class{TestRunner} rather than by the end-user -test harness. - - -\subsection{TestResult Objects - \label{testresult-objects}} - -A \class{TestResult} object stores the results of a set of tests. The -\class{TestCase} and \class{TestSuite} classes ensure that results are -properly recorded; test authors do not need to worry about recording the -outcome of tests. - -Testing frameworks built on top of \refmodule{unittest} may want -access to the \class{TestResult} object generated by running a set of -tests for reporting purposes; a \class{TestResult} instance is -returned by the \method{TestRunner.run()} method for this purpose. - -\class{TestResult} instances have the following attributes that will -be of interest when inspecting the results of running a set of tests: - -\begin{memberdesc}[TestResult]{errors} - A list containing 2-tuples of \class{TestCase} instances and - strings holding formatted tracebacks. Each tuple represents a test which - raised an unexpected exception. - \versionchanged[Contains formatted tracebacks instead of - \function{sys.exc_info()} results]{2.2} -\end{memberdesc} - -\begin{memberdesc}[TestResult]{failures} - A list containing 2-tuples of \class{TestCase} instances and strings - holding formatted tracebacks. Each tuple represents a test where a failure - was explicitly signalled using the \method{TestCase.fail*()} or - \method{TestCase.assert*()} methods. - \versionchanged[Contains formatted tracebacks instead of - \function{sys.exc_info()} results]{2.2} -\end{memberdesc} - -\begin{memberdesc}[TestResult]{testsRun} - The total number of tests run so far. -\end{memberdesc} - -\begin{methoddesc}[TestResult]{wasSuccessful}{} - Returns \constant{True} if all tests run so far have passed, - otherwise returns \constant{False}. -\end{methoddesc} - -\begin{methoddesc}[TestResult]{stop}{} - This method can be called to signal that the set of tests being run - should be aborted by setting the \class{TestResult}'s \code{shouldStop} - attribute to \constant{True}. \class{TestRunner} objects should respect - this flag and return without running any additional tests. - - For example, this feature is used by the \class{TextTestRunner} class - to stop the test framework when the user signals an interrupt from - the keyboard. Interactive tools which provide \class{TestRunner} - implementations can use this in a similar manner. -\end{methoddesc} - - -The following methods of the \class{TestResult} class are used to -maintain the internal data structures, and may be extended in -subclasses to support additional reporting requirements. This is -particularly useful in building tools which support interactive -reporting while tests are being run. - -\begin{methoddesc}[TestResult]{startTest}{test} - Called when the test case \var{test} is about to be run. - - The default implementation simply increments the instance's - \code{testsRun} counter. -\end{methoddesc} - -\begin{methoddesc}[TestResult]{stopTest}{test} - Called after the test case \var{test} has been executed, regardless - of the outcome. - - The default implementation does nothing. -\end{methoddesc} - -\begin{methoddesc}[TestResult]{addError}{test, err} - Called when the test case \var{test} raises an unexpected exception - \var{err} is a tuple of the form returned by \function{sys.exc_info()}: - \code{(\var{type}, \var{value}, \var{traceback})}. - - The default implementation appends \code{(\var{test}, \var{err})} to - the instance's \code{errors} attribute. -\end{methoddesc} - -\begin{methoddesc}[TestResult]{addFailure}{test, err} - Called when the test case \var{test} signals a failure. - \var{err} is a tuple of the form returned by - \function{sys.exc_info()}: \code{(\var{type}, \var{value}, - \var{traceback})}. - - The default implementation appends \code{(\var{test}, \var{err})} to - the instance's \code{failures} attribute. -\end{methoddesc} - -\begin{methoddesc}[TestResult]{addSuccess}{test} - Called when the test case \var{test} succeeds. - - The default implementation does nothing. -\end{methoddesc} - - - -\subsection{TestLoader Objects - \label{testloader-objects}} - -The \class{TestLoader} class is used to create test suites from -classes and modules. Normally, there is no need to create an instance -of this class; the \refmodule{unittest} module provides an instance -that can be shared as \code{unittest.defaultTestLoader}. -Using a subclass or instance, however, allows customization of some -configurable properties. - -\class{TestLoader} objects have the following methods: - -\begin{methoddesc}[TestLoader]{loadTestsFromTestCase}{testCaseClass} - Return a suite of all tests cases contained in the - \class{TestCase}-derived \class{testCaseClass}. -\end{methoddesc} - -\begin{methoddesc}[TestLoader]{loadTestsFromModule}{module} - Return a suite of all tests cases contained in the given module. - This method searches \var{module} for classes derived from - \class{TestCase} and creates an instance of the class for each test - method defined for the class. - - \warning{While using a hierarchy of - \class{TestCase}-derived classes can be convenient in sharing - fixtures and helper functions, defining test methods on base classes - that are not intended to be instantiated directly does not play well - with this method. Doing so, however, can be useful when the - fixtures are different and defined in subclasses.} -\end{methoddesc} - -\begin{methoddesc}[TestLoader]{loadTestsFromName}{name\optional{, module}} - Return a suite of all tests cases given a string specifier. - - The specifier \var{name} is a ``dotted name'' that may resolve - either to a module, a test case class, a test method within a test - case class, a \class{TestSuite} instance, or a callable object which - returns a \class{TestCase} or \class{TestSuite} instance. These checks - are applied in the order listed here; that is, a method on a possible - test case class will be picked up as ``a test method within a test - case class'', rather than ``a callable object''. - - For example, if you have a module \module{SampleTests} containing a - \class{TestCase}-derived class \class{SampleTestCase} with three test - methods (\method{test_one()}, \method{test_two()}, and - \method{test_three()}), the specifier \code{'SampleTests.SampleTestCase'} - would cause this method to return a suite which will run all three test - methods. Using the specifier \code{'SampleTests.SampleTestCase.test_two'} - would cause it to return a test suite which will run only the - \method{test_two()} test method. The specifier can refer to modules - and packages which have not been imported; they will be imported as - a side-effect. - - The method optionally resolves \var{name} relative to the given - \var{module}. -\end{methoddesc} - -\begin{methoddesc}[TestLoader]{loadTestsFromNames}{names\optional{, module}} - Similar to \method{loadTestsFromName()}, but takes a sequence of - names rather than a single name. The return value is a test suite - which supports all the tests defined for each name. -\end{methoddesc} - -\begin{methoddesc}[TestLoader]{getTestCaseNames}{testCaseClass} - Return a sorted sequence of method names found within - \var{testCaseClass}; this should be a subclass of \class{TestCase}. -\end{methoddesc} - - -The following attributes of a \class{TestLoader} can be configured -either by subclassing or assignment on an instance: - -\begin{memberdesc}[TestLoader]{testMethodPrefix} - String giving the prefix of method names which will be interpreted - as test methods. The default value is \code{'test'}. - - This affects \method{getTestCaseNames()} and all the - \method{loadTestsFrom*()} methods. -\end{memberdesc} - -\begin{memberdesc}[TestLoader]{sortTestMethodsUsing} - Function to be used to compare method names when sorting them in - \method{getTestCaseNames()} and all the \method{loadTestsFrom*()} methods. - The default value is the built-in \function{cmp()} function; the attribute - can also be set to \constant{None} to disable the sort. -\end{memberdesc} - -\begin{memberdesc}[TestLoader]{suiteClass} - Callable object that constructs a test suite from a list of tests. - No methods on the resulting object are needed. The default value is - the \class{TestSuite} class. - - This affects all the \method{loadTestsFrom*()} methods. -\end{memberdesc} |