diff options
author | Benjamin Peterson <benjamin@python.org> | 2009-03-23 23:10:14 (GMT) |
---|---|---|
committer | Benjamin Peterson <benjamin@python.org> | 2009-03-23 23:10:14 (GMT) |
commit | 99721e0c44f28895f62af33850a7de974542c3bd (patch) | |
tree | bb43c71d87b0c4ec45d8fb57b68082e6de8f0e74 /Doc | |
parent | cb2b0e45d4384a45e0852e8e7eba958a70949abb (diff) | |
download | cpython-99721e0c44f28895f62af33850a7de974542c3bd.zip cpython-99721e0c44f28895f62af33850a7de974542c3bd.tar.gz cpython-99721e0c44f28895f62af33850a7de974542c3bd.tar.bz2 |
refactor unittest docs
Diffstat (limited to 'Doc')
-rw-r--r-- | Doc/library/unittest.rst | 802 |
1 files changed, 413 insertions, 389 deletions
diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index 69227cf..a52677d 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -11,9 +11,9 @@ .. versionadded:: 2.1 -.. versionchanged:: 2.7 - Added :ref:`skipping and expected failures <unittest-skipping>`. +.. versionchanged:: 2.7 + Added test :ref:`skipping and expected failures <unittest-skipping>`. The Python unit testing framework, sometimes referred to as "PyUnit," is a Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in @@ -52,27 +52,28 @@ The test case and test fixture concepts are supported through the :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be used when creating new tests, and the latter can be used when integrating existing test code with a :mod:`unittest`\ -driven framework. When building test -fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods -can be overridden to provide initialization and cleanup for the fixture. With -:class:`FunctionTestCase`, existing functions can be passed to the constructor -for these purposes. When the test is run, the fixture initialization is run -first; if it succeeds, the cleanup method is run after the test has been -executed, regardless of the outcome of the test. Each instance of the -:class:`TestCase` will only be used to run a single test method, so a new -fixture is created for each test. +fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and +:meth:`~TestCase.tearDown` methods can be overridden to provide initialization +and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions +can be passed to the constructor for these purposes. When the test is run, the +fixture initialization is run first; if it succeeds, the cleanup method is run +after the test has been executed, regardless of the outcome of the test. Each +instance of the :class:`TestCase` will only be used to run a single test method, +so a new fixture is created for each test. Test suites are implemented by the :class:`TestSuite` class. This class allows individual tests and test suites to be aggregated; when the suite is executed, all tests added directly to the suite and in "child" test suites are run. A :class:`ClassTestSuite` contains the test cases of a class. -A test runner is an object that provides a single method, :meth:`run`, which -accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and -returns a result object. The class :class:`TestResult` is provided for use as -the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an -example test runner which reports test results on the standard error stream by -default. Alternate runners can be implemented for other environments (such as -graphical environments) without any need to derive from a specific class. +A test runner is an object that provides a single method, +:meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite` +object as a parameter, and returns a result object. The class +:class:`TestResult` is provided for use as the result object. :mod:`unittest` +provides the :class:`TextTestRunner` as an example test runner which reports +test results on the standard error stream by default. Alternate runners can be +implemented for other environments (such as graphical environments) without any +need to derive from a specific class. .. seealso:: @@ -104,17 +105,17 @@ Here is a short script to test three functions from the :mod:`random` module:: def setUp(self): self.seq = range(10) - def testshuffle(self): + def test_shuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10)) - def testchoice(self): + def test_choice(self): element = random.choice(self.seq) self.assert_(element in self.seq) - def testsample(self): + def test_sample(self): self.assertRaises(ValueError, random.sample, self.seq, 20) for element in random.sample(self.seq, 5): self.assert_(element in self.seq) @@ -122,21 +123,22 @@ Here is a short script to test three functions from the :mod:`random` module:: if __name__ == '__main__': unittest.main() -A testcase is created by subclassing :class:`unittest.TestCase`. The three +A testcase is created by subclassing :class:`unittest.TestCase`. The three individual tests are defined with methods whose names start with the letters ``test``. This naming convention informs the test runner about which methods represent tests. -The crux of each test is a call to :meth:`assertEqual` to check for an expected -result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify -that an expected exception gets raised. These methods are used instead of the -:keyword:`assert` statement so the test runner can accumulate all test results -and produce a report. +The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an +expected result; :meth:`~TestCase.assert_` to verify a condition; or +:meth:`~TestCase.assertRaises` to verify that an expected exception gets raised. +These methods are used instead of the :keyword:`assert` statement so the test +runner can accumulate all test results and produce a report. -When a :meth:`setUp` method is defined, the test runner will run that method -prior to each test. Likewise, if a :meth:`tearDown` method is defined, the test -runner will invoke that method after each test. In the example, :meth:`setUp` -was used to create a fresh sequence for each test. +When a :meth:`~TestCase.setUp` method is defined, the test runner will run that +method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is +defined, the test runner will invoke that method after each test. In the +example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each +test. The final block shows a simple way to run the tests. :func:`unittest.main` provides a command line interface to the test script. When run from the command @@ -191,8 +193,8 @@ The testing code of a :class:`TestCase` instance should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases. -The simplest :class:`TestCase` subclass will simply override the :meth:`runTest` -method in order to perform specific testing code:: +The simplest :class:`TestCase` subclass will simply override the +:meth:`~TestCase.runTest` method in order to perform specific testing code:: import unittest @@ -221,8 +223,8 @@ the above case, constructing a :class:`Widget` in each of 100 Widget test case subclasses would mean unsightly duplication. Luckily, we can factor out such set-up code by implementing a method called -:meth:`setUp`, which the testing framework will automatically call for us when -we run the test:: +:meth:`~TestCase.setUp`, which the testing framework will automatically call for +us when we run the test:: import unittest @@ -241,12 +243,12 @@ we run the test:: self.failUnless(self.widget.size() == (100,150), 'wrong size after resize') -If the :meth:`setUp` method raises an exception while the test is running, the -framework will consider the test to have suffered an error, and the -:meth:`runTest` method will not be executed. +If the :meth:`~TestCase.setUp` method raises an exception while the test is +running, the framework will consider the test to have suffered an error, and the +:meth:`~TestCase.runTest` method will not be executed. -Similarly, we can provide a :meth:`tearDown` method that tidies up after the -:meth:`runTest` method has been run:: +Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up +after the :meth:`~TestCase.runTest` method has been run:: import unittest @@ -258,8 +260,8 @@ Similarly, we can provide a :meth:`tearDown` method that tidies up after the self.widget.dispose() self.widget = None -If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether -:meth:`runTest` succeeded or not. +If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will +be run whether :meth:`~TestCase.runTest` succeeded or not. Such a working environment for the testing code is called a :dfn:`fixture`. @@ -288,11 +290,12 @@ mechanism:: self.failUnless(self.widget.size() == (100,150), 'wrong size after resize') -Here we have not provided a :meth:`runTest` method, but have instead provided -two different test methods. Class instances will now each run one of the -:meth:`test\*` methods, with ``self.widget`` created and destroyed separately -for each instance. When creating an instance we must specify the test method it -is to run. We do this by passing the method name in the constructor:: +Here we have not provided a :meth:`~TestCase.runTest` method, but have instead +provided two different test methods. Class instances will now each run one of +the :meth:`test\*` methods, with ``self.widget`` created and destroyed +separately for each instance. When creating an instance we must specify the +test method it is to run. We do this by passing the method name in the +constructor:: defaultSizeTestCase = WidgetTestCase('testDefaultSize') resizeTestCase = WidgetTestCase('testResize') @@ -411,6 +414,11 @@ may treat :exc:`AssertionError` differently. recommended. Taking the time to set up proper :class:`TestCase` subclasses will make future test refactorings infinitely easier. +In some cases, the existing tests may have been written using the :mod:`doctest` +module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can +automatically build :class:`unittest.TestSuite` instances from the existing +:mod:`doctest`\ -based tests. + .. _unittest-skipping: @@ -489,6 +497,13 @@ The following decorators implement test skipping and expected failures: Classes and functions --------------------- +This section describes in depth the API of :mod:`unittest`. + + +.. _testcase-objects: + +Test cases +~~~~~~~~~~ .. class:: TestCase([methodName]) @@ -512,560 +527,569 @@ Classes and functions Here, we create two instances of :class:`WidgetTestCase`, each of which runs a single test. - *methodName* defaults to ``'runTest'``. + *methodName* defaults to :meth:`runTest`. + :class:`TestCase` instances provide three groups of methods: one group used + to run the test, another used by the test implementation to check conditions + and report failures, and some inquiry methods allowing information about the + test itself to be gathered. -.. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]]) + Methods in the first group (running the test) are: - This class implements the portion of the :class:`TestCase` interface which - allows the test runner to drive the test, but does not provide the methods which - test code can use to check and report errors. This is used to create test cases - using legacy test code, allowing it to be integrated into a :mod:`unittest`\ - -based test framework. + .. method:: setUp() -.. class:: TestSuite([tests]) + Method called to prepare the test fixture. This is called immediately + before calling the test method; any exception raised by this method will + be considered an error rather than a test failure. The default + implementation does nothing. - This class represents an aggregation of individual tests cases and test suites. - The class presents the interface needed by the test runner to allow it to be run - as any other test case. Running a :class:`TestSuite` instance is the same as - iterating over the suite, running each test individually. - If *tests* is given, it must be an iterable of individual test cases or other - test suites that will be used to build the suite initially. Additional methods - are provided to add test cases and suites to the collection later on. + .. method:: tearDown() -.. class:: ClassTestSuite(tests, collected_from) + Method called immediately after the test method has been called and the + result recorded. This is called even if the test method raised an + exception, so the implementation in subclasses may need to be particularly + careful about checking internal state. Any exception raised by this + method will be considered an error rather than a test failure. This + method will only be called if the :meth:`setUp` succeeds, regardless of + the outcome of the test method. The default implementation does nothing. - This subclass of :class:`TestSuite` repesents an aggregation of individuals - tests from one :class:`TestCase` class. *tests* is an iterable of - :class:`TestCase` instances created from the class. *collected_from* is the - class they came from. + .. method:: run([result]) -.. class:: TestLoader() + Run the test, collecting the result into the test result object passed as + *result*. If *result* is omitted or :const:`None`, a temporary result + object is created (by calling the :meth:`defaultTestCase` method) and + used; this result object is not returned to :meth:`run`'s caller. - This class is responsible for loading tests according to various criteria and - returning them wrapped in a :class:`TestSuite`. It can load all tests within a - given module or :class:`TestCase` subclass. + The same effect may be had by simply calling the :class:`TestCase` + instance. -.. class:: TestResult() + .. method:: skip(reason) - This class is used to compile information about which tests have succeeded and - which have failed. + Skips the current test. See :ref:`unittest-skipping` for more + information. -.. data:: defaultTestLoader + .. method:: debug() - Instance of the :class:`TestLoader` class intended to be shared. If no - customization of the :class:`TestLoader` is needed, this instance can be used - instead of repeatedly creating new instances. + Run the test without collecting the result. This allows exceptions raised + by the test to be propagated to the caller, and can be used to support + running tests under a debugger. + The test code can use any of the following methods to check for and report + failures. -.. class:: TextTestRunner([stream[, descriptions[, verbosity]]]) - A basic test runner implementation which prints results on standard error. It - has a few configurable parameters, but is essentially very simple. Graphical - applications which run test suites should provide alternate implementations. + .. method:: assert_(expr[, msg]) + failUnless(expr[, msg]) + assertTrue(expr[, msg]) + Signal a test failure if *expr* is false; the explanation for the error + will be *msg* if given, otherwise it will be :const:`None`. -.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]]) - A command-line program that runs a set of tests; this is primarily for making - test modules conveniently executable. The simplest use for this function is to - include the following line at the end of a test script:: + .. method:: assertEqual(first, second[, msg]) + failUnlessEqual(first, second[, msg]) - if __name__ == '__main__': - unittest.main() + Test that *first* and *second* are equal. If the values do not compare + equal, the test will fail with the explanation given by *msg*, or + :const:`None`. Note that using :meth:`failUnlessEqual` improves upon + doing the comparison as the first parameter to :meth:`failUnless`: the + default value for *msg* can be computed to include representations of both + *first* and *second*. - The *testRunner* argument can either be a test runner class or an already - created instance of it. -In some cases, the existing tests may have been written using the :mod:`doctest` -module. If so, that module provides a :class:`DocTestSuite` class that can -automatically build :class:`unittest.TestSuite` instances from the existing -:mod:`doctest`\ -based tests. + .. method:: assertNotEqual(first, second[, msg]) + failIfEqual(first, second[, msg]) -.. versionadded:: 2.3 + Test that *first* and *second* are not equal. If the values do compare + equal, the test will fail with the explanation given by *msg*, or + :const:`None`. Note that using :meth:`failIfEqual` improves upon doing + the comparison as the first parameter to :meth:`failUnless` is that the + default value for *msg* can be computed to include representations of both + *first* and *second*. -.. _testcase-objects: + .. method:: assertAlmostEqual(first, second[, places[, msg]]) + failUnlessAlmostEqual(first, second[, places[, msg]]) -TestCase Objects ----------------- + Test that *first* and *second* are approximately equal by computing the + difference, rounding to the given number of decimal *places* (default 7), + and comparing to zero. -Each :class:`TestCase` instance represents a single test, but each concrete -subclass may be used to define multiple tests --- the concrete class represents -a single test fixture. The fixture is created and cleaned up for each test -case. + Note that comparing a given number of decimal places is not the same as + comparing a given number of significant digits. If the values do not + compare equal, the test will fail with the explanation given by *msg*, or + :const:`None`. -:class:`TestCase` instances provide three groups of methods: one group used to -run the test, another used by the test implementation to check conditions and -report failures, and some inquiry methods allowing information about the test -itself to be gathered. -Methods in the first group (running the test) are: + .. method:: assertNotAlmostEqual(first, second[, places[, msg]]) + failIfAlmostEqual(first, second[, places[, msg]]) + Test that *first* and *second* are not approximately equal by computing + the difference, rounding to the given number of decimal *places* (default + 7), and comparing to zero. -.. method:: TestCase.setUp() + Note that comparing a given number of decimal places is not the same as + comparing a given number of significant digits. If the values do not + compare equal, the test will fail with the explanation given by *msg*, or + :const:`None`. - Method called to prepare the test fixture. This is called immediately before - calling the test method; any exception raised by this method will be considered - an error rather than a test failure. The default implementation does nothing. + .. method:: assertRaises(exception[, callable, ...]) + failUnlessRaises(exception[, callable, ...]) -.. method:: TestCase.tearDown() + Test that an exception is raised when *callable* is called with any + positional or keyword arguments that are also passed to + :meth:`assertRaises`. The test passes if *exception* is raised, is an + error if another exception is raised, or fails if no exception is raised. + To catch any of a group of exceptions, a tuple containing the exception + classes may be passed as *exception*. - Method called immediately after the test method has been called and the result - recorded. This is called even if the test method raised an exception, so the - implementation in subclasses may need to be particularly careful about checking - internal state. Any exception raised by this method will be considered an error - rather than a test failure. This method will only be called if the - :meth:`setUp` succeeds, regardless of the outcome of the test method. The - default implementation does nothing. + .. versionchanged:: 2.7 + If *callable* is omitted or None, returns a context manager so that the + code under test can be written inline rather than as a function:: -.. method:: TestCase.run([result]) + with self.failUnlessRaises(some_error_class): + do_something() - Run the test, collecting the result into the test result object passed as - *result*. If *result* is omitted or :const:`None`, a temporary result object is - created (by calling the :meth:`defaultTestCase` method) and used; this result - object is not returned to :meth:`run`'s caller. + .. method:: failIf(expr[, msg]) + assertFalse(expr[, msg]) - The same effect may be had by simply calling the :class:`TestCase` instance. + The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. + This signals a test failure if *expr* is true, with *msg* or :const:`None` + for the error message. -.. method:: TestCase.skip(reason) + .. method:: fail([msg]) - Skips the current test. See :ref:`unittest-skipping`. + Signals a test failure unconditionally, with *msg* or :const:`None` for + the error message. -.. method:: TestCase.debug() + .. attribute:: failureException - Run the test without collecting the result. This allows exceptions raised by - the test to be propagated to the caller, and can be used to support running - tests under a debugger. + This class attribute gives the exception raised by the test method. If a + test framework needs to use a specialized exception, possibly to carry + additional information, it must subclass this exception in order to "play + fair" with the framework. The initial value of this attribute is + :exc:`AssertionError`. -The test code can use any of the following methods to check for and report -failures. + Testing frameworks can use the following methods to collect information on + the test: -.. method:: TestCase.assert_(expr[, msg]) - TestCase.failUnless(expr[, msg]) - TestCase.assertTrue(expr[, msg]) + .. method:: countTestCases() - Signal a test failure if *expr* is false; the explanation for the error will be - *msg* if given, otherwise it will be :const:`None`. + Return the number of tests represented by this test object. For + :class:`TestCase` instances, this will always be ``1``. -.. method:: TestCase.assertEqual(first, second[, msg]) - TestCase.failUnlessEqual(first, second[, msg]) + .. method:: defaultTestResult() - Test that *first* and *second* are equal. If the values do not compare equal, - the test will fail with the explanation given by *msg*, or :const:`None`. Note - that using :meth:`failUnlessEqual` improves upon doing the comparison as the - first parameter to :meth:`failUnless`: the default value for *msg* can be - computed to include representations of both *first* and *second*. + Return an instance of the test result class that should be used for this + test case class (if no other result instance is provided to the + :meth:`run` method). + For :class:`TestCase` instances, this will always be an instance of + :class:`TestResult`; subclasses of :class:`TestCase` should override this + as necessary. -.. method:: TestCase.assertNotEqual(first, second[, msg]) - TestCase.failIfEqual(first, second[, msg]) - Test that *first* and *second* are not equal. If the values do compare equal, - the test will fail with the explanation given by *msg*, or :const:`None`. Note - that using :meth:`failIfEqual` improves upon doing the comparison as the first - parameter to :meth:`failUnless` is that the default value for *msg* can be - computed to include representations of both *first* and *second*. + .. method:: id() + Return a string identifying the specific test case. This is usually the + full name of the test method, including the module and class name. -.. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]]) - TestCase.failUnlessAlmostEqual(first, second[, places[, msg]]) - Test that *first* and *second* are approximately equal by computing the - difference, rounding to the given number of decimal *places* (default 7), - and comparing to zero. - Note that comparing a given number of decimal places is not the same as - comparing a given number of significant digits. If the values do not compare - equal, the test will fail with the explanation given by *msg*, or :const:`None`. + .. method:: shortDescription() + Returns a one-line description of the test, or :const:`None` if no + description has been provided. The default implementation of this method + returns the first line of the test method's docstring, if available, or + :const:`None`. -.. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]]) - TestCase.failIfAlmostEqual(first, second[, places[, msg]]) - Test that *first* and *second* are not approximately equal by computing the - difference, rounding to the given number of decimal *places* (default 7), - and comparing to zero. - Note that comparing a given number of decimal places is not the same as - comparing a given number of significant digits. If the values do not compare - equal, the test will fail with the explanation given by *msg*, or :const:`None`. +.. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]]) + This class implements the portion of the :class:`TestCase` interface which + allows the test runner to drive the test, but does not provide the methods which + test code can use to check and report errors. This is used to create test cases + using legacy test code, allowing it to be integrated into a :mod:`unittest`\ + -based test framework. -.. method:: TestCase.assertRaises(exception[, callable, ...]) - TestCase.failUnlessRaises(exception[, callable, ...]) - Test that an exception is raised when *callable* is called with any positional - or keyword arguments that are also passed to :meth:`assertRaises`. The test - passes if *exception* is raised, is an error if another exception is raised, or - fails if no exception is raised. To catch any of a group of exceptions, a tuple - containing the exception classes may be passed as *exception*. +.. _testsuite-objects: - .. versionchanged:: 2.7 +Grouping tests +~~~~~~~~~~~~~~ - If *callable* is omitted or None, returns a context manager so that the code - under test can be written inline rather than as a function:: +.. class:: TestSuite([tests]) - with self.failUnlessRaises(some_error_class): - do_something() + This class represents an aggregation of individual tests cases and test suites. + The class presents the interface needed by the test runner to allow it to be run + as any other test case. Running a :class:`TestSuite` instance is the same as + iterating over the suite, running each test individually. -.. method:: TestCase.failIf(expr[, msg]) - TestCase.assertFalse(expr[, msg]) + If *tests* is given, it must be an iterable of individual test cases or other + test suites that will be used to build the suite initially. Additional methods + are provided to add test cases and suites to the collection later on. - The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. This - signals a test failure if *expr* is true, with *msg* or :const:`None` for the - error message. + :class:`TestSuite` (including :class:`ClassTestSuite`) objects behave much + like :class:`TestCase` objects, except they do not actually implement a test. + Instead, they are used to aggregate tests into groups of tests that should be + run together. Some additional methods are available to add tests to + :class:`TestSuite` instances: -.. method:: TestCase.fail([msg]) + .. method:: TestSuite.addTest(test) - Signals a test failure unconditionally, with *msg* or :const:`None` for the - error message. + Add a :class:`TestCase` or :class:`TestSuite` to the suite. -.. attribute:: TestCase.failureException + .. method:: TestSuite.addTests(tests) - This class attribute gives the exception raised by the :meth:`test` method. If - a test framework needs to use a specialized exception, possibly to carry - additional information, it must subclass this exception in order to "play fair" - with the framework. The initial value of this attribute is - :exc:`AssertionError`. + Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` + instances to this test suite. -Testing frameworks can use the following methods to collect information on the -test: + This is equivalent to iterating over *tests*, calling :meth:`addTest` for each + element. + :class:`TestSuite` shares the following methods with :class:`TestCase`: -.. method:: TestCase.countTestCases() - Return the number of tests represented by this test object. For - :class:`TestCase` instances, this will always be ``1``. + .. method:: run(result) + Run the tests associated with this suite, collecting the result into the + test result object passed as *result*. Note that unlike + :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to + be passed in. -.. method:: TestCase.defaultTestResult() - Return an instance of the test result class that should be used for this test - case class (if no other result instance is provided to the :meth:`run` method). + .. method:: debug() - For :class:`TestCase` instances, this will always be an instance of - :class:`TestResult`; subclasses of :class:`TestCase` should override this as - necessary. + Run the tests associated with this suite without collecting the + result. This allows exceptions raised by the test to be propagated to the + caller and can be used to support running tests under a debugger. -.. method:: TestCase.id() + .. method:: countTestCases() - Return a string identifying the specific test case. This is usually the full - name of the test method, including the module and class name. + Return the number of tests represented by this test object, including all + individual tests and sub-suites. + In the typical usage of a :class:`TestSuite` object, the :meth:`run` method + is invoked by a :class:`TestRunner` rather than by the end-user test harness. -.. method:: TestCase.shortDescription() - Returns a one-line description of the test, or :const:`None` if no description - has been provided. The default implementation of this method returns the first - line of the test method's docstring, if available, or :const:`None`. +.. class:: ClassTestSuite(tests, collected_from) + This subclass of :class:`TestSuite` repesents an aggregation of individuals + tests from one :class:`TestCase` class. *tests* is an iterable of + :class:`TestCase` instances created from the class. *collected_from* is the + class they came from. -.. _testsuite-objects: -TestSuite Objects ------------------ +Loading and running tests +~~~~~~~~~~~~~~~~~~~~~~~~~ -:class:`TestSuite` (including :class:`ClassTestSuite`) objects behave much like -:class:`TestCase` objects, except they do not actually implement a test. -Instead, they are used to aggregate tests into groups of tests that should be -run together. Some additional methods are available to add tests to -:class:`TestSuite` instances: +.. class:: TestLoader() + The :class:`TestLoader` class is used to create test suites from classes and + modules. Normally, there is no need to create an instance of this class; the + :mod:`unittest` module provides an instance that can be shared as + ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows + customization of some configurable properties. -.. method:: TestSuite.addTest(test) + :class:`TestLoader` objects have the following methods: - Add a :class:`TestCase` or :class:`TestSuite` to the suite. + .. method:: loadTestsFromTestCase(testCaseClass) -.. method:: TestSuite.addTests(tests) + Return a suite of all tests cases contained in the :class:`TestCase`\ -derived + :class:`testCaseClass`. - Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` - instances to this test suite. - This is equivalent to iterating over *tests*, calling :meth:`addTest` for each - element. + .. method:: loadTestsFromModule(module) -:class:`TestSuite` shares the following methods with :class:`TestCase`: + Return a suite of all tests cases contained in the given module. This + method searches *module* for classes derived from :class:`TestCase` and + creates an instance of the class for each test method defined for the + class. + .. warning:: -.. method:: TestSuite.run(result) + While using a hierarchy of :class:`TestCase`\ -derived classes can be + convenient in sharing fixtures and helper functions, defining test + methods on base classes that are not intended to be instantiated + directly does not play well with this method. Doing so, however, can + be useful when the fixtures are different and defined in subclasses. - Run the tests associated with this suite, collecting the result into the test - result object passed as *result*. Note that unlike :meth:`TestCase.run`, - :meth:`TestSuite.run` requires the result object to be passed in. + .. method:: loadTestsFromName(name[, module]) -.. method:: TestSuite.debug() + Return a suite of all tests cases given a string specifier. - Run the tests associated with this suite without collecting the result. This - allows exceptions raised by the test to be propagated to the caller and can be - used to support running tests under a debugger. + The specifier *name* is a "dotted name" that may resolve either to a + module, a test case class, a test method within a test case class, a + :class:`TestSuite` instance, or a callable object which returns a + :class:`TestCase` or :class:`TestSuite` instance. These checks are + applied in the order listed here; that is, a method on a possible test + case class will be picked up as "a test method within a test case class", + rather than "a callable object". + For example, if you have a module :mod:`SampleTests` containing a + :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test + methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the + specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a + suite which will run all three test methods. Using the specifier + ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite + which will run only the :meth:`test_two` test method. The specifier can refer + to modules and packages which have not been imported; they will be imported as a + side-effect. -.. method:: TestSuite.countTestCases() + The method optionally resolves *name* relative to the given *module*. - Return the number of tests represented by this test object, including all - individual tests and sub-suites. -In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is -invoked by a :class:`TestRunner` rather than by the end-user test harness. + .. method:: loadTestsFromNames(names[, module]) + Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather + than a single name. The return value is a test suite which supports all + the tests defined for each name. -.. _testresult-objects: -TestResult Objects ------------------- + .. method:: getTestCaseNames(testCaseClass) -A :class:`TestResult` object stores the results of a set of tests. The -:class:`TestCase` and :class:`TestSuite` classes ensure that results are -properly recorded; test authors do not need to worry about recording the outcome -of tests. + Return a sorted sequence of method names found within *testCaseClass*; + this should be a subclass of :class:`TestCase`. -Testing frameworks built on top of :mod:`unittest` may want access to the -:class:`TestResult` object generated by running a set of tests for reporting -purposes; a :class:`TestResult` instance is returned by the -:meth:`TestRunner.run` method for this purpose. + The following attributes of a :class:`TestLoader` can be configured either by + subclassing or assignment on an instance: -:class:`TestResult` instances have the following attributes that will be of -interest when inspecting the results of running a set of tests: + .. attribute:: testMethodPrefix -.. attribute:: TestResult.errors + String giving the prefix of method names which will be interpreted as test + methods. The default value is ``'test'``. - A list containing 2-tuples of :class:`TestCase` instances and strings holding - formatted tracebacks. Each tuple represents a test which raised an unexpected - exception. + This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` + methods. - .. versionchanged:: 2.2 - Contains formatted tracebacks instead of :func:`sys.exc_info` results. + .. attribute:: sortTestMethodsUsing -.. attribute:: TestResult.failures + Function to be used to compare method names when sorting them in + :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The + default value is the built-in :func:`cmp` function; the attribute can also + be set to :const:`None` to disable the sort. - A list containing 2-tuples of :class:`TestCase` instances and strings holding - formatted tracebacks. Each tuple represents a test where a failure was - explicitly signalled using the :meth:`TestCase.fail\*` or - :meth:`TestCase.assert\*` methods. - .. versionchanged:: 2.2 - Contains formatted tracebacks instead of :func:`sys.exc_info` results. + .. attribute:: suiteClass + Callable object that constructs a test suite from a list of tests. No + methods on the resulting object are needed. The default value is the + :class:`TestSuite` class. -.. attribute:: TestResult.testsRun + This affects all the :meth:`loadTestsFrom\*` methods. - The total number of tests run so far. + .. attribute:: classSuiteClass -.. method:: TestResult.wasSuccessful() + Callable object that constructs a test suite for the tests cases from one + class. The default value is :class:`ClassTestSuite`. - Returns :const:`True` if all tests run so far have passed, otherwise returns - :const:`False`. +.. class:: TestResult -.. method:: TestResult.stop() + This class is used to compile information about which tests have succeeded + and which have failed. - This method can be called to signal that the set of tests being run should be - aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to - :const:`True`. :class:`TestRunner` objects should respect this flag and return - without running any additional tests. + A :class:`TestResult` object stores the results of a set of tests. The + :class:`TestCase` and :class:`TestSuite` classes ensure that results are + properly recorded; test authors do not need to worry about recording the + outcome of tests. - For example, this feature is used by the :class:`TextTestRunner` class to stop - the test framework when the user signals an interrupt from the keyboard. - Interactive tools which provide :class:`TestRunner` implementations can use this - in a similar manner. + Testing frameworks built on top of :mod:`unittest` may want access to the + :class:`TestResult` object generated by running a set of tests for reporting + purposes; a :class:`TestResult` instance is returned by the + :meth:`TestRunner.run` method for this purpose. -The following methods of the :class:`TestResult` class are used to maintain the -internal data structures, and may be extended in subclasses to support -additional reporting requirements. This is particularly useful in building -tools which support interactive reporting while tests are being run. + :class:`TestResult` instances have the following attributes that will be of + interest when inspecting the results of running a set of tests: -.. method:: TestResult.startTest(test) + .. attribute:: errors - Called when the test case *test* is about to be run. + A list containing 2-tuples of :class:`TestCase` instances and strings + holding formatted tracebacks. Each tuple represents a test which raised an + unexpected exception. - The default implementation simply increments the instance's ``testsRun`` - counter. + .. versionchanged:: 2.2 + Contains formatted tracebacks instead of :func:`sys.exc_info` results. -.. method:: TestResult.stopTest(test) - Called after the test case *test* has been executed, regardless of the outcome. + .. attribute:: failures - The default implementation does nothing. + A list containing 2-tuples of :class:`TestCase` instances and strings + holding formatted tracebacks. Each tuple represents a test where a failure + was explicitly signalled using the :meth:`TestCase.fail\*` or + :meth:`TestCase.assert\*` methods. + .. versionchanged:: 2.2 -.. method:: TestResult.addError(test, err) + Contains formatted tracebacks instead of :func:`sys.exc_info` results. - Called when the test case *test* raises an unexpected exception *err* is a tuple - of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. + .. attribute:: skipped - The default implementation appends a tuple ``(test, formatted_err)`` to the - instance's ``errors`` attribute, where *formatted_err* is a formatted - traceback derived from *err*. + A list containing 2-tuples of :class:`TestCase` instances and strings + holding the reason for skipping the test. + .. versionadded:: 2.7 -.. method:: TestResult.addFailure(test, err) + .. attribute:: expectedFailures - Called when the test case *test* signals a failure. *err* is a tuple of the form - returned by :func:`sys.exc_info`: ``(type, value, traceback)``. + A list contaning 2-tuples of :class:`TestCase` instances and strings + holding formatted tracebacks. Each tuple represents a expected failures + of the test case. - The default implementation appends a tuple ``(test, formatted_err)`` to the - instance's ``failures`` attribute, where *formatted_err* is a formatted - traceback derived from *err*. + .. attribute:: unexpectedSuccesses + A list containing :class:`TestCase` instances that were marked as expected + failures, but succeeded. -.. method:: TestResult.addSuccess(test) + .. attribute:: shouldStop - Called when the test case *test* succeeds. + Set to ``True`` when the execution of tests should stop by :meth:`stop`. - The default implementation does nothing. + .. attribute:: testsRun -.. method:: TestResult.addSkip(test, reason) + The total number of tests run so far. - Called when the test case *test* is skipped. *reason* is the reason the test - gave for skipping. - The default implementation appends a tuple ``(test, reason)`` to the - instance's ``skipped`` attribute. + .. method:: wasSuccessful() + Return :const:`True` if all tests run so far have passed, otherwise returns + :const:`False`. -.. method:: TestResult.addExpectedFailure(test, err) - Called when the test case *test* fails, but was marked with the - :func:`expectedFailure` decorator. + .. method:: stop() - The default implementation appends a tuple ``(test, formatted_err)`` to the - instance's ``expectedFailures`` attribute, where *formatted_err* is a - formatted traceback derived from *err*. + This method can be called to signal that the set of tests being run should + be aborted by setting the :attr:`shouldStop` attribute to :const:`True`. + :class:`TestRunner` objects should respect this flag and return without + running any additional tests. + For example, this feature is used by the :class:`TextTestRunner` class to + stop the test framework when the user signals an interrupt from the + keyboard. Interactive tools which provide :class:`TestRunner` + implementations can use this in a similar manner. -.. method:: TestResult.addUnexpectedSuccess(test) + The following methods of the :class:`TestResult` class are used to maintain + the internal data structures, and may be extended in subclasses to support + additional reporting requirements. This is particularly useful in building + tools which support interactive reporting while tests are being run. - Called when the test case *test* was marked with the :func:`expectedFailure` - decorator, but succeeded. - The default implementation appends the test to the instance's - ``unexpectedSuccesses`` attribute. + .. method:: startTest(test) + Called when the test case *test* is about to be run. -.. _testloader-objects: + The default implementation simply increments the instance's :attr:`testsRun` + counter. -TestLoader Objects ------------------- -The :class:`TestLoader` class is used to create test suites from classes and -modules. Normally, there is no need to create an instance of this class; the -:mod:`unittest` module provides an instance that can be shared as -``unittest.defaultTestLoader``. Using a subclass or instance, however, allows -customization of some configurable properties. + .. method:: stopTest(test) -:class:`TestLoader` objects have the following methods: + Called after the test case *test* has been executed, regardless of the + outcome. + The default implementation does nothing. -.. method:: TestLoader.loadTestsFromTestCase(testCaseClass) - Return a suite of all tests cases contained in the :class:`TestCase`\ -derived - :class:`testCaseClass`. + .. method:: addError(test, err) + Called when the test case *test* raises an unexpected exception *err* is a + tuple of the form returned by :func:`sys.exc_info`: ``(type, value, + traceback)``. -.. method:: TestLoader.loadTestsFromModule(module) + The default implementation appends a tuple ``(test, formatted_err)`` to + the instance's :attr:`errors` attribute, where *formatted_err* is a + formatted traceback derived from *err*. - Return a suite of all tests cases contained in the given module. This method - searches *module* for classes derived from :class:`TestCase` and creates an - instance of the class for each test method defined for the class. - .. warning:: + .. method:: addFailure(test, err) - While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient - in sharing fixtures and helper functions, defining test methods on base classes - that are not intended to be instantiated directly does not play well with this - method. Doing so, however, can be useful when the fixtures are different and - defined in subclasses. + Called when the test case *test* signals a failure. *err* is a tuple of the form + returned by :func:`sys.exc_info`: ``(type, value, traceback)``. + The default implementation appends a tuple ``(test, formatted_err)`` to + the instance's :attr:`failures` attribute, where *formatted_err* is a + formatted traceback derived from *err*. -.. method:: TestLoader.loadTestsFromName(name[, module]) - Return a suite of all tests cases given a string specifier. + .. method:: addSuccess(test) - The specifier *name* is a "dotted name" that may resolve either to a module, a - test case class, a test method within a test case class, a :class:`TestSuite` - instance, or a callable object which returns a :class:`TestCase` or - :class:`TestSuite` instance. These checks are applied in the order listed here; - that is, a method on a possible test case class will be picked up as "a test - method within a test case class", rather than "a callable object". + Called when the test case *test* succeeds. - For example, if you have a module :mod:`SampleTests` containing a - :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test - methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the - specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a - suite which will run all three test methods. Using the specifier - ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite - which will run only the :meth:`test_two` test method. The specifier can refer - to modules and packages which have not been imported; they will be imported as a - side-effect. + The default implementation does nothing. - The method optionally resolves *name* relative to the given *module*. + .. method:: addSkip(test, reason) -.. method:: TestLoader.loadTestsFromNames(names[, module]) + Called when the test case *test* is skipped. *reason* is the reason the + test gave for skipping. - Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than - a single name. The return value is a test suite which supports all the tests - defined for each name. + The default implementation appends a tuple ``(test, reason)`` to the + instance's :attr:`skipped` attribute. -.. method:: TestLoader.getTestCaseNames(testCaseClass) + .. method:: addExpectedFailure(test, err) - Return a sorted sequence of method names found within *testCaseClass*; this - should be a subclass of :class:`TestCase`. + Called when the test case *test* fails, but was marked with the + :func:`expectedFailure` decorator. -The following attributes of a :class:`TestLoader` can be configured either by -subclassing or assignment on an instance: + The default implementation appends a tuple ``(test, formatted_err)`` to + the instance's :attr:`expectedFailures` attribute, where *formatted_err* + is a formatted traceback derived from *err*. -.. attribute:: TestLoader.testMethodPrefix + .. method:: addUnexpectedSuccess(test) - String giving the prefix of method names which will be interpreted as test - methods. The default value is ``'test'``. + Called when the test case *test* was marked with the + :func:`expectedFailure` decorator, but succeeded. - This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` - methods. + The default implementation appends the test to the instance's + :attr:`unexpectedSuccesses` attribute. -.. attribute:: TestLoader.sortTestMethodsUsing +.. data:: defaultTestLoader - Function to be used to compare method names when sorting them in - :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The - default value is the built-in :func:`cmp` function; the attribute can also be - set to :const:`None` to disable the sort. + Instance of the :class:`TestLoader` class intended to be shared. If no + customization of the :class:`TestLoader` is needed, this instance can be used + instead of repeatedly creating new instances. -.. attribute:: TestLoader.suiteClass +.. class:: TextTestRunner([stream[, descriptions[, verbosity]]]) - Callable object that constructs a test suite from a list of tests. No methods on - the resulting object are needed. The default value is the :class:`TestSuite` - class. + A basic test runner implementation which prints results on standard error. It + has a few configurable parameters, but is essentially very simple. Graphical + applications which run test suites should provide alternate implementations. - This affects all the :meth:`loadTestsFrom\*` methods. +.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]]) -.. attribute:: TestLoader.classSuiteClass + A command-line program that runs a set of tests; this is primarily for making + test modules conveniently executable. The simplest use for this function is to + include the following line at the end of a test script:: - Callable object that constructs a test suite for the tests cases from one - class. The default value is :class:`ClassTestSuite`. + if __name__ == '__main__': + unittest.main() + The *testRunner* argument can either be a test runner class or an already + created instance of it. |