summaryrefslogtreecommitdiffstats
path: root/Doc
diff options
context:
space:
mode:
authorBenjamin Peterson <benjamin@python.org>2009-03-23 23:10:14 (GMT)
committerBenjamin Peterson <benjamin@python.org>2009-03-23 23:10:14 (GMT)
commit99721e0c44f28895f62af33850a7de974542c3bd (patch)
treebb43c71d87b0c4ec45d8fb57b68082e6de8f0e74 /Doc
parentcb2b0e45d4384a45e0852e8e7eba958a70949abb (diff)
downloadcpython-99721e0c44f28895f62af33850a7de974542c3bd.zip
cpython-99721e0c44f28895f62af33850a7de974542c3bd.tar.gz
cpython-99721e0c44f28895f62af33850a7de974542c3bd.tar.bz2
refactor unittest docs
Diffstat (limited to 'Doc')
-rw-r--r--Doc/library/unittest.rst802
1 files changed, 413 insertions, 389 deletions
diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
index 69227cf..a52677d 100644
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -11,9 +11,9 @@
.. versionadded:: 2.1
-.. versionchanged:: 2.7
- Added :ref:`skipping and expected failures <unittest-skipping>`.
+.. versionchanged:: 2.7
+ Added test :ref:`skipping and expected failures <unittest-skipping>`.
The Python unit testing framework, sometimes referred to as "PyUnit," is a
Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
@@ -52,27 +52,28 @@ The test case and test fixture concepts are supported through the
:class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
used when creating new tests, and the latter can be used when integrating
existing test code with a :mod:`unittest`\ -driven framework. When building test
-fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods
-can be overridden to provide initialization and cleanup for the fixture. With
-:class:`FunctionTestCase`, existing functions can be passed to the constructor
-for these purposes. When the test is run, the fixture initialization is run
-first; if it succeeds, the cleanup method is run after the test has been
-executed, regardless of the outcome of the test. Each instance of the
-:class:`TestCase` will only be used to run a single test method, so a new
-fixture is created for each test.
+fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
+:meth:`~TestCase.tearDown` methods can be overridden to provide initialization
+and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions
+can be passed to the constructor for these purposes. When the test is run, the
+fixture initialization is run first; if it succeeds, the cleanup method is run
+after the test has been executed, regardless of the outcome of the test. Each
+instance of the :class:`TestCase` will only be used to run a single test method,
+so a new fixture is created for each test.
Test suites are implemented by the :class:`TestSuite` class. This class allows
individual tests and test suites to be aggregated; when the suite is executed,
all tests added directly to the suite and in "child" test suites are run. A
:class:`ClassTestSuite` contains the test cases of a class.
-A test runner is an object that provides a single method, :meth:`run`, which
-accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and
-returns a result object. The class :class:`TestResult` is provided for use as
-the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an
-example test runner which reports test results on the standard error stream by
-default. Alternate runners can be implemented for other environments (such as
-graphical environments) without any need to derive from a specific class.
+A test runner is an object that provides a single method,
+:meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
+object as a parameter, and returns a result object. The class
+:class:`TestResult` is provided for use as the result object. :mod:`unittest`
+provides the :class:`TextTestRunner` as an example test runner which reports
+test results on the standard error stream by default. Alternate runners can be
+implemented for other environments (such as graphical environments) without any
+need to derive from a specific class.
.. seealso::
@@ -104,17 +105,17 @@ Here is a short script to test three functions from the :mod:`random` module::
def setUp(self):
self.seq = range(10)
- def testshuffle(self):
+ def test_shuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
- def testchoice(self):
+ def test_choice(self):
element = random.choice(self.seq)
self.assert_(element in self.seq)
- def testsample(self):
+ def test_sample(self):
self.assertRaises(ValueError, random.sample, self.seq, 20)
for element in random.sample(self.seq, 5):
self.assert_(element in self.seq)
@@ -122,21 +123,22 @@ Here is a short script to test three functions from the :mod:`random` module::
if __name__ == '__main__':
unittest.main()
-A testcase is created by subclassing :class:`unittest.TestCase`. The three
+A testcase is created by subclassing :class:`unittest.TestCase`. The three
individual tests are defined with methods whose names start with the letters
``test``. This naming convention informs the test runner about which methods
represent tests.
-The crux of each test is a call to :meth:`assertEqual` to check for an expected
-result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify
-that an expected exception gets raised. These methods are used instead of the
-:keyword:`assert` statement so the test runner can accumulate all test results
-and produce a report.
+The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
+expected result; :meth:`~TestCase.assert_` to verify a condition; or
+:meth:`~TestCase.assertRaises` to verify that an expected exception gets raised.
+These methods are used instead of the :keyword:`assert` statement so the test
+runner can accumulate all test results and produce a report.
-When a :meth:`setUp` method is defined, the test runner will run that method
-prior to each test. Likewise, if a :meth:`tearDown` method is defined, the test
-runner will invoke that method after each test. In the example, :meth:`setUp`
-was used to create a fresh sequence for each test.
+When a :meth:`~TestCase.setUp` method is defined, the test runner will run that
+method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is
+defined, the test runner will invoke that method after each test. In the
+example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each
+test.
The final block shows a simple way to run the tests. :func:`unittest.main`
provides a command line interface to the test script. When run from the command
@@ -191,8 +193,8 @@ The testing code of a :class:`TestCase` instance should be entirely self
contained, such that it can be run either in isolation or in arbitrary
combination with any number of other test cases.
-The simplest :class:`TestCase` subclass will simply override the :meth:`runTest`
-method in order to perform specific testing code::
+The simplest :class:`TestCase` subclass will simply override the
+:meth:`~TestCase.runTest` method in order to perform specific testing code::
import unittest
@@ -221,8 +223,8 @@ the above case, constructing a :class:`Widget` in each of 100 Widget test case
subclasses would mean unsightly duplication.
Luckily, we can factor out such set-up code by implementing a method called
-:meth:`setUp`, which the testing framework will automatically call for us when
-we run the test::
+:meth:`~TestCase.setUp`, which the testing framework will automatically call for
+us when we run the test::
import unittest
@@ -241,12 +243,12 @@ we run the test::
self.failUnless(self.widget.size() == (100,150),
'wrong size after resize')
-If the :meth:`setUp` method raises an exception while the test is running, the
-framework will consider the test to have suffered an error, and the
-:meth:`runTest` method will not be executed.
+If the :meth:`~TestCase.setUp` method raises an exception while the test is
+running, the framework will consider the test to have suffered an error, and the
+:meth:`~TestCase.runTest` method will not be executed.
-Similarly, we can provide a :meth:`tearDown` method that tidies up after the
-:meth:`runTest` method has been run::
+Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
+after the :meth:`~TestCase.runTest` method has been run::
import unittest
@@ -258,8 +260,8 @@ Similarly, we can provide a :meth:`tearDown` method that tidies up after the
self.widget.dispose()
self.widget = None
-If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether
-:meth:`runTest` succeeded or not.
+If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
+be run whether :meth:`~TestCase.runTest` succeeded or not.
Such a working environment for the testing code is called a :dfn:`fixture`.
@@ -288,11 +290,12 @@ mechanism::
self.failUnless(self.widget.size() == (100,150),
'wrong size after resize')
-Here we have not provided a :meth:`runTest` method, but have instead provided
-two different test methods. Class instances will now each run one of the
-:meth:`test\*` methods, with ``self.widget`` created and destroyed separately
-for each instance. When creating an instance we must specify the test method it
-is to run. We do this by passing the method name in the constructor::
+Here we have not provided a :meth:`~TestCase.runTest` method, but have instead
+provided two different test methods. Class instances will now each run one of
+the :meth:`test\*` methods, with ``self.widget`` created and destroyed
+separately for each instance. When creating an instance we must specify the
+test method it is to run. We do this by passing the method name in the
+constructor::
defaultSizeTestCase = WidgetTestCase('testDefaultSize')
resizeTestCase = WidgetTestCase('testResize')
@@ -411,6 +414,11 @@ may treat :exc:`AssertionError` differently.
recommended. Taking the time to set up proper :class:`TestCase` subclasses will
make future test refactorings infinitely easier.
+In some cases, the existing tests may have been written using the :mod:`doctest`
+module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
+automatically build :class:`unittest.TestSuite` instances from the existing
+:mod:`doctest`\ -based tests.
+
.. _unittest-skipping:
@@ -489,6 +497,13 @@ The following decorators implement test skipping and expected failures:
Classes and functions
---------------------
+This section describes in depth the API of :mod:`unittest`.
+
+
+.. _testcase-objects:
+
+Test cases
+~~~~~~~~~~
.. class:: TestCase([methodName])
@@ -512,560 +527,569 @@ Classes and functions
Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
single test.
- *methodName* defaults to ``'runTest'``.
+ *methodName* defaults to :meth:`runTest`.
+ :class:`TestCase` instances provide three groups of methods: one group used
+ to run the test, another used by the test implementation to check conditions
+ and report failures, and some inquiry methods allowing information about the
+ test itself to be gathered.
-.. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
+ Methods in the first group (running the test) are:
- This class implements the portion of the :class:`TestCase` interface which
- allows the test runner to drive the test, but does not provide the methods which
- test code can use to check and report errors. This is used to create test cases
- using legacy test code, allowing it to be integrated into a :mod:`unittest`\
- -based test framework.
+ .. method:: setUp()
-.. class:: TestSuite([tests])
+ Method called to prepare the test fixture. This is called immediately
+ before calling the test method; any exception raised by this method will
+ be considered an error rather than a test failure. The default
+ implementation does nothing.
- This class represents an aggregation of individual tests cases and test suites.
- The class presents the interface needed by the test runner to allow it to be run
- as any other test case. Running a :class:`TestSuite` instance is the same as
- iterating over the suite, running each test individually.
- If *tests* is given, it must be an iterable of individual test cases or other
- test suites that will be used to build the suite initially. Additional methods
- are provided to add test cases and suites to the collection later on.
+ .. method:: tearDown()
-.. class:: ClassTestSuite(tests, collected_from)
+ Method called immediately after the test method has been called and the
+ result recorded. This is called even if the test method raised an
+ exception, so the implementation in subclasses may need to be particularly
+ careful about checking internal state. Any exception raised by this
+ method will be considered an error rather than a test failure. This
+ method will only be called if the :meth:`setUp` succeeds, regardless of
+ the outcome of the test method. The default implementation does nothing.
- This subclass of :class:`TestSuite` repesents an aggregation of individuals
- tests from one :class:`TestCase` class. *tests* is an iterable of
- :class:`TestCase` instances created from the class. *collected_from* is the
- class they came from.
+ .. method:: run([result])
-.. class:: TestLoader()
+ Run the test, collecting the result into the test result object passed as
+ *result*. If *result* is omitted or :const:`None`, a temporary result
+ object is created (by calling the :meth:`defaultTestCase` method) and
+ used; this result object is not returned to :meth:`run`'s caller.
- This class is responsible for loading tests according to various criteria and
- returning them wrapped in a :class:`TestSuite`. It can load all tests within a
- given module or :class:`TestCase` subclass.
+ The same effect may be had by simply calling the :class:`TestCase`
+ instance.
-.. class:: TestResult()
+ .. method:: skip(reason)
- This class is used to compile information about which tests have succeeded and
- which have failed.
+ Skips the current test. See :ref:`unittest-skipping` for more
+ information.
-.. data:: defaultTestLoader
+ .. method:: debug()
- Instance of the :class:`TestLoader` class intended to be shared. If no
- customization of the :class:`TestLoader` is needed, this instance can be used
- instead of repeatedly creating new instances.
+ Run the test without collecting the result. This allows exceptions raised
+ by the test to be propagated to the caller, and can be used to support
+ running tests under a debugger.
+ The test code can use any of the following methods to check for and report
+ failures.
-.. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
- A basic test runner implementation which prints results on standard error. It
- has a few configurable parameters, but is essentially very simple. Graphical
- applications which run test suites should provide alternate implementations.
+ .. method:: assert_(expr[, msg])
+ failUnless(expr[, msg])
+ assertTrue(expr[, msg])
+ Signal a test failure if *expr* is false; the explanation for the error
+ will be *msg* if given, otherwise it will be :const:`None`.
-.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
- A command-line program that runs a set of tests; this is primarily for making
- test modules conveniently executable. The simplest use for this function is to
- include the following line at the end of a test script::
+ .. method:: assertEqual(first, second[, msg])
+ failUnlessEqual(first, second[, msg])
- if __name__ == '__main__':
- unittest.main()
+ Test that *first* and *second* are equal. If the values do not compare
+ equal, the test will fail with the explanation given by *msg*, or
+ :const:`None`. Note that using :meth:`failUnlessEqual` improves upon
+ doing the comparison as the first parameter to :meth:`failUnless`: the
+ default value for *msg* can be computed to include representations of both
+ *first* and *second*.
- The *testRunner* argument can either be a test runner class or an already
- created instance of it.
-In some cases, the existing tests may have been written using the :mod:`doctest`
-module. If so, that module provides a :class:`DocTestSuite` class that can
-automatically build :class:`unittest.TestSuite` instances from the existing
-:mod:`doctest`\ -based tests.
+ .. method:: assertNotEqual(first, second[, msg])
+ failIfEqual(first, second[, msg])
-.. versionadded:: 2.3
+ Test that *first* and *second* are not equal. If the values do compare
+ equal, the test will fail with the explanation given by *msg*, or
+ :const:`None`. Note that using :meth:`failIfEqual` improves upon doing
+ the comparison as the first parameter to :meth:`failUnless` is that the
+ default value for *msg* can be computed to include representations of both
+ *first* and *second*.
-.. _testcase-objects:
+ .. method:: assertAlmostEqual(first, second[, places[, msg]])
+ failUnlessAlmostEqual(first, second[, places[, msg]])
-TestCase Objects
-----------------
+ Test that *first* and *second* are approximately equal by computing the
+ difference, rounding to the given number of decimal *places* (default 7),
+ and comparing to zero.
-Each :class:`TestCase` instance represents a single test, but each concrete
-subclass may be used to define multiple tests --- the concrete class represents
-a single test fixture. The fixture is created and cleaned up for each test
-case.
+ Note that comparing a given number of decimal places is not the same as
+ comparing a given number of significant digits. If the values do not
+ compare equal, the test will fail with the explanation given by *msg*, or
+ :const:`None`.
-:class:`TestCase` instances provide three groups of methods: one group used to
-run the test, another used by the test implementation to check conditions and
-report failures, and some inquiry methods allowing information about the test
-itself to be gathered.
-Methods in the first group (running the test) are:
+ .. method:: assertNotAlmostEqual(first, second[, places[, msg]])
+ failIfAlmostEqual(first, second[, places[, msg]])
+ Test that *first* and *second* are not approximately equal by computing
+ the difference, rounding to the given number of decimal *places* (default
+ 7), and comparing to zero.
-.. method:: TestCase.setUp()
+ Note that comparing a given number of decimal places is not the same as
+ comparing a given number of significant digits. If the values do not
+ compare equal, the test will fail with the explanation given by *msg*, or
+ :const:`None`.
- Method called to prepare the test fixture. This is called immediately before
- calling the test method; any exception raised by this method will be considered
- an error rather than a test failure. The default implementation does nothing.
+ .. method:: assertRaises(exception[, callable, ...])
+ failUnlessRaises(exception[, callable, ...])
-.. method:: TestCase.tearDown()
+ Test that an exception is raised when *callable* is called with any
+ positional or keyword arguments that are also passed to
+ :meth:`assertRaises`. The test passes if *exception* is raised, is an
+ error if another exception is raised, or fails if no exception is raised.
+ To catch any of a group of exceptions, a tuple containing the exception
+ classes may be passed as *exception*.
- Method called immediately after the test method has been called and the result
- recorded. This is called even if the test method raised an exception, so the
- implementation in subclasses may need to be particularly careful about checking
- internal state. Any exception raised by this method will be considered an error
- rather than a test failure. This method will only be called if the
- :meth:`setUp` succeeds, regardless of the outcome of the test method. The
- default implementation does nothing.
+ .. versionchanged:: 2.7
+ If *callable* is omitted or None, returns a context manager so that the
+ code under test can be written inline rather than as a function::
-.. method:: TestCase.run([result])
+ with self.failUnlessRaises(some_error_class):
+ do_something()
- Run the test, collecting the result into the test result object passed as
- *result*. If *result* is omitted or :const:`None`, a temporary result object is
- created (by calling the :meth:`defaultTestCase` method) and used; this result
- object is not returned to :meth:`run`'s caller.
+ .. method:: failIf(expr[, msg])
+ assertFalse(expr[, msg])
- The same effect may be had by simply calling the :class:`TestCase` instance.
+ The inverse of the :meth:`failUnless` method is the :meth:`failIf` method.
+ This signals a test failure if *expr* is true, with *msg* or :const:`None`
+ for the error message.
-.. method:: TestCase.skip(reason)
+ .. method:: fail([msg])
- Skips the current test. See :ref:`unittest-skipping`.
+ Signals a test failure unconditionally, with *msg* or :const:`None` for
+ the error message.
-.. method:: TestCase.debug()
+ .. attribute:: failureException
- Run the test without collecting the result. This allows exceptions raised by
- the test to be propagated to the caller, and can be used to support running
- tests under a debugger.
+ This class attribute gives the exception raised by the test method. If a
+ test framework needs to use a specialized exception, possibly to carry
+ additional information, it must subclass this exception in order to "play
+ fair" with the framework. The initial value of this attribute is
+ :exc:`AssertionError`.
-The test code can use any of the following methods to check for and report
-failures.
+ Testing frameworks can use the following methods to collect information on
+ the test:
-.. method:: TestCase.assert_(expr[, msg])
- TestCase.failUnless(expr[, msg])
- TestCase.assertTrue(expr[, msg])
+ .. method:: countTestCases()
- Signal a test failure if *expr* is false; the explanation for the error will be
- *msg* if given, otherwise it will be :const:`None`.
+ Return the number of tests represented by this test object. For
+ :class:`TestCase` instances, this will always be ``1``.
-.. method:: TestCase.assertEqual(first, second[, msg])
- TestCase.failUnlessEqual(first, second[, msg])
+ .. method:: defaultTestResult()
- Test that *first* and *second* are equal. If the values do not compare equal,
- the test will fail with the explanation given by *msg*, or :const:`None`. Note
- that using :meth:`failUnlessEqual` improves upon doing the comparison as the
- first parameter to :meth:`failUnless`: the default value for *msg* can be
- computed to include representations of both *first* and *second*.
+ Return an instance of the test result class that should be used for this
+ test case class (if no other result instance is provided to the
+ :meth:`run` method).
+ For :class:`TestCase` instances, this will always be an instance of
+ :class:`TestResult`; subclasses of :class:`TestCase` should override this
+ as necessary.
-.. method:: TestCase.assertNotEqual(first, second[, msg])
- TestCase.failIfEqual(first, second[, msg])
- Test that *first* and *second* are not equal. If the values do compare equal,
- the test will fail with the explanation given by *msg*, or :const:`None`. Note
- that using :meth:`failIfEqual` improves upon doing the comparison as the first
- parameter to :meth:`failUnless` is that the default value for *msg* can be
- computed to include representations of both *first* and *second*.
+ .. method:: id()
+ Return a string identifying the specific test case. This is usually the
+ full name of the test method, including the module and class name.
-.. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]])
- TestCase.failUnlessAlmostEqual(first, second[, places[, msg]])
- Test that *first* and *second* are approximately equal by computing the
- difference, rounding to the given number of decimal *places* (default 7),
- and comparing to zero.
- Note that comparing a given number of decimal places is not the same as
- comparing a given number of significant digits. If the values do not compare
- equal, the test will fail with the explanation given by *msg*, or :const:`None`.
+ .. method:: shortDescription()
+ Returns a one-line description of the test, or :const:`None` if no
+ description has been provided. The default implementation of this method
+ returns the first line of the test method's docstring, if available, or
+ :const:`None`.
-.. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]])
- TestCase.failIfAlmostEqual(first, second[, places[, msg]])
- Test that *first* and *second* are not approximately equal by computing the
- difference, rounding to the given number of decimal *places* (default 7),
- and comparing to zero.
- Note that comparing a given number of decimal places is not the same as
- comparing a given number of significant digits. If the values do not compare
- equal, the test will fail with the explanation given by *msg*, or :const:`None`.
+.. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
+ This class implements the portion of the :class:`TestCase` interface which
+ allows the test runner to drive the test, but does not provide the methods which
+ test code can use to check and report errors. This is used to create test cases
+ using legacy test code, allowing it to be integrated into a :mod:`unittest`\
+ -based test framework.
-.. method:: TestCase.assertRaises(exception[, callable, ...])
- TestCase.failUnlessRaises(exception[, callable, ...])
- Test that an exception is raised when *callable* is called with any positional
- or keyword arguments that are also passed to :meth:`assertRaises`. The test
- passes if *exception* is raised, is an error if another exception is raised, or
- fails if no exception is raised. To catch any of a group of exceptions, a tuple
- containing the exception classes may be passed as *exception*.
+.. _testsuite-objects:
- .. versionchanged:: 2.7
+Grouping tests
+~~~~~~~~~~~~~~
- If *callable* is omitted or None, returns a context manager so that the code
- under test can be written inline rather than as a function::
+.. class:: TestSuite([tests])
- with self.failUnlessRaises(some_error_class):
- do_something()
+ This class represents an aggregation of individual tests cases and test suites.
+ The class presents the interface needed by the test runner to allow it to be run
+ as any other test case. Running a :class:`TestSuite` instance is the same as
+ iterating over the suite, running each test individually.
-.. method:: TestCase.failIf(expr[, msg])
- TestCase.assertFalse(expr[, msg])
+ If *tests* is given, it must be an iterable of individual test cases or other
+ test suites that will be used to build the suite initially. Additional methods
+ are provided to add test cases and suites to the collection later on.
- The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. This
- signals a test failure if *expr* is true, with *msg* or :const:`None` for the
- error message.
+ :class:`TestSuite` (including :class:`ClassTestSuite`) objects behave much
+ like :class:`TestCase` objects, except they do not actually implement a test.
+ Instead, they are used to aggregate tests into groups of tests that should be
+ run together. Some additional methods are available to add tests to
+ :class:`TestSuite` instances:
-.. method:: TestCase.fail([msg])
+ .. method:: TestSuite.addTest(test)
- Signals a test failure unconditionally, with *msg* or :const:`None` for the
- error message.
+ Add a :class:`TestCase` or :class:`TestSuite` to the suite.
-.. attribute:: TestCase.failureException
+ .. method:: TestSuite.addTests(tests)
- This class attribute gives the exception raised by the :meth:`test` method. If
- a test framework needs to use a specialized exception, possibly to carry
- additional information, it must subclass this exception in order to "play fair"
- with the framework. The initial value of this attribute is
- :exc:`AssertionError`.
+ Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
+ instances to this test suite.
-Testing frameworks can use the following methods to collect information on the
-test:
+ This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
+ element.
+ :class:`TestSuite` shares the following methods with :class:`TestCase`:
-.. method:: TestCase.countTestCases()
- Return the number of tests represented by this test object. For
- :class:`TestCase` instances, this will always be ``1``.
+ .. method:: run(result)
+ Run the tests associated with this suite, collecting the result into the
+ test result object passed as *result*. Note that unlike
+ :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
+ be passed in.
-.. method:: TestCase.defaultTestResult()
- Return an instance of the test result class that should be used for this test
- case class (if no other result instance is provided to the :meth:`run` method).
+ .. method:: debug()
- For :class:`TestCase` instances, this will always be an instance of
- :class:`TestResult`; subclasses of :class:`TestCase` should override this as
- necessary.
+ Run the tests associated with this suite without collecting the
+ result. This allows exceptions raised by the test to be propagated to the
+ caller and can be used to support running tests under a debugger.
-.. method:: TestCase.id()
+ .. method:: countTestCases()
- Return a string identifying the specific test case. This is usually the full
- name of the test method, including the module and class name.
+ Return the number of tests represented by this test object, including all
+ individual tests and sub-suites.
+ In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
+ is invoked by a :class:`TestRunner` rather than by the end-user test harness.
-.. method:: TestCase.shortDescription()
- Returns a one-line description of the test, or :const:`None` if no description
- has been provided. The default implementation of this method returns the first
- line of the test method's docstring, if available, or :const:`None`.
+.. class:: ClassTestSuite(tests, collected_from)
+ This subclass of :class:`TestSuite` repesents an aggregation of individuals
+ tests from one :class:`TestCase` class. *tests* is an iterable of
+ :class:`TestCase` instances created from the class. *collected_from* is the
+ class they came from.
-.. _testsuite-objects:
-TestSuite Objects
------------------
+Loading and running tests
+~~~~~~~~~~~~~~~~~~~~~~~~~
-:class:`TestSuite` (including :class:`ClassTestSuite`) objects behave much like
-:class:`TestCase` objects, except they do not actually implement a test.
-Instead, they are used to aggregate tests into groups of tests that should be
-run together. Some additional methods are available to add tests to
-:class:`TestSuite` instances:
+.. class:: TestLoader()
+ The :class:`TestLoader` class is used to create test suites from classes and
+ modules. Normally, there is no need to create an instance of this class; the
+ :mod:`unittest` module provides an instance that can be shared as
+ ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
+ customization of some configurable properties.
-.. method:: TestSuite.addTest(test)
+ :class:`TestLoader` objects have the following methods:
- Add a :class:`TestCase` or :class:`TestSuite` to the suite.
+ .. method:: loadTestsFromTestCase(testCaseClass)
-.. method:: TestSuite.addTests(tests)
+ Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
+ :class:`testCaseClass`.
- Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
- instances to this test suite.
- This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
- element.
+ .. method:: loadTestsFromModule(module)
-:class:`TestSuite` shares the following methods with :class:`TestCase`:
+ Return a suite of all tests cases contained in the given module. This
+ method searches *module* for classes derived from :class:`TestCase` and
+ creates an instance of the class for each test method defined for the
+ class.
+ .. warning::
-.. method:: TestSuite.run(result)
+ While using a hierarchy of :class:`TestCase`\ -derived classes can be
+ convenient in sharing fixtures and helper functions, defining test
+ methods on base classes that are not intended to be instantiated
+ directly does not play well with this method. Doing so, however, can
+ be useful when the fixtures are different and defined in subclasses.
- Run the tests associated with this suite, collecting the result into the test
- result object passed as *result*. Note that unlike :meth:`TestCase.run`,
- :meth:`TestSuite.run` requires the result object to be passed in.
+ .. method:: loadTestsFromName(name[, module])
-.. method:: TestSuite.debug()
+ Return a suite of all tests cases given a string specifier.
- Run the tests associated with this suite without collecting the result. This
- allows exceptions raised by the test to be propagated to the caller and can be
- used to support running tests under a debugger.
+ The specifier *name* is a "dotted name" that may resolve either to a
+ module, a test case class, a test method within a test case class, a
+ :class:`TestSuite` instance, or a callable object which returns a
+ :class:`TestCase` or :class:`TestSuite` instance. These checks are
+ applied in the order listed here; that is, a method on a possible test
+ case class will be picked up as "a test method within a test case class",
+ rather than "a callable object".
+ For example, if you have a module :mod:`SampleTests` containing a
+ :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
+ methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
+ specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
+ suite which will run all three test methods. Using the specifier
+ ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
+ which will run only the :meth:`test_two` test method. The specifier can refer
+ to modules and packages which have not been imported; they will be imported as a
+ side-effect.
-.. method:: TestSuite.countTestCases()
+ The method optionally resolves *name* relative to the given *module*.
- Return the number of tests represented by this test object, including all
- individual tests and sub-suites.
-In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is
-invoked by a :class:`TestRunner` rather than by the end-user test harness.
+ .. method:: loadTestsFromNames(names[, module])
+ Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
+ than a single name. The return value is a test suite which supports all
+ the tests defined for each name.
-.. _testresult-objects:
-TestResult Objects
-------------------
+ .. method:: getTestCaseNames(testCaseClass)
-A :class:`TestResult` object stores the results of a set of tests. The
-:class:`TestCase` and :class:`TestSuite` classes ensure that results are
-properly recorded; test authors do not need to worry about recording the outcome
-of tests.
+ Return a sorted sequence of method names found within *testCaseClass*;
+ this should be a subclass of :class:`TestCase`.
-Testing frameworks built on top of :mod:`unittest` may want access to the
-:class:`TestResult` object generated by running a set of tests for reporting
-purposes; a :class:`TestResult` instance is returned by the
-:meth:`TestRunner.run` method for this purpose.
+ The following attributes of a :class:`TestLoader` can be configured either by
+ subclassing or assignment on an instance:
-:class:`TestResult` instances have the following attributes that will be of
-interest when inspecting the results of running a set of tests:
+ .. attribute:: testMethodPrefix
-.. attribute:: TestResult.errors
+ String giving the prefix of method names which will be interpreted as test
+ methods. The default value is ``'test'``.
- A list containing 2-tuples of :class:`TestCase` instances and strings holding
- formatted tracebacks. Each tuple represents a test which raised an unexpected
- exception.
+ This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
+ methods.
- .. versionchanged:: 2.2
- Contains formatted tracebacks instead of :func:`sys.exc_info` results.
+ .. attribute:: sortTestMethodsUsing
-.. attribute:: TestResult.failures
+ Function to be used to compare method names when sorting them in
+ :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
+ default value is the built-in :func:`cmp` function; the attribute can also
+ be set to :const:`None` to disable the sort.
- A list containing 2-tuples of :class:`TestCase` instances and strings holding
- formatted tracebacks. Each tuple represents a test where a failure was
- explicitly signalled using the :meth:`TestCase.fail\*` or
- :meth:`TestCase.assert\*` methods.
- .. versionchanged:: 2.2
- Contains formatted tracebacks instead of :func:`sys.exc_info` results.
+ .. attribute:: suiteClass
+ Callable object that constructs a test suite from a list of tests. No
+ methods on the resulting object are needed. The default value is the
+ :class:`TestSuite` class.
-.. attribute:: TestResult.testsRun
+ This affects all the :meth:`loadTestsFrom\*` methods.
- The total number of tests run so far.
+ .. attribute:: classSuiteClass
-.. method:: TestResult.wasSuccessful()
+ Callable object that constructs a test suite for the tests cases from one
+ class. The default value is :class:`ClassTestSuite`.
- Returns :const:`True` if all tests run so far have passed, otherwise returns
- :const:`False`.
+.. class:: TestResult
-.. method:: TestResult.stop()
+ This class is used to compile information about which tests have succeeded
+ and which have failed.
- This method can be called to signal that the set of tests being run should be
- aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to
- :const:`True`. :class:`TestRunner` objects should respect this flag and return
- without running any additional tests.
+ A :class:`TestResult` object stores the results of a set of tests. The
+ :class:`TestCase` and :class:`TestSuite` classes ensure that results are
+ properly recorded; test authors do not need to worry about recording the
+ outcome of tests.
- For example, this feature is used by the :class:`TextTestRunner` class to stop
- the test framework when the user signals an interrupt from the keyboard.
- Interactive tools which provide :class:`TestRunner` implementations can use this
- in a similar manner.
+ Testing frameworks built on top of :mod:`unittest` may want access to the
+ :class:`TestResult` object generated by running a set of tests for reporting
+ purposes; a :class:`TestResult` instance is returned by the
+ :meth:`TestRunner.run` method for this purpose.
-The following methods of the :class:`TestResult` class are used to maintain the
-internal data structures, and may be extended in subclasses to support
-additional reporting requirements. This is particularly useful in building
-tools which support interactive reporting while tests are being run.
+ :class:`TestResult` instances have the following attributes that will be of
+ interest when inspecting the results of running a set of tests:
-.. method:: TestResult.startTest(test)
+ .. attribute:: errors
- Called when the test case *test* is about to be run.
+ A list containing 2-tuples of :class:`TestCase` instances and strings
+ holding formatted tracebacks. Each tuple represents a test which raised an
+ unexpected exception.
- The default implementation simply increments the instance's ``testsRun``
- counter.
+ .. versionchanged:: 2.2
+ Contains formatted tracebacks instead of :func:`sys.exc_info` results.
-.. method:: TestResult.stopTest(test)
- Called after the test case *test* has been executed, regardless of the outcome.
+ .. attribute:: failures
- The default implementation does nothing.
+ A list containing 2-tuples of :class:`TestCase` instances and strings
+ holding formatted tracebacks. Each tuple represents a test where a failure
+ was explicitly signalled using the :meth:`TestCase.fail\*` or
+ :meth:`TestCase.assert\*` methods.
+ .. versionchanged:: 2.2
-.. method:: TestResult.addError(test, err)
+ Contains formatted tracebacks instead of :func:`sys.exc_info` results.
- Called when the test case *test* raises an unexpected exception *err* is a tuple
- of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
+ .. attribute:: skipped
- The default implementation appends a tuple ``(test, formatted_err)`` to the
- instance's ``errors`` attribute, where *formatted_err* is a formatted
- traceback derived from *err*.
+ A list containing 2-tuples of :class:`TestCase` instances and strings
+ holding the reason for skipping the test.
+ .. versionadded:: 2.7
-.. method:: TestResult.addFailure(test, err)
+ .. attribute:: expectedFailures
- Called when the test case *test* signals a failure. *err* is a tuple of the form
- returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
+ A list contaning 2-tuples of :class:`TestCase` instances and strings
+ holding formatted tracebacks. Each tuple represents a expected failures
+ of the test case.
- The default implementation appends a tuple ``(test, formatted_err)`` to the
- instance's ``failures`` attribute, where *formatted_err* is a formatted
- traceback derived from *err*.
+ .. attribute:: unexpectedSuccesses
+ A list containing :class:`TestCase` instances that were marked as expected
+ failures, but succeeded.
-.. method:: TestResult.addSuccess(test)
+ .. attribute:: shouldStop
- Called when the test case *test* succeeds.
+ Set to ``True`` when the execution of tests should stop by :meth:`stop`.
- The default implementation does nothing.
+ .. attribute:: testsRun
-.. method:: TestResult.addSkip(test, reason)
+ The total number of tests run so far.
- Called when the test case *test* is skipped. *reason* is the reason the test
- gave for skipping.
- The default implementation appends a tuple ``(test, reason)`` to the
- instance's ``skipped`` attribute.
+ .. method:: wasSuccessful()
+ Return :const:`True` if all tests run so far have passed, otherwise returns
+ :const:`False`.
-.. method:: TestResult.addExpectedFailure(test, err)
- Called when the test case *test* fails, but was marked with the
- :func:`expectedFailure` decorator.
+ .. method:: stop()
- The default implementation appends a tuple ``(test, formatted_err)`` to the
- instance's ``expectedFailures`` attribute, where *formatted_err* is a
- formatted traceback derived from *err*.
+ This method can be called to signal that the set of tests being run should
+ be aborted by setting the :attr:`shouldStop` attribute to :const:`True`.
+ :class:`TestRunner` objects should respect this flag and return without
+ running any additional tests.
+ For example, this feature is used by the :class:`TextTestRunner` class to
+ stop the test framework when the user signals an interrupt from the
+ keyboard. Interactive tools which provide :class:`TestRunner`
+ implementations can use this in a similar manner.
-.. method:: TestResult.addUnexpectedSuccess(test)
+ The following methods of the :class:`TestResult` class are used to maintain
+ the internal data structures, and may be extended in subclasses to support
+ additional reporting requirements. This is particularly useful in building
+ tools which support interactive reporting while tests are being run.
- Called when the test case *test* was marked with the :func:`expectedFailure`
- decorator, but succeeded.
- The default implementation appends the test to the instance's
- ``unexpectedSuccesses`` attribute.
+ .. method:: startTest(test)
+ Called when the test case *test* is about to be run.
-.. _testloader-objects:
+ The default implementation simply increments the instance's :attr:`testsRun`
+ counter.
-TestLoader Objects
-------------------
-The :class:`TestLoader` class is used to create test suites from classes and
-modules. Normally, there is no need to create an instance of this class; the
-:mod:`unittest` module provides an instance that can be shared as
-``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
-customization of some configurable properties.
+ .. method:: stopTest(test)
-:class:`TestLoader` objects have the following methods:
+ Called after the test case *test* has been executed, regardless of the
+ outcome.
+ The default implementation does nothing.
-.. method:: TestLoader.loadTestsFromTestCase(testCaseClass)
- Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
- :class:`testCaseClass`.
+ .. method:: addError(test, err)
+ Called when the test case *test* raises an unexpected exception *err* is a
+ tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
+ traceback)``.
-.. method:: TestLoader.loadTestsFromModule(module)
+ The default implementation appends a tuple ``(test, formatted_err)`` to
+ the instance's :attr:`errors` attribute, where *formatted_err* is a
+ formatted traceback derived from *err*.
- Return a suite of all tests cases contained in the given module. This method
- searches *module* for classes derived from :class:`TestCase` and creates an
- instance of the class for each test method defined for the class.
- .. warning::
+ .. method:: addFailure(test, err)
- While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient
- in sharing fixtures and helper functions, defining test methods on base classes
- that are not intended to be instantiated directly does not play well with this
- method. Doing so, however, can be useful when the fixtures are different and
- defined in subclasses.
+ Called when the test case *test* signals a failure. *err* is a tuple of the form
+ returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
+ The default implementation appends a tuple ``(test, formatted_err)`` to
+ the instance's :attr:`failures` attribute, where *formatted_err* is a
+ formatted traceback derived from *err*.
-.. method:: TestLoader.loadTestsFromName(name[, module])
- Return a suite of all tests cases given a string specifier.
+ .. method:: addSuccess(test)
- The specifier *name* is a "dotted name" that may resolve either to a module, a
- test case class, a test method within a test case class, a :class:`TestSuite`
- instance, or a callable object which returns a :class:`TestCase` or
- :class:`TestSuite` instance. These checks are applied in the order listed here;
- that is, a method on a possible test case class will be picked up as "a test
- method within a test case class", rather than "a callable object".
+ Called when the test case *test* succeeds.
- For example, if you have a module :mod:`SampleTests` containing a
- :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
- methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
- specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
- suite which will run all three test methods. Using the specifier
- ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
- which will run only the :meth:`test_two` test method. The specifier can refer
- to modules and packages which have not been imported; they will be imported as a
- side-effect.
+ The default implementation does nothing.
- The method optionally resolves *name* relative to the given *module*.
+ .. method:: addSkip(test, reason)
-.. method:: TestLoader.loadTestsFromNames(names[, module])
+ Called when the test case *test* is skipped. *reason* is the reason the
+ test gave for skipping.
- Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than
- a single name. The return value is a test suite which supports all the tests
- defined for each name.
+ The default implementation appends a tuple ``(test, reason)`` to the
+ instance's :attr:`skipped` attribute.
-.. method:: TestLoader.getTestCaseNames(testCaseClass)
+ .. method:: addExpectedFailure(test, err)
- Return a sorted sequence of method names found within *testCaseClass*; this
- should be a subclass of :class:`TestCase`.
+ Called when the test case *test* fails, but was marked with the
+ :func:`expectedFailure` decorator.
-The following attributes of a :class:`TestLoader` can be configured either by
-subclassing or assignment on an instance:
+ The default implementation appends a tuple ``(test, formatted_err)`` to
+ the instance's :attr:`expectedFailures` attribute, where *formatted_err*
+ is a formatted traceback derived from *err*.
-.. attribute:: TestLoader.testMethodPrefix
+ .. method:: addUnexpectedSuccess(test)
- String giving the prefix of method names which will be interpreted as test
- methods. The default value is ``'test'``.
+ Called when the test case *test* was marked with the
+ :func:`expectedFailure` decorator, but succeeded.
- This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
- methods.
+ The default implementation appends the test to the instance's
+ :attr:`unexpectedSuccesses` attribute.
-.. attribute:: TestLoader.sortTestMethodsUsing
+.. data:: defaultTestLoader
- Function to be used to compare method names when sorting them in
- :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
- default value is the built-in :func:`cmp` function; the attribute can also be
- set to :const:`None` to disable the sort.
+ Instance of the :class:`TestLoader` class intended to be shared. If no
+ customization of the :class:`TestLoader` is needed, this instance can be used
+ instead of repeatedly creating new instances.
-.. attribute:: TestLoader.suiteClass
+.. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
- Callable object that constructs a test suite from a list of tests. No methods on
- the resulting object are needed. The default value is the :class:`TestSuite`
- class.
+ A basic test runner implementation which prints results on standard error. It
+ has a few configurable parameters, but is essentially very simple. Graphical
+ applications which run test suites should provide alternate implementations.
- This affects all the :meth:`loadTestsFrom\*` methods.
+.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
-.. attribute:: TestLoader.classSuiteClass
+ A command-line program that runs a set of tests; this is primarily for making
+ test modules conveniently executable. The simplest use for this function is to
+ include the following line at the end of a test script::
- Callable object that constructs a test suite for the tests cases from one
- class. The default value is :class:`ClassTestSuite`.
+ if __name__ == '__main__':
+ unittest.main()
+ The *testRunner* argument can either be a test runner class or an already
+ created instance of it.