diff options
Diffstat (limited to 'Doc/library/unittest.rst')
| -rw-r--r-- | Doc/library/unittest.rst | 936 |
1 files changed, 936 insertions, 0 deletions
diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst new file mode 100644 index 0000000..3d3727f --- /dev/null +++ b/Doc/library/unittest.rst @@ -0,0 +1,936 @@ + +:mod:`unittest` --- Unit testing framework +========================================== + +.. module:: unittest + :synopsis: Unit testing framework for Python. +.. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com> +.. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com> +.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> +.. sectionauthor:: Raymond Hettinger <python@rcn.com> + + +.. versionadded:: 2.1 + +The Python unit testing framework, sometimes referred to as "PyUnit," is a +Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in +turn, a Java version of Kent's Smalltalk testing framework. Each is the de +facto standard unit testing framework for its respective language. + +:mod:`unittest` supports test automation, sharing of setup and shutdown code for +tests, aggregation of tests into collections, and independence of the tests from +the reporting framework. The :mod:`unittest` module provides classes that make +it easy to support these qualities for a set of tests. + +To achieve this, :mod:`unittest` supports some important concepts: + +test fixture + A :dfn:`test fixture` represents the preparation needed to perform one or more + tests, and any associate cleanup actions. This may involve, for example, + creating temporary or proxy databases, directories, or starting a server + process. + +test case + A :dfn:`test case` is the smallest unit of testing. It checks for a specific + response to a particular set of inputs. :mod:`unittest` provides a base class, + :class:`TestCase`, which may be used to create new test cases. + +test suite + A :dfn:`test suite` is a collection of test cases, test suites, or both. It is + used to aggregate tests that should be executed together. + +test runner + A :dfn:`test runner` is a component which orchestrates the execution of tests + and provides the outcome to the user. The runner may use a graphical interface, + a textual interface, or return a special value to indicate the results of + executing the tests. + +The test case and test fixture concepts are supported through the +:class:`TestCase` and :class:`FunctionTestCase` classes; the former should be +used when creating new tests, and the latter can be used when integrating +existing test code with a :mod:`unittest`\ -driven framework. When building test +fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods +can be overridden to provide initialization and cleanup for the fixture. With +:class:`FunctionTestCase`, existing functions can be passed to the constructor +for these purposes. When the test is run, the fixture initialization is run +first; if it succeeds, the cleanup method is run after the test has been +executed, regardless of the outcome of the test. Each instance of the +:class:`TestCase` will only be used to run a single test method, so a new +fixture is created for each test. + +Test suites are implemented by the :class:`TestSuite` class. This class allows +individual tests and test suites to be aggregated; when the suite is executed, +all tests added directly to the suite and in "child" test suites are run. + +A test runner is an object that provides a single method, :meth:`run`, which +accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and +returns a result object. The class :class:`TestResult` is provided for use as +the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an +example test runner which reports test results on the standard error stream by +default. Alternate runners can be implemented for other environments (such as +graphical environments) without any need to derive from a specific class. + + +.. seealso:: + + Module :mod:`doctest` + Another test-support module with a very different flavor. + + `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_ + Kent Beck's original paper on testing frameworks using the pattern shared by + :mod:`unittest`. + + +.. _unittest-minimal-example: + +Basic example +------------- + +The :mod:`unittest` module provides a rich set of tools for constructing and +running tests. This section demonstrates that a small subset of the tools +suffice to meet the needs of most users. + +Here is a short script to test three functions from the :mod:`random` module:: + + import random + import unittest + + class TestSequenceFunctions(unittest.TestCase): + + def setUp(self): + self.seq = range(10) + + def testshuffle(self): + # make sure the shuffled sequence does not lose any elements + random.shuffle(self.seq) + self.seq.sort() + self.assertEqual(self.seq, range(10)) + + def testchoice(self): + element = random.choice(self.seq) + self.assert_(element in self.seq) + + def testsample(self): + self.assertRaises(ValueError, random.sample, self.seq, 20) + for element in random.sample(self.seq, 5): + self.assert_(element in self.seq) + + if __name__ == '__main__': + unittest.main() + +A testcase is created by subclassing :class:`unittest.TestCase`. The three +individual tests are defined with methods whose names start with the letters +``test``. This naming convention informs the test runner about which methods +represent tests. + +The crux of each test is a call to :meth:`assertEqual` to check for an expected +result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify +that an expected exception gets raised. These methods are used instead of the +:keyword:`assert` statement so the test runner can accumulate all test results +and produce a report. + +When a :meth:`setUp` method is defined, the test runner will run that method +prior to each test. Likewise, if a :meth:`tearDown` method is defined, the test +runner will invoke that method after each test. In the example, :meth:`setUp` +was used to create a fresh sequence for each test. + +The final block shows a simple way to run the tests. :func:`unittest.main` +provides a command line interface to the test script. When run from the command +line, the above script produces an output that looks like this:: + + ... + ---------------------------------------------------------------------- + Ran 3 tests in 0.000s + + OK + +Instead of :func:`unittest.main`, there are other ways to run the tests with a +finer level of control, less terse output, and no requirement to be run from the +command line. For example, the last two lines may be replaced with:: + + suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions) + unittest.TextTestRunner(verbosity=2).run(suite) + +Running the revised script from the interpreter or another script produces the +following output:: + + testchoice (__main__.TestSequenceFunctions) ... ok + testsample (__main__.TestSequenceFunctions) ... ok + testshuffle (__main__.TestSequenceFunctions) ... ok + + ---------------------------------------------------------------------- + Ran 3 tests in 0.110s + + OK + +The above examples show the most commonly used :mod:`unittest` features which +are sufficient to meet many everyday testing needs. The remainder of the +documentation explores the full feature set from first principles. + + +.. _organizing-tests: + +Organizing test code +-------------------- + +The basic building blocks of unit testing are :dfn:`test cases` --- single +scenarios that must be set up and checked for correctness. In :mod:`unittest`, +test cases are represented by instances of :mod:`unittest`'s :class:`TestCase` +class. To make your own test cases you must write subclasses of +:class:`TestCase`, or use :class:`FunctionTestCase`. + +An instance of a :class:`TestCase`\ -derived class is an object that can +completely run a single test method, together with optional set-up and tidy-up +code. + +The testing code of a :class:`TestCase` instance should be entirely self +contained, such that it can be run either in isolation or in arbitrary +combination with any number of other test cases. + +The simplest :class:`TestCase` subclass will simply override the :meth:`runTest` +method in order to perform specific testing code:: + + import unittest + + class DefaultWidgetSizeTestCase(unittest.TestCase): + def runTest(self): + widget = Widget('The widget') + self.assertEqual(widget.size(), (50, 50), 'incorrect default size') + +Note that in order to test something, we use the one of the :meth:`assert\*` or +:meth:`fail\*` methods provided by the :class:`TestCase` base class. If the +test fails, an exception will be raised, and :mod:`unittest` will identify the +test case as a :dfn:`failure`. Any other exceptions will be treated as +:dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are +caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are +caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect +function call. + +The way to run a test case will be described later. For now, note that to +construct an instance of such a test case, we call its constructor without +arguments:: + + testCase = DefaultWidgetSizeTestCase() + +Now, such test cases can be numerous, and their set-up can be repetitive. In +the above case, constructing a :class:`Widget` in each of 100 Widget test case +subclasses would mean unsightly duplication. + +Luckily, we can factor out such set-up code by implementing a method called +:meth:`setUp`, which the testing framework will automatically call for us when +we run the test:: + + import unittest + + class SimpleWidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + class DefaultWidgetSizeTestCase(SimpleWidgetTestCase): + def runTest(self): + self.failUnless(self.widget.size() == (50,50), + 'incorrect default size') + + class WidgetResizeTestCase(SimpleWidgetTestCase): + def runTest(self): + self.widget.resize(100,150) + self.failUnless(self.widget.size() == (100,150), + 'wrong size after resize') + +If the :meth:`setUp` method raises an exception while the test is running, the +framework will consider the test to have suffered an error, and the +:meth:`runTest` method will not be executed. + +Similarly, we can provide a :meth:`tearDown` method that tidies up after the +:meth:`runTest` method has been run:: + + import unittest + + class SimpleWidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + def tearDown(self): + self.widget.dispose() + self.widget = None + +If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether +:meth:`runTest` succeeded or not. + +Such a working environment for the testing code is called a :dfn:`fixture`. + +Often, many small test cases will use the same fixture. In this case, we would +end up subclassing :class:`SimpleWidgetTestCase` into many small one-method +classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and + +discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler +mechanism:: + + import unittest + + class WidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + def tearDown(self): + self.widget.dispose() + self.widget = None + + def testDefaultSize(self): + self.failUnless(self.widget.size() == (50,50), + 'incorrect default size') + + def testResize(self): + self.widget.resize(100,150) + self.failUnless(self.widget.size() == (100,150), + 'wrong size after resize') + +Here we have not provided a :meth:`runTest` method, but have instead provided +two different test methods. Class instances will now each run one of the +:meth:`test\*` methods, with ``self.widget`` created and destroyed separately +for each instance. When creating an instance we must specify the test method it +is to run. We do this by passing the method name in the constructor:: + + defaultSizeTestCase = WidgetTestCase('testDefaultSize') + resizeTestCase = WidgetTestCase('testResize') + +Test case instances are grouped together according to the features they test. +:mod:`unittest` provides a mechanism for this: the :dfn:`test suite`, +represented by :mod:`unittest`'s :class:`TestSuite` class:: + + widgetTestSuite = unittest.TestSuite() + widgetTestSuite.addTest(WidgetTestCase('testDefaultSize')) + widgetTestSuite.addTest(WidgetTestCase('testResize')) + +For the ease of running tests, as we will see later, it is a good idea to +provide in each test module a callable object that returns a pre-built test +suite:: + + def suite(): + suite = unittest.TestSuite() + suite.addTest(WidgetTestCase('testDefaultSize')) + suite.addTest(WidgetTestCase('testResize')) + return suite + +or even:: + + def suite(): + tests = ['testDefaultSize', 'testResize'] + + return unittest.TestSuite(map(WidgetTestCase, tests)) + +Since it is a common pattern to create a :class:`TestCase` subclass with many +similarly named test functions, :mod:`unittest` provides a :class:`TestLoader` +class that can be used to automate the process of creating a test suite and +populating it with individual tests. For example, :: + + suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase) + +will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and +``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method +name prefix to identify test methods automatically. + +Note that the order in which the various test cases will be run is determined by +sorting the test function names with the built-in :func:`cmp` function. + +Often it is desirable to group suites of test cases together, so as to run tests +for the whole system at once. This is easy, since :class:`TestSuite` instances +can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be +added to a :class:`TestSuite`:: + + suite1 = module1.TheTestSuite() + suite2 = module2.TheTestSuite() + alltests = unittest.TestSuite([suite1, suite2]) + +You can place the definitions of test cases and test suites in the same modules +as the code they are to test (such as :file:`widget.py`), but there are several +advantages to placing the test code in a separate module, such as +:file:`test_widget.py`: + +* The test module can be run standalone from the command line. + +* The test code can more easily be separated from shipped code. + +* There is less temptation to change test code to fit the code it tests without + a good reason. + +* Test code should be modified much less frequently than the code it tests. + +* Tested code can be refactored more easily. + +* Tests for modules written in C must be in separate modules anyway, so why not + be consistent? + +* If the testing strategy changes, there is no need to change the source code. + + +.. _legacy-unit-tests: + +Re-using old test code +---------------------- + +Some users will find that they have existing test code that they would like to +run from :mod:`unittest`, without converting every old test function to a +:class:`TestCase` subclass. + +For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class. +This subclass of :class:`TestCase` can be used to wrap an existing test +function. Set-up and tear-down functions can also be provided. + +Given the following test function:: + + def testSomething(): + something = makeSomething() + assert something.name is not None + # ... + +one can create an equivalent test case instance as follows:: + + testcase = unittest.FunctionTestCase(testSomething) + +If there are additional set-up and tear-down methods that should be called as +part of the test case's operation, they can also be provided like so:: + + testcase = unittest.FunctionTestCase(testSomething, + setUp=makeSomethingDB, + tearDown=deleteSomethingDB) + +To make migrating existing test suites easier, :mod:`unittest` supports tests +raising :exc:`AssertionError` to indicate test failure. However, it is +recommended that you use the explicit :meth:`TestCase.fail\*` and +:meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest` +may treat :exc:`AssertionError` differently. + +.. note:: + + Even though :class:`FunctionTestCase` can be used to quickly convert an existing + test base over to a :mod:`unittest`\ -based system, this approach is not + recommended. Taking the time to set up proper :class:`TestCase` subclasses will + make future test refactorings infinitely easier. + + +.. _unittest-contents: + +Classes and functions +--------------------- + + +.. class:: TestCase([methodName]) + + Instances of the :class:`TestCase` class represent the smallest testable units + in the :mod:`unittest` universe. This class is intended to be used as a base + class, with specific tests being implemented by concrete subclasses. This class + implements the interface needed by the test runner to allow it to drive the + test, and methods that the test code can use to check for and report various + kinds of failure. + + Each instance of :class:`TestCase` will run a single test method: the method + named *methodName*. If you remember, we had an earlier example that went + something like this:: + + def suite(): + suite = unittest.TestSuite() + suite.addTest(WidgetTestCase('testDefaultSize')) + suite.addTest(WidgetTestCase('testResize')) + return suite + + Here, we create two instances of :class:`WidgetTestCase`, each of which runs a + single test. + + *methodName* defaults to ``'runTest'``. + + +.. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]]) + + This class implements the portion of the :class:`TestCase` interface which + allows the test runner to drive the test, but does not provide the methods which + test code can use to check and report errors. This is used to create test cases + using legacy test code, allowing it to be integrated into a :mod:`unittest`\ + -based test framework. + + +.. class:: TestSuite([tests]) + + This class represents an aggregation of individual tests cases and test suites. + The class presents the interface needed by the test runner to allow it to be run + as any other test case. Running a :class:`TestSuite` instance is the same as + iterating over the suite, running each test individually. + + If *tests* is given, it must be an iterable of individual test cases or other + test suites that will be used to build the suite initially. Additional methods + are provided to add test cases and suites to the collection later on. + + +.. class:: TestLoader() + + This class is responsible for loading tests according to various criteria and + returning them wrapped in a :class:`TestSuite`. It can load all tests within a + given module or :class:`TestCase` subclass. + + +.. class:: TestResult() + + This class is used to compile information about which tests have succeeded and + which have failed. + + +.. data:: defaultTestLoader + + Instance of the :class:`TestLoader` class intended to be shared. If no + customization of the :class:`TestLoader` is needed, this instance can be used + instead of repeatedly creating new instances. + + +.. class:: TextTestRunner([stream[, descriptions[, verbosity]]]) + + A basic test runner implementation which prints results on standard error. It + has a few configurable parameters, but is essentially very simple. Graphical + applications which run test suites should provide alternate implementations. + + +.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]]) + + A command-line program that runs a set of tests; this is primarily for making + test modules conveniently executable. The simplest use for this function is to + include the following line at the end of a test script:: + + if __name__ == '__main__': + unittest.main() + + The *testRunner* argument can either be a test runner class or an already + created instance of it. + +In some cases, the existing tests may have been written using the :mod:`doctest` +module. If so, that module provides a :class:`DocTestSuite` class that can +automatically build :class:`unittest.TestSuite` instances from the existing +:mod:`doctest`\ -based tests. + +.. versionadded:: 2.3 + + +.. _testcase-objects: + +TestCase Objects +---------------- + +Each :class:`TestCase` instance represents a single test, but each concrete +subclass may be used to define multiple tests --- the concrete class represents +a single test fixture. The fixture is created and cleaned up for each test +case. + +:class:`TestCase` instances provide three groups of methods: one group used to +run the test, another used by the test implementation to check conditions and +report failures, and some inquiry methods allowing information about the test +itself to be gathered. + +Methods in the first group (running the test) are: + + +.. method:: TestCase.setUp() + + Method called to prepare the test fixture. This is called immediately before + calling the test method; any exception raised by this method will be considered + an error rather than a test failure. The default implementation does nothing. + + +.. method:: TestCase.tearDown() + + Method called immediately after the test method has been called and the result + recorded. This is called even if the test method raised an exception, so the + implementation in subclasses may need to be particularly careful about checking + internal state. Any exception raised by this method will be considered an error + rather than a test failure. This method will only be called if the + :meth:`setUp` succeeds, regardless of the outcome of the test method. The + default implementation does nothing. + + +.. method:: TestCase.run([result]) + + Run the test, collecting the result into the test result object passed as + *result*. If *result* is omitted or :const:`None`, a temporary result object is + created (by calling the :meth:`defaultTestCase` method) and used; this result + object is not returned to :meth:`run`'s caller. + + The same effect may be had by simply calling the :class:`TestCase` instance. + + +.. method:: TestCase.debug() + + Run the test without collecting the result. This allows exceptions raised by + the test to be propagated to the caller, and can be used to support running + tests under a debugger. + +The test code can use any of the following methods to check for and report +failures. + + +.. method:: TestCase.assert_(expr[, msg]) + TestCase.failUnless(expr[, msg]) + + Signal a test failure if *expr* is false; the explanation for the error will be + *msg* if given, otherwise it will be :const:`None`. + + +.. method:: TestCase.assertEqual(first, second[, msg]) + TestCase.failUnlessEqual(first, second[, msg]) + + Test that *first* and *second* are equal. If the values do not compare equal, + the test will fail with the explanation given by *msg*, or :const:`None`. Note + that using :meth:`failUnlessEqual` improves upon doing the comparison as the + first parameter to :meth:`failUnless`: the default value for *msg* can be + computed to include representations of both *first* and *second*. + + +.. method:: TestCase.assertNotEqual(first, second[, msg]) + TestCase.failIfEqual(first, second[, msg]) + + Test that *first* and *second* are not equal. If the values do compare equal, + the test will fail with the explanation given by *msg*, or :const:`None`. Note + that using :meth:`failIfEqual` improves upon doing the comparison as the first + parameter to :meth:`failUnless` is that the default value for *msg* can be + computed to include representations of both *first* and *second*. + + +.. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]]) + TestCase.failUnlessAlmostEqual(first, second[, places[, msg]]) + + Test that *first* and *second* are approximately equal by computing the + difference, rounding to the given number of *places*, and comparing to zero. + Note that comparing a given number of decimal places is not the same as + comparing a given number of significant digits. If the values do not compare + equal, the test will fail with the explanation given by *msg*, or :const:`None`. + + +.. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]]) + TestCase.failIfAlmostEqual(first, second[, places[, msg]]) + + Test that *first* and *second* are not approximately equal by computing the + difference, rounding to the given number of *places*, and comparing to zero. + Note that comparing a given number of decimal places is not the same as + comparing a given number of significant digits. If the values do not compare + equal, the test will fail with the explanation given by *msg*, or :const:`None`. + + +.. method:: TestCase.assertRaises(exception, callable, ...) + TestCase.failUnlessRaises(exception, callable, ...) + + Test that an exception is raised when *callable* is called with any positional + or keyword arguments that are also passed to :meth:`assertRaises`. The test + passes if *exception* is raised, is an error if another exception is raised, or + fails if no exception is raised. To catch any of a group of exceptions, a tuple + containing the exception classes may be passed as *exception*. + + +.. method:: TestCase.failIf(expr[, msg]) + + The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. This + signals a test failure if *expr* is true, with *msg* or :const:`None` for the + error message. + + +.. method:: TestCase.fail([msg]) + + Signals a test failure unconditionally, with *msg* or :const:`None` for the + error message. + + +.. attribute:: TestCase.failureException + + This class attribute gives the exception raised by the :meth:`test` method. If + a test framework needs to use a specialized exception, possibly to carry + additional information, it must subclass this exception in order to "play fair" + with the framework. The initial value of this attribute is + :exc:`AssertionError`. + +Testing frameworks can use the following methods to collect information on the +test: + + +.. method:: TestCase.countTestCases() + + Return the number of tests represented by this test object. For + :class:`TestCase` instances, this will always be ``1``. + + +.. method:: TestCase.defaultTestResult() + + Return an instance of the test result class that should be used for this test + case class (if no other result instance is provided to the :meth:`run` method). + + For :class:`TestCase` instances, this will always be an instance of + :class:`TestResult`; subclasses of :class:`TestCase` should override this as + necessary. + + +.. method:: TestCase.id() + + Return a string identifying the specific test case. This is usually the full + name of the test method, including the module and class name. + + +.. method:: TestCase.shortDescription() + + Returns a one-line description of the test, or :const:`None` if no description + has been provided. The default implementation of this method returns the first + line of the test method's docstring, if available, or :const:`None`. + + +.. _testsuite-objects: + +TestSuite Objects +----------------- + +:class:`TestSuite` objects behave much like :class:`TestCase` objects, except +they do not actually implement a test. Instead, they are used to aggregate +tests into groups of tests that should be run together. Some additional methods +are available to add tests to :class:`TestSuite` instances: + + +.. method:: TestSuite.addTest(test) + + Add a :class:`TestCase` or :class:`TestSuite` to the suite. + + +.. method:: TestSuite.addTests(tests) + + Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` + instances to this test suite. + + This is equivalent to iterating over *tests*, calling :meth:`addTest` for each + element. + +:class:`TestSuite` shares the following methods with :class:`TestCase`: + + +.. method:: TestSuite.run(result) + + Run the tests associated with this suite, collecting the result into the test + result object passed as *result*. Note that unlike :meth:`TestCase.run`, + :meth:`TestSuite.run` requires the result object to be passed in. + + +.. method:: TestSuite.debug() + + Run the tests associated with this suite without collecting the result. This + allows exceptions raised by the test to be propagated to the caller and can be + used to support running tests under a debugger. + + +.. method:: TestSuite.countTestCases() + + Return the number of tests represented by this test object, including all + individual tests and sub-suites. + +In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is +invoked by a :class:`TestRunner` rather than by the end-user test harness. + + +.. _testresult-objects: + +TestResult Objects +------------------ + +A :class:`TestResult` object stores the results of a set of tests. The +:class:`TestCase` and :class:`TestSuite` classes ensure that results are +properly recorded; test authors do not need to worry about recording the outcome +of tests. + +Testing frameworks built on top of :mod:`unittest` may want access to the +:class:`TestResult` object generated by running a set of tests for reporting +purposes; a :class:`TestResult` instance is returned by the +:meth:`TestRunner.run` method for this purpose. + +:class:`TestResult` instances have the following attributes that will be of +interest when inspecting the results of running a set of tests: + + +.. attribute:: TestResult.errors + + A list containing 2-tuples of :class:`TestCase` instances and strings holding + formatted tracebacks. Each tuple represents a test which raised an unexpected + exception. + + .. versionchanged:: 2.2 + Contains formatted tracebacks instead of :func:`sys.exc_info` results. + + +.. attribute:: TestResult.failures + + A list containing 2-tuples of :class:`TestCase` instances and strings holding + formatted tracebacks. Each tuple represents a test where a failure was + explicitly signalled using the :meth:`TestCase.fail\*` or + :meth:`TestCase.assert\*` methods. + + .. versionchanged:: 2.2 + Contains formatted tracebacks instead of :func:`sys.exc_info` results. + + +.. attribute:: TestResult.testsRun + + The total number of tests run so far. + + +.. method:: TestResult.wasSuccessful() + + Returns :const:`True` if all tests run so far have passed, otherwise returns + :const:`False`. + + +.. method:: TestResult.stop() + + This method can be called to signal that the set of tests being run should be + aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to + :const:`True`. :class:`TestRunner` objects should respect this flag and return + without running any additional tests. + + For example, this feature is used by the :class:`TextTestRunner` class to stop + the test framework when the user signals an interrupt from the keyboard. + Interactive tools which provide :class:`TestRunner` implementations can use this + in a similar manner. + +The following methods of the :class:`TestResult` class are used to maintain the +internal data structures, and may be extended in subclasses to support +additional reporting requirements. This is particularly useful in building +tools which support interactive reporting while tests are being run. + + +.. method:: TestResult.startTest(test) + + Called when the test case *test* is about to be run. + + The default implementation simply increments the instance's ``testsRun`` + counter. + + +.. method:: TestResult.stopTest(test) + + Called after the test case *test* has been executed, regardless of the outcome. + + The default implementation does nothing. + + +.. method:: TestResult.addError(test, err) + + Called when the test case *test* raises an unexpected exception *err* is a tuple + of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. + + The default implementation appends ``(test, err)`` to the instance's ``errors`` + attribute. + + +.. method:: TestResult.addFailure(test, err) + + Called when the test case *test* signals a failure. *err* is a tuple of the form + returned by :func:`sys.exc_info`: ``(type, value, traceback)``. + + The default implementation appends ``(test, err)`` to the instance's + ``failures`` attribute. + + +.. method:: TestResult.addSuccess(test) + + Called when the test case *test* succeeds. + + The default implementation does nothing. + + +.. _testloader-objects: + +TestLoader Objects +------------------ + +The :class:`TestLoader` class is used to create test suites from classes and +modules. Normally, there is no need to create an instance of this class; the +:mod:`unittest` module provides an instance that can be shared as +``unittest.defaultTestLoader``. Using a subclass or instance, however, allows +customization of some configurable properties. + +:class:`TestLoader` objects have the following methods: + + +.. method:: TestLoader.loadTestsFromTestCase(testCaseClass) + + Return a suite of all tests cases contained in the :class:`TestCase`\ -derived + :class:`testCaseClass`. + + +.. method:: TestLoader.loadTestsFromModule(module) + + Return a suite of all tests cases contained in the given module. This method + searches *module* for classes derived from :class:`TestCase` and creates an + instance of the class for each test method defined for the class. + + .. warning:: + + While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient + in sharing fixtures and helper functions, defining test methods on base classes + that are not intended to be instantiated directly does not play well with this + method. Doing so, however, can be useful when the fixtures are different and + defined in subclasses. + + +.. method:: TestLoader.loadTestsFromName(name[, module]) + + Return a suite of all tests cases given a string specifier. + + The specifier *name* is a "dotted name" that may resolve either to a module, a + test case class, a test method within a test case class, a :class:`TestSuite` + instance, or a callable object which returns a :class:`TestCase` or + :class:`TestSuite` instance. These checks are applied in the order listed here; + that is, a method on a possible test case class will be picked up as "a test + method within a test case class", rather than "a callable object". + + For example, if you have a module :mod:`SampleTests` containing a + :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test + methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the + specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a + suite which will run all three test methods. Using the specifier + ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite + which will run only the :meth:`test_two` test method. The specifier can refer + to modules and packages which have not been imported; they will be imported as a + side-effect. + + The method optionally resolves *name* relative to the given *module*. + + +.. method:: TestLoader.loadTestsFromNames(names[, module]) + + Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than + a single name. The return value is a test suite which supports all the tests + defined for each name. + + +.. method:: TestLoader.getTestCaseNames(testCaseClass) + + Return a sorted sequence of method names found within *testCaseClass*; this + should be a subclass of :class:`TestCase`. + +The following attributes of a :class:`TestLoader` can be configured either by +subclassing or assignment on an instance: + + +.. attribute:: TestLoader.testMethodPrefix + + String giving the prefix of method names which will be interpreted as test + methods. The default value is ``'test'``. + + This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` + methods. + + +.. attribute:: TestLoader.sortTestMethodsUsing + + Function to be used to compare method names when sorting them in + :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The + default value is the built-in :func:`cmp` function; the attribute can also be + set to :const:`None` to disable the sort. + + +.. attribute:: TestLoader.suiteClass + + Callable object that constructs a test suite from a list of tests. No methods on + the resulting object are needed. The default value is the :class:`TestSuite` + class. + + This affects all the :meth:`loadTestsFrom\*` methods. + |