summaryrefslogtreecommitdiffstats
path: root/testing/README.txt
blob: 94528963dc2c78d121b763c01ee5c597c4af1d7e (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Doxygen regession test suite
============================

This directory contains a set of regression tests. Each test consists of a
file starting with a 3 digit number and a corresponding directory whose name
has the same 3 digit number. The directory contains one or more reference
files that are compared against the XML output produced by doxygen. If the
result is the same, there is no regression and the test passes. If there is a
difference the test fails and the difference (in diff -u format) will be shown.

The runtest.py script responsible for running the tests takes a number of
optional parameters:
-id n:        run test with number n only (the option may be specified
              multiple times) default is to run all tests.
-updateref:   update the reference files. Should be used in combination
              with -id to update the reference file(s) for the given test.
-all:         can be used in combination with -updateref to update the
              reference files for all tests.
-doxygen exe: run the specified doxygen executable.
-xmllint exe: run the specified xmllint executable.

The runtest.pl has the following dependencies on 3rd party tools:
- python  to run the script
- xmllint to normalize the XML output
- diff    to show the differences in case a test fails

Each test file can have a number of special comment lines that are extracted by
the runtest.pl script and take the form:
// <identifier>: 'argument'
Where <identifier> can be one of:
- objective: 'argument' provides the objective for the test (i.e. its purpose)
- check:     'argument' names a file that is generated by doxygen, which should
             be compared against the reference.
- config:    'argument' is a line that is added to the default Doxyfile used to
             run doxygen on the test file.

Example to run all tests:
    python runtest.py

Example to run a test
    python runtest.py -id 10

Example to update the reference files for a test
    python runtest.py -updateref -id 10

There is also a CMakeLists.txt, which can be used from the build directory
to run all tests by simply invoking 'make tests'.