summaryrefslogtreecommitdiffstats
path: root/testing/README.txt
blob: 4988a07204bfd578b07d0d1a1f62014c431919e0 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
Doxygen regression test suite
============================

This directory contains a set of regression tests. Each test consists of a
file starting with a 3 digit number and a corresponding directory whose name
has the same 3 digit number. The directory contains one or more reference
files that are compared against the XML output produced by doxygen. If the
result is the same, there is no regression and the test passes. If there is a
difference the test fails and the difference (in diff -u format) will be shown.
It is also possible to see whether or not the test can be built to an xhtml set
of files (and tested against a DTD), it is also possible to create a pdf file
for each test to see if the LaTeX / pdf generation is possible.

The runtest.py script responsible for running the tests takes a number of
optional parameters:
  --updateref           update the reference data for a test
  --doxygen [DOXYGEN]   path/name of the doxygen executable
  --xmllint [XMLLINT]   path/name of the xmllint executable
  --id IDS [IDS ...]    id of the test to perform
  --all                 perform all tests
  --inputdir [INPUTDIR]
                        input directory containing the tests
  --outputdir [OUTPUTDIR]
                        output directory to write the doxygen output to
  --noredir             disable redirection of doxygen warnings
  --xml                 create xml output and check
  --xhtml               create xhtml output and check with xmllint
  --pdf                 create LaTeX output and create pdf from it
  --keep                keep result directories
  --cfg CFGS [CFGS ...]
                        run test with extra doxygen configuration settings
                        (the option may be specified multiple times
In case neither --xml, --pdf or --xhtml is used the default is set to --xml.

The runtest.pl has the following dependencies on 3rd party tools:
- python  to run the script
- xmllint to normalize the XML output
- diff    to show the differences in case a test fails

Each test file can have a number of special comment lines that are extracted by
the runtest.pl script and take the form:
// <identifier>: 'argument'
Where <identifier> can be one of:
- objective: 'argument' provides the objective for the test (i.e. its purpose)
- check:     'argument' names a file that is generated by doxygen, which should
             be compared against the reference.
- config:    'argument' is a line that is added to the default Doxyfile used to
             run doxygen on the test file.

Example to run all tests:
    python runtest.py

Example to run a test
    python runtest.py -id 10

Example to update the reference files for a test
    python runtest.py -updateref -id 10

There is also a CMakeLists.txt, which can be used from the build directory
to run all tests by simply invoking 'make tests', to use the specific options use
the flag TEST_FLAGS with make
  e.g. make tests TEST_FLAGS="--id=5 --id=10 --pdf --xhtml"