Long live Qt!

1 files changed, 84 insertions, 0 deletions

diff --git a/tests/arthur/README b/tests/arthur/README
new file mode 100644
index 0000000..0178351
--- /dev/null
+++ b/tests/arthur/README
@@ -0,0 +1,84 @@
+This is a simple regression testing framework for Arthur.
+
+There are three parts to it.
+
+1) datagenerator . which is the main part. it's used to 
+   run all the tests, create output png files,
+   measure rendering performance and output it in
+   data.xml file. datagenerator reads in
+   framework.ini to figure out where to look for tests.
+   if the framework.ini file isn't in the current directory
+   you have to specify its location with the -framework
+   parameter. by default datagenerator generates output in the
+   current directory - to specify output directory pass in
+   -output parameter followed by a directory name. if the given
+   directory doesn't exist it will be created. 
+
+   it's recommended to always specify -iterations argument (followed
+   by the number). if the argument has been specified datagenerator
+   will try to render all testcases the given amount of times yielding 
+   a lot more precise results.
+
+   if one wants to test just a specified engine -engine argument
+   should be specified (followed by the desired engine name).
+
+   individual testsuites can be run with -suite paramenter.
+   individual testcases from testcases can also be reran with
+   -testcase paramenter.
+
+   given files can also be tested individually. if the option -file
+   is specified with a SVG file engines will be tested against just
+   that file and output will be written directly to stdout (not files).
+   it's useful when optimizing one engine with particular case in
+   mind.
+
+   Example command line:
+   ./bin/datagenerator -framework data/framework.ini -output sampleout -iterations 5 -engine [ENGINE]
+
+2) htmlgenerator. which is used to generate html pages out of
+   generated output (generated by datagenerator). htmlgenerator also
+   reads in the framework.ini file. so the option -framework also
+   applies. but htmlgenerator takes a mandatory directory name as the
+   second argument. the mandatory directory name is the directory to
+   which you output the files generated by datagenerator (either
+   whatever followed the -output or the current directory)
+
+   htmlgenerator will generate html pages that can be viewed to
+   manually inspect the results.
+
+   Example command line:
+   ./bin/htmlgenerator -framework data/framework.ini sampleout
+
+3) performancediff. diffs the performance data between two output
+   directories. so assuming you already have data for all the engines,
+   you change something in an engine. to see the performance
+   difference run
+   ./bin/datagenerator -framework data/framework.ini -output newoutput -iterations 5
+   and then
+   ./bin/performancediff sampleout newoutput
+   
+   The performance results will be printed out.
+
+   The two first columns are in ms. The last one is a 
+   percentage of difference. 
+
+   After the percentage a '+' or a '-' sign followed by a 
+   number will be sometimes printed.
+
+   '+' signifies the new results were mathematically better. '-' is of
+   course just the opposite.
+   the number following '+' is the minimum rendering time the engine
+   spent rendering the given file up till this moment.
+   the number following '-' is the maximum rendering time the engine
+   spent rendering the given file up till this moment.
+
+   (if the current results are out of the scope from within maximum and
+   minimum number then the respective sign is printed out.)
+-------------------------------------------------------------------
+
+Note that the generated output directories can be copied from one
+machine to the other. htmlgenerator will generate webpages for all
+engines with valid data.xml files. So if you want add NativeWin32 and
+NativeMac engines you'll have to have someone with a windows and mac
+run datagenerator with "-engine NativeWin32" or "-engine NativeMac"
+options and then sending you the respective output directories.