Added QTest::setBenchmarkResult() to public API.

A benchmark result can now be registered explicitly
using the QTest::setBenchmarkResult() function as
an alternative to using a QBENCHMARK* macro.
This gives benchmark writers the freedom to compute
the benchmark result however they like and not be
dependent on the automatic iteration/accumulation
algorithm of a QBENCHMARK* macro.

The function takes two arguments: a value and a metric.
The value must be a non-negative real, while the metric
is an enum type.

WARNING: The set of available metrics include metrics
for which a high value is considered better
(i.e. faster) than a low value (one example is
'frames per second'). QTestLib currently does not
distinguish between the two cases of "higher is better"
and "lower is better", and leaves it to external
tools (parsing the QTestLib output) to recognize
them and do any necessary conversion. A tool that
plots benchmark result history could e.g. convert
all "higher is better" to "lower is better". For
the 'frames per second' case, this would typically
mean converting FPS to SPF (= 1/FPS).

Reviewed-by: MortenS

1 files changed, 5 insertions, 0 deletions

diff --git a/src/testlib/qtestcoreelement.h b/src/testlib/qtestcoreelement.h
index b5e5d0f..ceeba52 100644
--- a/src/testlib/qtestcoreelement.h
+++ b/src/testlib/qtestcoreelement.h
@@ -45,6 +45,8 @@
 #include <QtTest/qtestcorelist.h>
 #include <QtTest/qtestelementattribute.h>
 
+#include <cstdlib>
+
 QT_BEGIN_HEADER
 
 QT_BEGIN_NAMESPACE
@@ -93,6 +95,9 @@ void QTestCoreElement<ElementType>::addAttribute(const QTest::AttributeIndex att
     if (attribute(attributeIndex))
         return;
 
+    // if (attributeIndex == QTest::AI_Metric)
+    //     abort();
+
     QTestElementAttribute *testAttribute = new QTestElementAttribute;
     testAttribute->setPair(attributeIndex, value);
     testAttribute->addToList(&listOfAttributes);