summaryrefslogtreecommitdiffstats
path: root/Help
diff options
context:
space:
mode:
authorBrad King <brad.king@kitware.com>2021-04-28 13:52:56 (GMT)
committerKitware Robot <kwrobot@kitware.com>2021-04-28 13:53:04 (GMT)
commit49c6d0f26199c252b493ed5f816dbcb2e9701112 (patch)
tree20d53d4d809c6e03884543489ebe55cc8e067c6d /Help
parentc58ec3209bbd117f8b5078f0e188d15d98cdd3fb (diff)
parentb60789a758ddd755be635ae60b24364ecf7fdd34 (diff)
downloadCMake-49c6d0f26199c252b493ed5f816dbcb2e9701112.zip
CMake-49c6d0f26199c252b493ed5f816dbcb2e9701112.tar.gz
CMake-49c6d0f26199c252b493ed5f816dbcb2e9701112.tar.bz2
Merge topic 'ctest_test_measurements_docs'
b60789a758 Help: Document CTest custom test measurements 63b5ddcce2 Tests: Add cases for CTest extra measurements from tests 52eac4573d Help: Fix link to cdash.org from CTest manual Acked-by: Kitware Robot <kwrobot@kitware.com> Acked-by: Zack Galbreath <zack.galbreath@kitware.com> Merge-request: !6029
Diffstat (limited to 'Help')
-rw-r--r--Help/command/ctest_test.rst80
-rw-r--r--Help/manual/ctest.1.rst4
2 files changed, 83 insertions, 1 deletions
diff --git a/Help/command/ctest_test.rst b/Help/command/ctest_test.rst
index 9081b3f..c61d01e 100644
--- a/Help/command/ctest_test.rst
+++ b/Help/command/ctest_test.rst
@@ -170,3 +170,83 @@ The options are:
See also the :variable:`CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE`
and :variable:`CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE` variables.
+
+.. _`Additional Test Measurements`:
+
+Additional Test Measurements
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+CTest can parse the output of your tests for extra measurements to report
+to CDash.
+
+When run as a :ref:`Dashboard Client`, CTest will include these custom
+measurements in the ``Test.xml`` file that gets uploaded to CDash.
+
+Check the `CDash test measurement documentation
+<https://github.com/Kitware/CDash/blob/master/docs/test_measurements.md>`_
+for more information on the types of test measurements that CDash recognizes.
+
+The following example demonstrates how to output a variety of custom test
+measurements.
+
+.. code-block:: c++
+
+ std::cout <<
+ "<DartMeasurement type=\"numeric/double\" name=\"score\">28.3</DartMeasurement>"
+ << std::endl;
+
+ std::cout <<
+ "<DartMeasurement type=\"text/string\" name=\"color\">red</DartMeasurement>"
+ << std::endl;
+
+ std::cout <<
+ "<DartMeasurement type=\"text/link\" name=\"CMake URL\">https://cmake.org</DartMeasurement>"
+ << std::endl;
+
+ std::cout <<
+ "<DartMeasurement type=\"text/preformatted\" name=\"Console Output\">" <<
+ "line 1.\n" <<
+ " \033[31;1m line 2. Bold red, and indented!\033[0;0ml\n" <<
+ "line 3. Not bold or indented...\n" <<
+ "</DartMeasurement>" << std::endl;
+
+Image Measurements
+""""""""""""""""""
+
+The following example demonstrates how to upload test images to CDash.
+
+.. code-block:: c++
+
+ std::cout <<
+ "<DartMeasurementFile type=\"image/jpg\" name=\"TestImage\">" <<
+ "/dir/to/test_img.jpg</DartMeasurementFile>" << std::endl;
+
+ std::cout <<
+ "<DartMeasurementFile type=\"image/gif\" name=\"ValidImage\">" <<
+ "/dir/to/valid_img.gif</DartMeasurementFile>" << std::endl;
+
+ std::cout <<
+ "<DartMeasurementFile type=\"image/png\" name=\"AlgoResult\"> <<
+ "/dir/to/img.png</DartMeasurementFile>"
+ << std::endl;
+
+Images will be displayed together in an interactive comparison mode on CDash
+if they are provided with two or more of the following names.
+
+* ``TestImage``
+* ``ValidImage``
+* ``BaselineImage``
+* ``DifferenceImage2``
+
+By convention, ``TestImage`` is the image generated by your test, and
+``ValidImage`` (or ``BaselineImage``) is basis of comparison used to determine
+if the test passed or failed.
+
+If another image name is used it will be displayed by CDash as a static image
+separate from the interactive comparison UI.
+
+Attached Files
+""""""""""""""
+
+To associate other types of files with a test, use the
+:prop_test:`ATTACHED_FILES` or :prop_test:`ATTACHED_FILES_ON_FAIL` test properties.
diff --git a/Help/manual/ctest.1.rst b/Help/manual/ctest.1.rst
index 3d80cec..811997c 100644
--- a/Help/manual/ctest.1.rst
+++ b/Help/manual/ctest.1.rst
@@ -1095,6 +1095,8 @@ Configuration settings include:
* `CTest Script`_ variable: :variable:`CTEST_TEST_TIMEOUT`
* :module:`CTest` module variable: ``DART_TESTING_TIMEOUT``
+To report extra test values to CDash, see :ref:`Additional Test Measurements`.
+
.. _`CTest Coverage Step`:
CTest Coverage Step
@@ -1671,4 +1673,4 @@ See Also
.. include:: LINKS.txt
-.. _`CDash`: http://cdash.org/
+_`CDash`: https://cdash.org