diff options
author | Zack Galbreath <zack.galbreath@kitware.com> | 2021-04-21 18:05:11 (GMT) |
---|---|---|
committer | Zack Galbreath <zack.galbreath@kitware.com> | 2021-04-27 19:00:26 (GMT) |
commit | b60789a758ddd755be635ae60b24364ecf7fdd34 (patch) | |
tree | 0378c90a11164197f2c039473187f514ee9c62d5 /Help | |
parent | 63b5ddcce2acff2a6496e6bed737cf6e2788bfce (diff) | |
download | CMake-b60789a758ddd755be635ae60b24364ecf7fdd34.zip CMake-b60789a758ddd755be635ae60b24364ecf7fdd34.tar.gz CMake-b60789a758ddd755be635ae60b24364ecf7fdd34.tar.bz2 |
Help: Document CTest custom test measurements
Diffstat (limited to 'Help')
-rw-r--r-- | Help/command/ctest_test.rst | 80 | ||||
-rw-r--r-- | Help/manual/ctest.1.rst | 2 |
2 files changed, 82 insertions, 0 deletions
diff --git a/Help/command/ctest_test.rst b/Help/command/ctest_test.rst index 9081b3f..c61d01e 100644 --- a/Help/command/ctest_test.rst +++ b/Help/command/ctest_test.rst @@ -170,3 +170,83 @@ The options are: See also the :variable:`CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE` and :variable:`CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE` variables. + +.. _`Additional Test Measurements`: + +Additional Test Measurements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +CTest can parse the output of your tests for extra measurements to report +to CDash. + +When run as a :ref:`Dashboard Client`, CTest will include these custom +measurements in the ``Test.xml`` file that gets uploaded to CDash. + +Check the `CDash test measurement documentation +<https://github.com/Kitware/CDash/blob/master/docs/test_measurements.md>`_ +for more information on the types of test measurements that CDash recognizes. + +The following example demonstrates how to output a variety of custom test +measurements. + +.. code-block:: c++ + + std::cout << + "<DartMeasurement type=\"numeric/double\" name=\"score\">28.3</DartMeasurement>" + << std::endl; + + std::cout << + "<DartMeasurement type=\"text/string\" name=\"color\">red</DartMeasurement>" + << std::endl; + + std::cout << + "<DartMeasurement type=\"text/link\" name=\"CMake URL\">https://cmake.org</DartMeasurement>" + << std::endl; + + std::cout << + "<DartMeasurement type=\"text/preformatted\" name=\"Console Output\">" << + "line 1.\n" << + " \033[31;1m line 2. Bold red, and indented!\033[0;0ml\n" << + "line 3. Not bold or indented...\n" << + "</DartMeasurement>" << std::endl; + +Image Measurements +"""""""""""""""""" + +The following example demonstrates how to upload test images to CDash. + +.. code-block:: c++ + + std::cout << + "<DartMeasurementFile type=\"image/jpg\" name=\"TestImage\">" << + "/dir/to/test_img.jpg</DartMeasurementFile>" << std::endl; + + std::cout << + "<DartMeasurementFile type=\"image/gif\" name=\"ValidImage\">" << + "/dir/to/valid_img.gif</DartMeasurementFile>" << std::endl; + + std::cout << + "<DartMeasurementFile type=\"image/png\" name=\"AlgoResult\"> << + "/dir/to/img.png</DartMeasurementFile>" + << std::endl; + +Images will be displayed together in an interactive comparison mode on CDash +if they are provided with two or more of the following names. + +* ``TestImage`` +* ``ValidImage`` +* ``BaselineImage`` +* ``DifferenceImage2`` + +By convention, ``TestImage`` is the image generated by your test, and +``ValidImage`` (or ``BaselineImage``) is basis of comparison used to determine +if the test passed or failed. + +If another image name is used it will be displayed by CDash as a static image +separate from the interactive comparison UI. + +Attached Files +"""""""""""""" + +To associate other types of files with a test, use the +:prop_test:`ATTACHED_FILES` or :prop_test:`ATTACHED_FILES_ON_FAIL` test properties. diff --git a/Help/manual/ctest.1.rst b/Help/manual/ctest.1.rst index e35e162..811997c 100644 --- a/Help/manual/ctest.1.rst +++ b/Help/manual/ctest.1.rst @@ -1095,6 +1095,8 @@ Configuration settings include: * `CTest Script`_ variable: :variable:`CTEST_TEST_TIMEOUT` * :module:`CTest` module variable: ``DART_TESTING_TIMEOUT`` +To report extra test values to CDash, see :ref:`Additional Test Measurements`. + .. _`CTest Coverage Step`: CTest Coverage Step |