diff options
author | Brad King <brad.king@kitware.com> | 2021-04-28 13:52:56 (GMT) |
---|---|---|
committer | Kitware Robot <kwrobot@kitware.com> | 2021-04-28 13:53:04 (GMT) |
commit | 49c6d0f26199c252b493ed5f816dbcb2e9701112 (patch) | |
tree | 20d53d4d809c6e03884543489ebe55cc8e067c6d /Help | |
parent | c58ec3209bbd117f8b5078f0e188d15d98cdd3fb (diff) | |
parent | b60789a758ddd755be635ae60b24364ecf7fdd34 (diff) | |
download | CMake-49c6d0f26199c252b493ed5f816dbcb2e9701112.zip CMake-49c6d0f26199c252b493ed5f816dbcb2e9701112.tar.gz CMake-49c6d0f26199c252b493ed5f816dbcb2e9701112.tar.bz2 |
Merge topic 'ctest_test_measurements_docs'
b60789a758 Help: Document CTest custom test measurements
63b5ddcce2 Tests: Add cases for CTest extra measurements from tests
52eac4573d Help: Fix link to cdash.org from CTest manual
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Zack Galbreath <zack.galbreath@kitware.com>
Merge-request: !6029
Diffstat (limited to 'Help')
-rw-r--r-- | Help/command/ctest_test.rst | 80 | ||||
-rw-r--r-- | Help/manual/ctest.1.rst | 4 |
2 files changed, 83 insertions, 1 deletions
diff --git a/Help/command/ctest_test.rst b/Help/command/ctest_test.rst index 9081b3f..c61d01e 100644 --- a/Help/command/ctest_test.rst +++ b/Help/command/ctest_test.rst @@ -170,3 +170,83 @@ The options are: See also the :variable:`CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE` and :variable:`CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE` variables. + +.. _`Additional Test Measurements`: + +Additional Test Measurements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +CTest can parse the output of your tests for extra measurements to report +to CDash. + +When run as a :ref:`Dashboard Client`, CTest will include these custom +measurements in the ``Test.xml`` file that gets uploaded to CDash. + +Check the `CDash test measurement documentation +<https://github.com/Kitware/CDash/blob/master/docs/test_measurements.md>`_ +for more information on the types of test measurements that CDash recognizes. + +The following example demonstrates how to output a variety of custom test +measurements. + +.. code-block:: c++ + + std::cout << + "<DartMeasurement type=\"numeric/double\" name=\"score\">28.3</DartMeasurement>" + << std::endl; + + std::cout << + "<DartMeasurement type=\"text/string\" name=\"color\">red</DartMeasurement>" + << std::endl; + + std::cout << + "<DartMeasurement type=\"text/link\" name=\"CMake URL\">https://cmake.org</DartMeasurement>" + << std::endl; + + std::cout << + "<DartMeasurement type=\"text/preformatted\" name=\"Console Output\">" << + "line 1.\n" << + " \033[31;1m line 2. Bold red, and indented!\033[0;0ml\n" << + "line 3. Not bold or indented...\n" << + "</DartMeasurement>" << std::endl; + +Image Measurements +"""""""""""""""""" + +The following example demonstrates how to upload test images to CDash. + +.. code-block:: c++ + + std::cout << + "<DartMeasurementFile type=\"image/jpg\" name=\"TestImage\">" << + "/dir/to/test_img.jpg</DartMeasurementFile>" << std::endl; + + std::cout << + "<DartMeasurementFile type=\"image/gif\" name=\"ValidImage\">" << + "/dir/to/valid_img.gif</DartMeasurementFile>" << std::endl; + + std::cout << + "<DartMeasurementFile type=\"image/png\" name=\"AlgoResult\"> << + "/dir/to/img.png</DartMeasurementFile>" + << std::endl; + +Images will be displayed together in an interactive comparison mode on CDash +if they are provided with two or more of the following names. + +* ``TestImage`` +* ``ValidImage`` +* ``BaselineImage`` +* ``DifferenceImage2`` + +By convention, ``TestImage`` is the image generated by your test, and +``ValidImage`` (or ``BaselineImage``) is basis of comparison used to determine +if the test passed or failed. + +If another image name is used it will be displayed by CDash as a static image +separate from the interactive comparison UI. + +Attached Files +"""""""""""""" + +To associate other types of files with a test, use the +:prop_test:`ATTACHED_FILES` or :prop_test:`ATTACHED_FILES_ON_FAIL` test properties. diff --git a/Help/manual/ctest.1.rst b/Help/manual/ctest.1.rst index 3d80cec..811997c 100644 --- a/Help/manual/ctest.1.rst +++ b/Help/manual/ctest.1.rst @@ -1095,6 +1095,8 @@ Configuration settings include: * `CTest Script`_ variable: :variable:`CTEST_TEST_TIMEOUT` * :module:`CTest` module variable: ``DART_TESTING_TIMEOUT`` +To report extra test values to CDash, see :ref:`Additional Test Measurements`. + .. _`CTest Coverage Step`: CTest Coverage Step @@ -1671,4 +1673,4 @@ See Also .. include:: LINKS.txt -.. _`CDash`: http://cdash.org/ +_`CDash`: https://cdash.org |