summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--Tests/RunCMake/CPack/README.txt168
1 files changed, 136 insertions, 32 deletions
diff --git a/Tests/RunCMake/CPack/README.txt b/Tests/RunCMake/CPack/README.txt
index 5c86880..f51d254 100644
--- a/Tests/RunCMake/CPack/README.txt
+++ b/Tests/RunCMake/CPack/README.txt
@@ -1,41 +1,75 @@
RunCMake.CPack is a test module that is intended for testing of package
generators that can be validated from command line.
-TODO: all tests should cover all packaging types (single package, grouped and
- component packaging)
-
-------------
Adding a test
-------------
-CPack test root directory: 'Tests/RunCMake/CPack'.
+CPack test root directory: 'Tests/RunCMake/CPack/tests'.
All phases are executed separately for each generator that is bound to a test.
Tests for each generator are subtests of test 'RunCMake.CPack_<generator_name>'.
Each test must also be added to 'RunCMakeTest.cmake' script located in CPack
test root directory.
+
Line that adds a test is:
-run_cpack_test(<test_name> "<generator_name>")
+run_cpack_test(<test_name> "<generator_name_list>" <compile_stage>
+ "<packaging_type_list>")
+
+<generator_name_list> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB"
+and will be run for all listed generators. In test and verification scripts
+current generator can be accessed through GENERATOR_TYPE variable.
+
+<compile_stage> is a boolean variable that enables or disables compile stage -
+most tests don't require compilation as a non binary file can be used for
+package content but sometimes an executable or a library has to be created
+before the packaging stage.
-<generator_name> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB" and
-will be run for all listed generators.
+<packaging_type_list> can be a list of one or more packaging types: MONOLITHIC,
+COMPONENT or GROUP - each type sets per generator default variables which can
+be overwritten in the test if needed
+(see <generator_type>/packaging_<packaging_type>_default.cmake for the variables
+that are set by default for each packaging type).
+Alternatively CUSTOM value can be set which means that default values will not
+be set and that values will be set manually in the test itself.
+
+Alternatively a test with subtests can be added:
+run_cpack_test_subtests(<test_name> "<subtests_list>" "<generator_name_list>"
+ <compile_stage> "<packaging_type_list>")
+
+<subtests_list> is the only new parameter and it is a list of names that will
+be used for subtests. In test and verification scripts subtest name can be
+accessed through RunCMake_SUBTEST_SUFFIX variable.
+
+Also source package tests can be added:
+run_cpack_source_test(<test_name> "<generator_name_list>" true)
Test consists of
- test prerequirements phase (optional)
- CMake execution phase
+- build phase (optional and not available for source package tests)
- CPack execution phase
- verification of generated files
+The phases are executed once per specified generator, packaging type and subtest
+combinatiion.
+
test prerequirements phase (optional):
--------------------------------------
In some cases individual tests for certain generator need additional
prerequirements met.
-In such cases '<generator_name>/<test_name>-Prerequirements.cmake' file
-containing 'function(get_test_prerequirements found_var)' should be created.
-Function should return true if all prerequirements are met.
+In such cases '<test_name>/<generator_name>-Prerequirements.cmake' file
+containing 'function(get_test_prerequirements found_var config_file)' should be
+created. Function should return true in found_var if all prerequirements are
+met. config_file variable contains the location of config file generated by the
+generator so that this function can check if prerequired variable is set in the
+file.
+NOTE: All required programs should be searched for in generator prerequirements
+function and only checked for the variable in configure file in per test
+function.
If prerequirements are not met test will be skipped outputting
'<test_name> - SKIPPED' string. Note that this doesn't fail the entire test
@@ -48,15 +82,21 @@ TODO: skipped tests should provide expected error string so test should fail
CMake execution phase:
----------------------
-To add a new CPack test we first create a <test_name>.cmake script that contains
-CMake commands that should be used as a preparation script for generation of
-different types of packages. This script is placed into CPack test root
-directory even if it will be used for only one of the generators.
+To add a new CPack test we first create a '<test_name>/test.cmake' script that
+contains CMake commands that should be used as a preparation script for
+generation of different types of packages. This script is placed into CPack
+test root directory.
If test will be used for multiple generators but some of them require some
-generator specific commands then those commands should be added to a separate
-file that should be located in '<generator_name>/<test_name>-specifics.cmake'
-in CPack test root directory.
+generator specific commands then those commands should be added to 'test.cmake'
+script wrapped with 'if(GENERATOR_TYPE STREQUAL <name_of_the_generator>)'.
+
+build phase (optional and not available for source package tests)
+-----------------------------------------------------------------
+
+This phase only runs make command.
+
+NOTE: By default all tests have CMAKE_BUILD_TYPE variable set to Debug.
CPack execution phase:
----------------------
@@ -64,6 +104,8 @@ CPack execution phase:
Only executes CPack for content that was generated during CMake execution
phase.
+NOTE: By default CPACK_PACKAGE_NAME variable is set to lower case test name.
+
Verification of generated files:
--------------------------------
@@ -76,24 +118,49 @@ contain expected files.
Mandatory verification phase also checks that no other unexpected package files
were generated (this is executed even if EXPECTED_FILES_COUNT contains 0 in
order to verify that no files were generated).
-CMake script '<generator_name>/<test_name>-ExpectedFiles.cmake' is required by
+CMake script '<test_name>/ExpectedFiles.cmake' is required by
this step and must contain
- EXPECTED_FILES_COUNT variable that contains the number of expected files that
will be generated (0 or more)
+
- EXPECTED_FILE_<file_number_starting_with_1> that contains globing expression
that uniquely defines expected file name (will be used to find expected file)
- and should be present once for each expected file
+ and should be present once for each expected file.
+ NOTE: This variable should be used only as last resort as it sets generator
+ specific globbing expression. Each generator can recreate file name from
+ parts that are already populated by default but can always be
+ overwritten if needed:
+ - EXPECTED_FILE_<file_number_starting_with_1>_NAME is the name component of
+ the file (by default it is set to package name that is same as test name
+ in lowercase)
+ - EXPECTED_FILE_<file_number_starting_with_1>_VERSION is the version of the
+ package (by default it is set to '0.1.1')
+ - EXPECTED_FILE_<file_number_starting_with_1>_REVISION is the revision of the
+ package (by default it is set to '1')
+
- EXPECTED_FILE_CONTENT_<file_number_starting_with_1> that contains regular
expression of files that should be present in generated file and should be
present once for each expected file
+ NOTE: This variable should be used only as last resort as it sets generator
+ specific regular expression.
+ EXPECTED_FILE_CONTENT_<file_number_starting_with_1>_LIST should be
+ prefered as it requires a list of expected files and directories that
+ is later changed automatically depending on the generator so expected
+ package content can be written only once per test for all generators.
Optional verification phase is generator specific and is optionaly executed.
-This phase is executed if '<generator_name>/<test_name>-VerifyResult.cmake'
-script exists.
-In case that the script doesn't exist VerifyResult.cmake script automatically
-prints out standard output and standard error from CPack execution phase that
-is compared with '<generator_name>/<test_name>-stdout.txt' regular expression
-and '<generator_name>/<test_name>-stderr.txt' regular expresson respectively.
+This phase is executed if '<test_name>/VerifyResult.cmake' script exists.
+
+VerifyResult.cmake script also automatically prints out standard output and
+standard error from CPack execution phase that is compared with
+'<test_name>/<generator_name>-stdout.txt' regular expression and
+and '<test_name>/<generator_name>-stderr.txt' regular expresson respectively.
+NOTE: For subtests generator name can also be suffixed with subtest name and in
+ such case subtest file is preferred.
+ File name format: '<generator_name>-<subtest_name>-std<type>.txt' where
+ <type> can either be 'out' or 'err'.
+NOTE: If none of the comparison files are present then the default generator
+ file is used if present.
----------------------
Adding a new generator
@@ -101,13 +168,44 @@ Adding a new generator
To add a new generator we must
- add new generator directory (e.g. RPM for RPM generator) to CPack test root
- directory that contains 'Helpers.cmake' script. In this script a function
- named 'getPackageContent' must exist. This function should list files that
- are contained in a package. Function should accept two parameters
- + FILE variable that will contain path to file for which the content should be
- listed
- + RESULT_VAR that will tell the function which variable in parent scope should
- contain the result (list of files inside package file)
+ directory that contains 'Helpers.cmake' script.
+ - In this script some functions must exist:
+ - getPackageContent: This function should list files that are contained in
+ a package.
+ Function parameters:
+ + FILE variable that will contain path to file for which the content
+ should be listed
+ + RESULT_VAR that will tell the function which variable in parent scope
+ should contain the result (list of files inside package file)
+ - getPackageNameGlobexpr: This function should generate package name
+ globbing expression.
+ Function parameters:
+ + NAME that will contain the expected package name
+ + COMPONENT that will contain the expected package component
+ + VERSION that will contain the expected package version
+ + REVISION that will contain the expected package revision number
+ + FILE_NO that will contain the file number for which the globbing
+ expression is generated
+ + RESULT_VAR that will tell the function which variable in parent scope
+ should contain the result (file name globbing expression)
+ - getPackageContentList: This function should return a list of files and
+ directories that are in the provided package.
+ Function parameters:
+ + FILE that will contain the package file for which the package content
+ should be returned.
+ + RESULT_VAR that will tell the function which variable in parent scope
+ should contain the result (list of pacakge content)
+ - toExpectedContentList: This function should convert an expected package
+ content list into one that is expected for the
+ generator (e.g. rpm packages have install/relocate
+ path prefixes which aren't part of the package so
+ those paths have to be removed from the expected
+ content list).
+ Function parameters:
+ + FILE_NO that will contain the file number for which the conversion
+ should be performed
+ + CONTENT_VAR that will contain the input list and is also the variable
+ in parent scope which should contain the result (converted content list)
- add 'Prerequirements.cmake' script to generator directory. In this script a
function named 'get_test_prerequirements' must exist. This function should
set a variable in parent scope (name of the variable is the first parameter)
@@ -119,3 +217,9 @@ To add a new generator we must
- add tests the same way as described above
- add generator to 'add_RunCMake_test_group' function call that is located in
RunCMake CMakeLists.txt file
+- if needed add 'packaging_<packaging_type>_default.cmake' scripts that define
+ default variables that will be set for each packaging type (MONOLITHIC,
+ COMPONENT and GROUP)
+- if needed add 'default_expected_std<type>.txt' files where <type> is either
+ 'out' or 'err' which will contain default expected output of package
+ generation regular expression.