diff options
| -rw-r--r-- | Tests/RunCMake/CPack/README.txt | 168 |
1 files changed, 136 insertions, 32 deletions
diff --git a/Tests/RunCMake/CPack/README.txt b/Tests/RunCMake/CPack/README.txt index 5c86880..f51d254 100644 --- a/Tests/RunCMake/CPack/README.txt +++ b/Tests/RunCMake/CPack/README.txt @@ -1,41 +1,75 @@ RunCMake.CPack is a test module that is intended for testing of package generators that can be validated from command line. -TODO: all tests should cover all packaging types (single package, grouped and - component packaging) - ------------- Adding a test ------------- -CPack test root directory: 'Tests/RunCMake/CPack'. +CPack test root directory: 'Tests/RunCMake/CPack/tests'. All phases are executed separately for each generator that is bound to a test. Tests for each generator are subtests of test 'RunCMake.CPack_<generator_name>'. Each test must also be added to 'RunCMakeTest.cmake' script located in CPack test root directory. + Line that adds a test is: -run_cpack_test(<test_name> "<generator_name>") +run_cpack_test(<test_name> "<generator_name_list>" <compile_stage> + "<packaging_type_list>") + +<generator_name_list> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB" +and will be run for all listed generators. In test and verification scripts +current generator can be accessed through GENERATOR_TYPE variable. + +<compile_stage> is a boolean variable that enables or disables compile stage - +most tests don't require compilation as a non binary file can be used for +package content but sometimes an executable or a library has to be created +before the packaging stage. -<generator_name> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB" and -will be run for all listed generators. +<packaging_type_list> can be a list of one or more packaging types: MONOLITHIC, +COMPONENT or GROUP - each type sets per generator default variables which can +be overwritten in the test if needed +(see <generator_type>/packaging_<packaging_type>_default.cmake for the variables +that are set by default for each packaging type). +Alternatively CUSTOM value can be set which means that default values will not +be set and that values will be set manually in the test itself. + +Alternatively a test with subtests can be added: +run_cpack_test_subtests(<test_name> "<subtests_list>" "<generator_name_list>" + <compile_stage> "<packaging_type_list>") + +<subtests_list> is the only new parameter and it is a list of names that will +be used for subtests. In test and verification scripts subtest name can be +accessed through RunCMake_SUBTEST_SUFFIX variable. + +Also source package tests can be added: +run_cpack_source_test(<test_name> "<generator_name_list>" true) Test consists of - test prerequirements phase (optional) - CMake execution phase +- build phase (optional and not available for source package tests) - CPack execution phase - verification of generated files +The phases are executed once per specified generator, packaging type and subtest +combinatiion. + test prerequirements phase (optional): -------------------------------------- In some cases individual tests for certain generator need additional prerequirements met. -In such cases '<generator_name>/<test_name>-Prerequirements.cmake' file -containing 'function(get_test_prerequirements found_var)' should be created. -Function should return true if all prerequirements are met. +In such cases '<test_name>/<generator_name>-Prerequirements.cmake' file +containing 'function(get_test_prerequirements found_var config_file)' should be +created. Function should return true in found_var if all prerequirements are +met. config_file variable contains the location of config file generated by the +generator so that this function can check if prerequired variable is set in the +file. +NOTE: All required programs should be searched for in generator prerequirements +function and only checked for the variable in configure file in per test +function. If prerequirements are not met test will be skipped outputting '<test_name> - SKIPPED' string. Note that this doesn't fail the entire test @@ -48,15 +82,21 @@ TODO: skipped tests should provide expected error string so test should fail CMake execution phase: ---------------------- -To add a new CPack test we first create a <test_name>.cmake script that contains -CMake commands that should be used as a preparation script for generation of -different types of packages. This script is placed into CPack test root -directory even if it will be used for only one of the generators. +To add a new CPack test we first create a '<test_name>/test.cmake' script that +contains CMake commands that should be used as a preparation script for +generation of different types of packages. This script is placed into CPack +test root directory. If test will be used for multiple generators but some of them require some -generator specific commands then those commands should be added to a separate -file that should be located in '<generator_name>/<test_name>-specifics.cmake' -in CPack test root directory. +generator specific commands then those commands should be added to 'test.cmake' +script wrapped with 'if(GENERATOR_TYPE STREQUAL <name_of_the_generator>)'. + +build phase (optional and not available for source package tests) +----------------------------------------------------------------- + +This phase only runs make command. + +NOTE: By default all tests have CMAKE_BUILD_TYPE variable set to Debug. CPack execution phase: ---------------------- @@ -64,6 +104,8 @@ CPack execution phase: Only executes CPack for content that was generated during CMake execution phase. +NOTE: By default CPACK_PACKAGE_NAME variable is set to lower case test name. + Verification of generated files: -------------------------------- @@ -76,24 +118,49 @@ contain expected files. Mandatory verification phase also checks that no other unexpected package files were generated (this is executed even if EXPECTED_FILES_COUNT contains 0 in order to verify that no files were generated). -CMake script '<generator_name>/<test_name>-ExpectedFiles.cmake' is required by +CMake script '<test_name>/ExpectedFiles.cmake' is required by this step and must contain - EXPECTED_FILES_COUNT variable that contains the number of expected files that will be generated (0 or more) + - EXPECTED_FILE_<file_number_starting_with_1> that contains globing expression that uniquely defines expected file name (will be used to find expected file) - and should be present once for each expected file + and should be present once for each expected file. + NOTE: This variable should be used only as last resort as it sets generator + specific globbing expression. Each generator can recreate file name from + parts that are already populated by default but can always be + overwritten if needed: + - EXPECTED_FILE_<file_number_starting_with_1>_NAME is the name component of + the file (by default it is set to package name that is same as test name + in lowercase) + - EXPECTED_FILE_<file_number_starting_with_1>_VERSION is the version of the + package (by default it is set to '0.1.1') + - EXPECTED_FILE_<file_number_starting_with_1>_REVISION is the revision of the + package (by default it is set to '1') + - EXPECTED_FILE_CONTENT_<file_number_starting_with_1> that contains regular expression of files that should be present in generated file and should be present once for each expected file + NOTE: This variable should be used only as last resort as it sets generator + specific regular expression. + EXPECTED_FILE_CONTENT_<file_number_starting_with_1>_LIST should be + prefered as it requires a list of expected files and directories that + is later changed automatically depending on the generator so expected + package content can be written only once per test for all generators. Optional verification phase is generator specific and is optionaly executed. -This phase is executed if '<generator_name>/<test_name>-VerifyResult.cmake' -script exists. -In case that the script doesn't exist VerifyResult.cmake script automatically -prints out standard output and standard error from CPack execution phase that -is compared with '<generator_name>/<test_name>-stdout.txt' regular expression -and '<generator_name>/<test_name>-stderr.txt' regular expresson respectively. +This phase is executed if '<test_name>/VerifyResult.cmake' script exists. + +VerifyResult.cmake script also automatically prints out standard output and +standard error from CPack execution phase that is compared with +'<test_name>/<generator_name>-stdout.txt' regular expression and +and '<test_name>/<generator_name>-stderr.txt' regular expresson respectively. +NOTE: For subtests generator name can also be suffixed with subtest name and in + such case subtest file is preferred. + File name format: '<generator_name>-<subtest_name>-std<type>.txt' where + <type> can either be 'out' or 'err'. +NOTE: If none of the comparison files are present then the default generator + file is used if present. ---------------------- Adding a new generator @@ -101,13 +168,44 @@ Adding a new generator To add a new generator we must - add new generator directory (e.g. RPM for RPM generator) to CPack test root - directory that contains 'Helpers.cmake' script. In this script a function - named 'getPackageContent' must exist. This function should list files that - are contained in a package. Function should accept two parameters - + FILE variable that will contain path to file for which the content should be - listed - + RESULT_VAR that will tell the function which variable in parent scope should - contain the result (list of files inside package file) + directory that contains 'Helpers.cmake' script. + - In this script some functions must exist: + - getPackageContent: This function should list files that are contained in + a package. + Function parameters: + + FILE variable that will contain path to file for which the content + should be listed + + RESULT_VAR that will tell the function which variable in parent scope + should contain the result (list of files inside package file) + - getPackageNameGlobexpr: This function should generate package name + globbing expression. + Function parameters: + + NAME that will contain the expected package name + + COMPONENT that will contain the expected package component + + VERSION that will contain the expected package version + + REVISION that will contain the expected package revision number + + FILE_NO that will contain the file number for which the globbing + expression is generated + + RESULT_VAR that will tell the function which variable in parent scope + should contain the result (file name globbing expression) + - getPackageContentList: This function should return a list of files and + directories that are in the provided package. + Function parameters: + + FILE that will contain the package file for which the package content + should be returned. + + RESULT_VAR that will tell the function which variable in parent scope + should contain the result (list of pacakge content) + - toExpectedContentList: This function should convert an expected package + content list into one that is expected for the + generator (e.g. rpm packages have install/relocate + path prefixes which aren't part of the package so + those paths have to be removed from the expected + content list). + Function parameters: + + FILE_NO that will contain the file number for which the conversion + should be performed + + CONTENT_VAR that will contain the input list and is also the variable + in parent scope which should contain the result (converted content list) - add 'Prerequirements.cmake' script to generator directory. In this script a function named 'get_test_prerequirements' must exist. This function should set a variable in parent scope (name of the variable is the first parameter) @@ -119,3 +217,9 @@ To add a new generator we must - add tests the same way as described above - add generator to 'add_RunCMake_test_group' function call that is located in RunCMake CMakeLists.txt file +- if needed add 'packaging_<packaging_type>_default.cmake' scripts that define + default variables that will be set for each packaging type (MONOLITHIC, + COMPONENT and GROUP) +- if needed add 'default_expected_std<type>.txt' files where <type> is either + 'out' or 'err' which will contain default expected output of package + generation regular expression. |