diff options
Diffstat (limited to 'tests/testutils.GUIDE')
| -rw-r--r-- | tests/testutils.GUIDE | 190 |
1 files changed, 190 insertions, 0 deletions
diff --git a/tests/testutils.GUIDE b/tests/testutils.GUIDE new file mode 100644 index 0000000..5125f5e --- /dev/null +++ b/tests/testutils.GUIDE @@ -0,0 +1,190 @@ +================================================================================ + TESTUTILS GUIDE FOR TEST AUTHORS AND MAINTAINERS + + Erik Leunissen +================================================================================ + + +INTRODUCTION +============ +"testutils" is a mechanism that manages utility procs that are used by multiple +test files: +- it keeps them in a central place to prevent code duplication. +- it provides these utility procs to test files, similar to what a Tcl package + (using a namespace) does: it exports the utilities, and the test files import + them. +The entire mechanism is implemented in a single file "testutils.tcl". + +Section A of this document explains the usage of the mechanism, targeted at +test authors. Section B provides a more detailed description of the innards and +workings of the testutils mechanism. This information is specifically targeted +at developers carrying out maintenance of the testutils mechanism. + + +A. USING UTILITY PROCS IN TESTS AND TEST FILES +============================================== +This section explains to test authors how utility procs are organized, how to +use existing utility procs in a test file, and how to create new utility procs. + +A1. Organization of utility procs using namespaces +-------------------------------------------------- +The utility procs that testutils provides are grouped into functional areas. +These functional areas are also called "domains", or "utility domains". They +carry names such as "dialog","entry", "text", which conform more or less to +names of test files in the Tk test suite. + +Utility procs are imported on demand by test files, using the command "testutils". +(See the explanation of this command in the next section.) Utility procs for +the domain "generic" are an exception to this general rule: these procs are +imported into the global namespace as a standard policy. They are readily +available to the test author, in each test file. + +Each domain has its own namespace below ::tk::test in which utility procs are +defined. For example: utilities that are specific for Tk dialogs are stored +inside the namespace ::tk::test::dialog. + +A2. Using existing utility procs in test files +---------------------------------------------- +The command "testutils" is the interface to the testutils mechanism for the test +author. The test author may use it to import utility procs into the namespace +in which tests are executed (normally, this is the global namespace). The command +takes the following form: + + testutils (import|forget) domain ?domain domain ...? + +The command "testutils import" is typically invoked at the top of a test file. +The command "testutils forget" is typically invoked at the end of a test file. +These commands take care of the importing and cleaning up of utility procs +for a specific domain. They also take care of importing any namespace variables +associated with these procs so that they can be accessed from within a test. + +Typical invocations in a test file (using the domain "dialog" as an example), are: + +┃ testutils import dialog +┃ ⋮ +┃ test foo-1.0 -body { +┃ ⋮ +┃ ⋮ +┃ SendButtonPress; # invoke utility proc imported from domain "dialog" +┃ ⋮ +┃ ⋮ +┃ } -result {foo_result} +┃ ⋮ +┃ testutils forget dialog + +The command "testutils import" fails if a proc or variable, unrelated to the +testutils mechanism, but having the same name as a utility proc or associated +variable, was already defined in the importing namespace. Therefore, test +authors need to take care that such procs and variables are cleaned up before +the end of a test file. + +A3. Adding new utility procs +---------------------------- +Test authors may define new utility procs in the file "testutils.tcl". When doing +so, there are several points to be aware of: + +1. Consider whether the new utility proc is used in multiple test files. If + it's not, it may as well be defined inside the specific test file that uses + it, because in that case the issue of code duplication doesn't exist. + +2. Add the proc definition to the proper domain namespace. If necessary, create + a new domain namespace. + +3. It may be the case that tests need to access (read/write) variables that are + associated with the new utility proc. The command "testutils" also handles + the importing and initialization of these associated variables, but attention + is needed for the following: + + Their definition needs to be to placed in the reserved proc "init" (inside + the proper domain namespace). The command "testutils import" will import any + variables defined in this proc into the namespace where tests are executing. + + Note that just placing associated namespace variables inside the "namespace eval" + scope for the specific domain, but outside the init proc, isn't good enough + because that foregoes the importing of the namespace variables as well as their + automatic initialization. + + Also: any namespace variables initialized inside the "namespace eval" scope + for the specific domain, but outside the init proc, will NOT be cleaned up + upon the invocation of "testutils forget", in contrast to imported + namespace variables. + +4. If you created a new domain namespace in step 2, then export the test + utilities using the command "testutils export". This ensures that all utility + procs in the domain namespace are exported, except any init proc. + +The file testutils.tcl contains various examples showing this practice. + + +B. INNER WORKINGS OF THE TESTUTILS MECHANISM +============================================ +This section is targeted at developers carrying out maintenance of the testutils +mechanism, whether debugging or improving it otherwise. + +B1. Files and file loading +-------------------------- +The entire testutils mechanism is implemented in a single file "testutils.tcl". +This file is sourced on behalf of each test file by a command in the file +"main.tcl", which in turn is loaded through the tcltest option "-loadfile" in +the file "all.tcl". + +B2. Importing procs and associated namespace variables +------------------------------------------------------ +The command "testutils" makes utility procs available to the namespace in which +test files execute. The command employs a plain "namespace export/namespace import" +for importing procs; there is nothing special about that. However, special +attention is needed for those utility procs that store state in a namespace +variable that is also accessed from the namespace in which tests are executing. +Such variables are made available to the namespace in which tests are executing +using an upvar statement. The process of importing these associated namespace +variables needs to handle some specifics: + +Besides making them available to test files, some tests require such variables +to be initialized, regardless whatever the previous test file did to them. +Therefore, the proc "testutils" needs to re-initialize these upvar'ed variables +for each test file that imports them. The steps in this auto-initialization +process are as follows: + +- if a namespace for a specific functional area holds a proc "init", the + command "testutils import xxx" will invoke it, thus initializing namespace + variables. It subsequently imports the variables into the namespace where + tests are executing, using "upvar"; +- upon test file cleanup, "testutils forget xxx" removes the imported utility + procs and unsets the upvar'ed variables. (Note that this doesn't remove the + upvar link in the source namespace.) When a subsequent test file invokes + "testutils import xxx" again, the command will re-initialize the namespace + variables. + +A typical init proc (for a fictitious domain "cuisine") is: + + proc init {} { + variable doneNess medium-rare + variable seasonings [list salt pepper] + variable tasteVerdict + } + +Note that the namespace variables "doneNess" and "seasonings" are initialized +with a value, while the namespace variable "tasteVerdict" is not. Both variants +of declaring/defining a namespace variable are supported. + +B3. Tricky aspects of repeated initialization +--------------------------------------------- +While the entire Tk test suite is running, many test files are loaded, each of +which may import and subsequently forget utility domains. When tracking a single +utility domain across test files that come and go, associated namespace variables +may be imported, initialized and cleaned up repeatedly. This repetitive cycle +presents tricky aspects for the re-initialization of those namespace variables +that were declared using the "variable" command without supplying a value. This +is caused by the fact that, once established, the upvar link for imported +namespace variables cannot be removed. The tricky details are explicitly +described by comments in the proc testutils. + +Another tricky detail - that testutils currently evades - is the fact that +unsetting an upvar'ed namespace variable changes its visibility for "info vars" +in the utility namespace where the variable was defined, but not in the namespace +where the upvar statement was invoked. + +B4. Test file +------------- +The correct functioning of the testutils mechanism is tested by the test +file "testutils.test". |