================================================================================ TESTUTILS GUIDE FOR TEST AUTHORS AND MAINTAINERS Erik Leunissen ================================================================================ INTRODUCTION ============ "testutils" is a mechanism that manages utility procs that are used by multiple test files: - it keeps them in a central place to prevent code duplication. - it provides these utility procs to test files, similar to what a Tcl package (using a namespace) does: it exports the utilities, and the test files import them. The entire mechanism is implemented in a single file "testutils.tcl". Section A of this document explains the usage of the mechanism, targeted at test authors. Section B provides a more detailed description of the innards and workings of the testutils mechanism. This information is specifically targeted at developers carrying out maintenance of the testutils mechanism. A. USING UTILITY PROCS IN TESTS AND TEST FILES ============================================== This section explains to test authors how utility procs are organized, how to use existing utility procs in a test file, and how to create new utility procs. A1. Organization of utility procs using namespaces -------------------------------------------------- The utility procs that testutils provides are grouped into functional areas. These functional areas are also called "domains", or "utility domains". They carry names such as "dialog","entry", "text", which conform more or less to names of test files in the Tk test suite. Utility procs are imported on demand by test files, using the command "testutils". (See the explanation of this command in the next section.) Utility procs for the domain "generic" are an exception to this general rule: these procs are imported into the global namespace as a standard policy. They are readily available to the test author, in each test file. Each domain has its own namespace below ::tk::test in which utility procs are defined. For example: utilities that are specific for Tk dialogs are stored inside the namespace ::tk::test::dialog. A2. Using existing utility procs in test files ---------------------------------------------- The command "testutils" is the interface to the testutils mechanism for the test author. The test author may use it to import utility procs into the namespace in which tests are executed (normally, this is the global namespace). The command takes the following form: testutils (import|forget) domain ?domain domain ...? The command "testutils import" is typically invoked at the top of a test file. The command "testutils forget" is typically invoked at the end of a test file. These commands take care of the importing and cleaning up of utility procs for a specific domain. They also take care of importing any namespace variables associated with these procs so that they can be accessed from within a test. Typical invocations in a test file (using the domain "dialog" as an example), are: ┃ testutils import dialog ┃ ⋮ ┃ test foo-1.0 -body { ┃ ⋮ ┃ ⋮ ┃ SendButtonPress; # invoke utility proc imported from domain "dialog" ┃ ⋮ ┃ ⋮ ┃ } -result {foo_result} ┃ ⋮ ┃ testutils forget dialog The command "testutils import" fails if a proc or variable, unrelated to the testutils mechanism, but having the same name as a utility proc or associated variable, was already defined in the importing namespace. Therefore, test authors need to take care that such procs and variables are cleaned up before the end of a test file. A3. Adding new utility procs ---------------------------- Test authors may define new utility procs in the file "testutils.tcl". When doing so, there are several points to be aware of: 1. Consider whether the new utility proc is used in multiple test files. If it's not, it may as well be defined inside the specific test file that uses it, because in that case the issue of code duplication doesn't exist. 2. Add the proc definition to the proper domain namespace. If necessary, create a new domain namespace. 3. It may be the case that tests need to access (read/write) variables that are associated with the new utility proc. The command "testutils" also handles the importing and initialization of these associated variables, but attention is needed for the following: Their definition needs to be to placed in the reserved proc "init" (inside the proper domain namespace). The command "testutils import" will import any variables defined in this proc into the namespace where tests are executing. Note that just placing associated namespace variables inside the "namespace eval" scope for the specific domain, but outside the init proc, isn't good enough because that foregoes the importing of the namespace variables as well as their automatic initialization. Also: any namespace variables initialized inside the "namespace eval" scope for the specific domain, but outside the init proc, will NOT be cleaned up upon the invocation of "testutils forget", in contrast to imported namespace variables. 4. If you created a new domain namespace in step 2, then export the test utilities using the command "testutils export". This ensures that all utility procs in the domain namespace are exported, except any init proc. The file testutils.tcl contains various examples showing this practice. B. INNER WORKINGS OF THE TESTUTILS MECHANISM ============================================ This section is targeted at developers carrying out maintenance of the testutils mechanism, whether debugging or improving it otherwise. B1. Files and file loading -------------------------- The entire testutils mechanism is implemented in a single file "testutils.tcl". This file is sourced by the file "main.tcl", which in turn is sourced by each testfile. B2. Importing procs and associated namespace variables ------------------------------------------------------ The command "testutils" makes utility procs available to the namespace in which test files execute. The command employs a plain "namespace export/namespace import" for importing procs; there is nothing special about that. However, special attention is needed for those utility procs that store state in a namespace variable that is also accessed from the namespace in which tests are executing. Such variables are made available to the namespace in which tests are executing using an upvar statement. The process of importing these associated namespace variables needs to handle some specifics: Besides making them available to test files, some tests require such variables to be initialized, regardless whatever the previous test file did to them. Therefore, the proc "testutils" needs to re-initialize these upvar'ed variables for each test file that imports them. The steps in this auto-initialization process are as follows: - if a namespace for a specific functional area holds a proc "init", the command "testutils import xxx" will invoke it, thus initializing namespace variables. It subsequently imports the variables into the namespace where tests are executing, using "upvar"; - upon test file cleanup, "testutils forget xxx" removes the imported utility procs and unsets the upvar'ed variables. (Note that this doesn't remove the upvar link in the source namespace.) When a subsequent test file invokes "testutils import xxx" again, the command will re-initialize the namespace variables. A typical init proc (for a fictitious domain "cuisine") is: proc init {} { variable doneNess medium-rare variable seasonings [list salt pepper] variable tasteVerdict } Note that the namespace variables "doneNess" and "seasonings" are initialized with a value, while the namespace variable "tasteVerdict" is not. Both variants of declaring/defining a namespace variable are supported. B3. Tricky aspects of repeated initialization (in mode "-singleproc 1") ----------------------------------------------------------------------- While the entire Tk test suite is running, many test files are loaded, each of which may import and subsequently forget utility domains. When tracking a single utility domain across test files that come and go, associated namespace variables may be imported, initialized and cleaned up repeatedly. This repetitive cycle presents tricky aspects for the re-initialization of those namespace variables that were declared using the "variable" command without supplying a value. This is caused by the fact that, once established, the upvar link for imported namespace variables cannot be removed. The tricky details are explicitly described by comments in the proc testutils. Another tricky detail - that testutils currently evades - is the fact that unsetting an upvar'ed namespace variable changes its visibility for "info vars" in the utility namespace where the variable was defined, but not in the namespace where the upvar statement was invoked. B4. Test file ------------- The correct functioning of the testutils mechanism is tested by the test file "testutils.test".