1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
|
================================================================================
TESTUTILS GUIDE FOR TEST AUTHORS AND MAINTAINERS
Erik Leunissen
================================================================================
INTRODUCTION
============
"testutils" is a mechanism that manages utility procs that are used by multiple
test files:
- it keeps them in a central place to prevent code duplication.
- it provides these utility procs to test files, similar to what a Tcl package
(using a namespace) does: it exports the utilities, and the test files import
them.
The entire mechanism is implemented in a single file "testutils.tcl".
Section A of this document explains the usage of the mechanism, targeted at
test authors. Section B provides a more detailed description of the innards and
workings of the testutils mechanism. This information is specifically targeted
at developers carrying out maintenance of the testutils mechanism.
A. USING UTILITY PROCS IN TESTS AND TEST FILES
==============================================
This section explains to test authors how utility procs are organized, how to
use existing utility procs in a test file, and how to create new utility procs.
A1. Organization of utility procs using namespaces
--------------------------------------------------
The utility procs that testutils provides are grouped into functional areas.
These functional areas are also called "domains", or "utility domains". They
carry names such as "dialog","entry", "text", which conform more or less to
names of test files in the Tk test suite.
Utility procs are imported on demand by test files, using the command "testutils".
(See the explanation of this command in the next section.) Utility procs for
the domain "generic" are an exception to this general rule: these procs are
imported into the global namespace as a standard policy. They are readily
available to the test author, in each test file.
Each domain has its own namespace below ::tk::test in which utility procs are
defined. For example: utilities that are specific for Tk dialogs are stored
inside the namespace ::tk::test::dialog.
A2. Using existing utility procs in test files
----------------------------------------------
The command "testutils" is the interface to the testutils mechanism for the test
author. The test author may use it to import utility procs into the namespace
in which tests are executed (normally, this is the global namespace). The command
takes the following form:
testutils (import|forget) domain ?domain domain ...?
The command "testutils import" is typically invoked at the top of a test file.
The command "testutils forget" is typically invoked at the end of a test file.
These commands take care of the importing and cleaning up of utility procs
for a specific domain. They also take care of importing any namespace variables
associated with these procs so that they can be accessed from within a test.
Typical invocations in a test file (using the domain "dialog" as an example), are:
┃ testutils import dialog
┃ ⋮
┃ test foo-1.0 -body {
┃ ⋮
┃ ⋮
┃ SendButtonPress; # invoke utility proc imported from domain "dialog"
┃ ⋮
┃ ⋮
┃ } -result {foo_result}
┃ ⋮
┃ testutils forget dialog
The command "testutils import" fails if a proc or variable, unrelated to the
testutils mechanism, but having the same name as a utility proc or associated
variable, was already defined in the importing namespace. Therefore, test
authors need to take care that such procs and variables are cleaned up before
the end of a test file.
A3. Adding new utility procs
----------------------------
Test authors may define new utility procs in the file "testutils.tcl". When doing
so, there are several points to be aware of:
1. Consider whether the new utility proc is used in multiple test files. If
it's not, it may as well be defined inside the specific test file that uses
it, because in that case the issue of code duplication doesn't exist.
2. Add the proc definition to the proper domain namespace. If necessary, create
a new domain namespace.
3. It may be the case that tests need to access (read/write) variables that are
associated with the new utility proc. The command "testutils" also handles
the importing and initialization of these associated variables, but attention
is needed for the following:
Their definition needs to be to placed in the reserved proc "init" (inside
the proper domain namespace). The command "testutils import" will import any
variables defined in this proc into the namespace where tests are executing.
Note that just placing associated namespace variables inside the "namespace eval"
scope for the specific domain, but outside the init proc, isn't good enough
because that foregoes the importing of the namespace variables as well as their
automatic initialization.
Also: any namespace variables initialized inside the "namespace eval" scope
for the specific domain, but outside the init proc, will NOT be cleaned up
upon the invocation of "testutils forget", in contrast to imported
namespace variables.
4. If you created a new domain namespace in step 2, then export the test
utilities using the command "testutils export". This ensures that all utility
procs in the domain namespace are exported, except any init proc.
The file testutils.tcl contains various examples showing this practice.
B. INNER WORKINGS OF THE TESTUTILS MECHANISM
============================================
This section is targeted at developers carrying out maintenance of the testutils
mechanism, whether debugging or improving it otherwise.
B1. Files and file loading
--------------------------
The entire testutils mechanism is implemented in a single file "testutils.tcl".
This file is sourced by the file "main.tcl", which in turn is sourced by
each testfile.
B2. Importing procs and associated namespace variables
------------------------------------------------------
The command "testutils" makes utility procs available to the namespace in which
test files execute. The command employs a plain "namespace export/namespace import"
for importing procs; there is nothing special about that. However, special
attention is needed for those utility procs that store state in a namespace
variable that is also accessed from the namespace in which tests are executing.
Such variables are made available to the namespace in which tests are executing
using an upvar statement. The process of importing these associated namespace
variables needs to handle some specifics:
Besides making them available to test files, some tests require such variables
to be initialized, regardless whatever the previous test file did to them.
Therefore, the proc "testutils" needs to re-initialize these upvar'ed variables
for each test file that imports them. The steps in this auto-initialization
process are as follows:
- if a namespace for a specific functional area holds a proc "init", the
command "testutils import xxx" will invoke it, thus initializing namespace
variables. It subsequently imports the variables into the namespace where
tests are executing, using "upvar";
- upon test file cleanup, "testutils forget xxx" removes the imported utility
procs and unsets the upvar'ed variables. (Note that this doesn't remove the
upvar link in the source namespace.) When a subsequent test file invokes
"testutils import xxx" again, the command will re-initialize the namespace
variables.
A typical init proc (for a fictitious domain "cuisine") is:
proc init {} {
variable doneNess medium-rare
variable seasonings [list salt pepper]
variable tasteVerdict
}
Note that the namespace variables "doneNess" and "seasonings" are initialized
with a value, while the namespace variable "tasteVerdict" is not. Both variants
of declaring/defining a namespace variable are supported.
B3. Tricky aspects of repeated initialization (in mode "-singleproc 1")
-----------------------------------------------------------------------
While the entire Tk test suite is running, many test files are loaded, each of
which may import and subsequently forget utility domains. When tracking a single
utility domain across test files that come and go, associated namespace variables
may be imported, initialized and cleaned up repeatedly. This repetitive cycle
presents tricky aspects for the re-initialization of those namespace variables
that were declared using the "variable" command without supplying a value. This
is caused by the fact that, once established, the upvar link for imported
namespace variables cannot be removed. The tricky details are explicitly
described by comments in the proc testutils.
Another tricky detail - that testutils currently evades - is the fact that
unsetting an upvar'ed namespace variable changes its visibility for "info vars"
in the utility namespace where the variable was defined, but not in the namespace
where the upvar statement was invoked.
B4. Test file
-------------
The correct functioning of the testutils mechanism is tested by the test
file "testutils.test".
|
