From 5462566a90e789894e65ec932a97b555e6146f1f Mon Sep 17 00:00:00 2001 From: jenn Date: Fri, 29 Sep 2000 22:47:33 +0000 Subject: * doc/tcltest2.n: Modified the new form of the test command to accept both attribute-value pairs and command line options. Updated the tests and the documentation for this new format. Also changed the option names for the test command. Fixed some typos. --- doc/tcltest2.n | 250 ++++++++++++++++++++++++++++++++------------------------- 1 file changed, 142 insertions(+), 108 deletions(-) diff --git a/doc/tcltest2.n b/doc/tcltest2.n index 86c2f48..f012195 100755 --- a/doc/tcltest2.n +++ b/doc/tcltest2.n @@ -7,7 +7,7 @@ '\" See the file "license.terms" for information on usage and redistribution '\" of this file, and for a DISCLAIMER OF ALL WARRANTIES. '\" -'\" RCS: @(#) $Id: tcltest2.n,v 1.1 2000/09/20 23:09:49 jenn Exp $ +'\" RCS: @(#) $Id: tcltest2.n,v 1.2 2000/09/29 22:47:33 jenn Exp $ '\" .so man.macros .TH "tcltest" n 8.4 Tcl "Tcl Built-In Commands" @@ -18,7 +18,8 @@ tcltest \- Test harness support code and utilities .SH SYNOPSIS \fBpackage require tcltest ?2.0?\fP .sp -\fBtcltest::test \fIname {?attribute value? ...}\fR +\fBtcltest::test \fIname desc ?option value? ?option value? ...\fR +\fBtcltest::test \fIname desc {?option value? ?option value? ...}\fR .sp \fBtcltest::cleanupTests \fI?runningMultipleTests?\fR .sp @@ -110,7 +111,9 @@ customized test harness implementation. For more defails, see the section \fI"How to Customize the Test Harness"\fR. .SH COMMANDS .TP -\fBtcltest::test\fP \fIname {?attribute value? ...}\fR +\fBtcltest::test\fP \fIname desc ?option value? ?option value? ...\fR +.TP +\fBtcltest::test\fP \fIname desc {?option value? ?option value? ...}\fR The \fBtcltest::test\fR command runs the value supplied for attribute \fIscript\fR and compares its result to possible results. It prints an error message if actual results and expected results do @@ -237,47 +240,48 @@ if the script cannot determine where the \fItests\fR directory is located. This variable should be explicitly set if tests are being run from an all.tcl file. .TP -\fBtcltest::match\fR \fI?globPatternList\fR +\fBtcltest::match\fR \fI?globPatternList?\fR Sets or returns the glob pattern list that determines which tests should be run. Only tests which match one of the glob patterns in \fIglobPatternList\fR are run by the test harness. The default value for \fIglobPatternList\fR is '*'. .TP -\fBtcltest::matchFiles\fR \fI?globPatternList\fR +\fBtcltest::matchFiles\fR \fI?globPatternList?\fR Sets or returns the glob pattern list that determines which test files should be run. Only test files which match one of the glob patterns in \fIglobPatternList\fR are run by the test harness. The default value for \fIglobPatternList\fR is '*.test'. .TP -\fBtcltest::matchDirectories\fR \fI?globPatternList\fR +\fBtcltest::matchDirectories\fR \fI?globPatternList?\fR Sets or returns the glob pattern list that determines which test subdirectories of the current test directory should be run. Only test subdirectories which match one of the glob patterns in \fIglobPatternList\fR are run by the test harness. The default value for \fIglobPatternList\fR is '*'. .TP -\fBtcltest::skip\fR \fI?globPatternList\fR +\fBtcltest::skip\fR \fI?globPatternList?\fR Sets or returns the glob pattern list that determines which tests (of those matched by tcltest::match) should be skipped. The default value for \fIglobPatternList\fR is {}. .TP -\fBtcltest::skipFiles\fR \fI?globPatternList\fR -Sets or returns the glob pattern list that determines which test files (of -those matched by tcltest::matchFiles) should be skipped. The default value -for \fIglobPatternList\fR is {}. +\fBtcltest::skipFiles\fR \fI?globPatternList?\fR +Sets or returns the glob pattern list that determines which test files +(of those matched by tcltest::matchFiles) should be skipped. The +default value for \fIglobPatternList\fR is {}. .TP -\fBtcltest::skipDirectories\fR \fI?globPatternList\fR -Sets or returns the glob pattern list that determines which test subdirectories (of -those matched by tcltest::matchDirectories) should be skipped. The default value -for \fIglobPatternList\fR is {}. +\fBtcltest::skipDirectories\fR \fI?globPatternList?\fR +Sets or returns the glob pattern list that determines which test +subdirectories (of those matched by tcltest::matchDirectories) should +be skipped. The default value for \fIglobPatternList\fR is {}. .TP \fBtcltest::loadTestedCommands\fP This command uses the script specified via the \fI-load\fR or -\fI-loadfile\fR to load the commands checked by the test suite. -Allowed to be empty, as the tested commands could have been compiled -into the interpreter running the test suite. +\fI-loadfile\fR options or the tcltest::loadScript or +tcltest::loadFile procs to load the commands checked by the test suite. +It is allowed to be empty, as the tested commands could have been +compiled into the interpreter running the test suite. .TP -\fBtcltest::loadScript\fR \fI?script\fR +\fBtcltest::loadScript\fR \fI?script?\fR Sets or returns the script executed by \fBloadTestedCommands\fR. .TP \fBtcltest::loadFile\fR \fI?filename?\fR @@ -384,27 +388,44 @@ number of existing threads at completion. .SH TESTS The \fBtest\fR procedure runs a test script and prints an error message if the script's result does not match the expected result. -The following is the spec for the \fBtest\fR command: +Two syntaxes are provided for specifying the attributes of the tests. +The first uses a separate argument for each of the attributes and +values. The second form places all of the attributes and values +together into a single argument; the argument must have proper list +structure, with teh elements of the list being the attributes and +values. The second form makes it easy to construct multi-line +scripts, since the braces around the whole list make it unnecessary to +include a backslash at the end of each line. In the second form, no +command or variable substitutions are performed on the attribute +names. This makes the behavior of the second form different from the +first form in some cases. +.PP +The first form for the \fBtest\fR command: .DS -test \fIname\fR { - description \fIdescription\fR - ?constraints \fIkeywordList|expression\fR - ?setup \fIsetupScript\fR? - ?script \fItestScript\fR? - ?cleanup \fIcleanupScript\fR? - ?expect \fIexpectedAnswer\fR? | - ?expect {-exact \fIexpectedAnswer\fR?}? | - ?expect {-regexp \fIexpectedAnswer\fR?}? | - ?expect {-glob \fIexpectedAnswer\fR?}? - ?expect_out \fIexpectedOutput\fR? | - ?expect_out {-exact \fIexpectedOutput\fR?}? | - ?expect_out {-regexp \fIexpectedOutput\fR?}? | - ?expect_out {-glob \fIexpectedOutput\fR?}? - ?expect_err \fIexpectedError\fR? | - ?expect_err {-exact \fIexpectedError\fR?}? | - ?expect_err {-regexp \fIexpectedError\fR?}? | - ?expect_err {-glob \fIexpectedError\fR?}? - ?expect_codes \fIcodeList\fR +test \fIname\fR \fIdescription\fR + ?-constraints \fIkeywordList|expression\fR + ?-setup \fIsetupScript\fR? + ?-body \fItestScript\fR? + ?-cleanup \fIcleanupScript\fR? + ?-result \fIexpectedAnswer\fR? + ?-output \fIexpectedOutput\fR? + ?-errorOutput \fIexpectedError\fR? + ?-returnCodes \fIcodeList\fR? + ?-match \fIexact|glob|regexp\fR? +.DE +.PP +The second form for the \fBtest\fR command: +.DS +test \fIname\fR \fIdescription\fR { + ?-constraints \fIkeywordList|expression\fR + ?-setup \fIsetupScript\fR? + ?-body \fItestScript\fR? + ?-cleanup \fIcleanupScript\fR? + ?-result \fIexpectedAnswer\fR? + ?-output\fIexpectedOutput\fR? + ?-errorOutput \fIexpectedError\fR? + ?-returnCodes \fIcodeList\fR? + ?-match \fIexact|glob|regexp? } .DE The \fIname\fR argument should follow the pattern: @@ -416,16 +437,16 @@ C function or Tcl procedure being tested. For black-box tests, the target should be the name of the feature being tested. Related tests should share a major number. .PP -Valid attributes and associated values are: -.TP -\fBdescription \fIdesc\fR\fP -A value must be provided for the required \fIdescription\fR attribute. -This should be a short textual description of the test used to help humans +The \fIdescription\fR should be a short textual description of the +test. It is generally used to help humans understand the purpose of the test. The name of a Tcl or C function -being tested should be included for regression tests. If the test -case exists to reproduce a bug, include the bug ID in the description. +being tested should be included in the description for regression +tests. If the test case exists to reproduce a bug, include the bug ID +in the description. +.PP +Valid attributes and associated values are: .TP -\fBconstraints \fIkeywordList|expression\fR\fP +\fB-constraints \fIkeywordList|expression\fR\fP The optional \fIconstraints\fR attribute can be list of one or more keywords or an expression. If the \fIconstraints\fR value consists of keywords, each of these keywords being the name of a constraint @@ -438,51 +459,61 @@ constraints should be added to any tests that should not always be run. See the "Test Constraints" section for a list of built-in constraints and information on how to add your own constraints. .TP -\fBsetup \fIscript\fR\fP +\fB-setup \fIscript\fR\fP The optional \fIsetup\fR attribute indicates a script that will be run before the script indicated by the \fIscript\fR attribute. If setup fails, the test will fail. .TP -\fBscript \fIscript\fR\fP -The \fIscript\fR attribute indicates the script to run to carry out the +\fB-body \fIscript\fR\fP +The \fIbody\fR attribute indicates the script to run to carry out the test. It must return a result that can be checked for correctness. +If left unspecified, the script value will be {}. .TP -\fBcleanup \fIscript\fR\fP +\fB-cleanup \fIscript\fR\fP The optional \fIcleanup\fR attribute indicates a script that will be run after the script indicated by the \fIscript\fR attribute. If -cleanup fails, the test will fail. -.TP -\fBexpect \fI{?option? expectedValue}|expectedValue\fR\fP -The \fIexpect\fR attribute supplies the comparison value with which -the return value from script will be compared. Valid options for the -value supplied are "-regexp", "-glob", and "-exact". The default -comparison option is "-exact". -.TP -\fBexpect_out \fI{?option? expectedValue}|expectedValue\fR\fP -The \fIexpect_out\fR attribute supplies the comparison value with which +cleanup fails, the test will fail. +.TP +\fB-match \fIregexp|glob|exact\fP +The \fImatch\fR attribute determines how expected answers supplied in +\fIresult\fR, \fIoutput\fR, and \fRerrorOutput\fR are compared. Valid +options for the value supplied are "regexp", "glob", and +"exact". If \fImatch\fR is not specified, the comparisons will be +done in "exact" mode by default. +.TP +\fB-result \fIexpectedValue\fR\fP +The \fIresult\fR attribute supplies the comparison value with which +the return value from script will be compared. +If left unspecified, the default +\fIexpectedValue\fR will be the empty list. +.TP +\fB-output \fIexpectedValue\fR\fP +The \fIoutput\fR attribute supplies the comparison value with which any output sent to stdout or tcltest::outputChannel during the script -run will be compared. Valid options for the -value supplied are "-regexp", "-glob", and "-exact". The default -comparison option is "-exact". Note that only output printed using -puts is used for comparison. +run will be compared. Note that only output printed using +puts is used for comparison. If \fIoutput\fR is not specified, output +sent to stdout and tcltest::outputChannel is not processed for comparison. .TP -\fBexpect_err \fI{?option? expectedValue}|expectedValue\fR\fP -The \fIexpect_err\fR attribute supplies the comparison value with which +\fB-errorOutut \fIexpectedValue\fR\fP +The \fIerrorOutput\fR attribute supplies the comparison value with which any output sent to stderr or tcltest::errorChannel during the script -run will be compared. Valid options for the -value supplied are "-regexp", "-glob", and "-exact". The default -comparison option is "-exact". Note that only output printed using -puts is used for comparison. +run will be compared. Note that only output printed using +puts is used for comparison. If \fIerrorOutut\fR is not specified, output +sent to stderr and tcltest::errorChannel is not processed for comparison. .TP -\fBexpect_codes \fIexpectedCodes\fR\fP -The optional \fIexpect_code\fR attribute indicates which return codes +\fB-returnCodes \fIexpectedCodeList\fR\fP +The optional \fIreturnCodes\fR attribute indicates which return codes from the script supplied with the \fIscript\fR attribute are correct. -Default values for \fIexpectedCodes\fR are 0 (normal return) and 2 -(return exception). +Default values for \fIexpectedCodeList\fR are 0 (normal return) and 2 +(return exception). Symbolic values \fInormal\fR (0), \fIerror\fR +(1), \fIreturn\fR (2), \fIbreak\fR (3), and \fIcontinue\fR (4) can be +used in the \fIexpectedCodeList\fR list. .PP -The argument will be compared against the result of -evaluating the