From 3667ec5d5acd7ccd1d1a94b148249952e3f5fef2 Mon Sep 17 00:00:00 2001 From: hershey Date: Wed, 24 Mar 1999 02:48:54 +0000 Subject: lint: changed a tcltest namespace variable name from tmpDir to workingDir. changes: made more tests atomic. README is now updated to match new test suite features. --- tests/README | 383 +++++++++++++++++++++++++++-------------------- tests/all.tcl | 5 +- tests/append.test | 3 +- tests/assocd.test | 3 +- tests/async.test | 3 +- tests/autoMkindex.tcl | 1 + tests/autoMkindex.test | 5 +- tests/basic.test | 3 +- tests/binary.test | 3 +- tests/case.test | 3 +- tests/clock.test | 3 +- tests/cmdAH.test | 3 +- tests/cmdIL.test | 3 +- tests/cmdInfo.test | 3 +- tests/cmdMZ.test | 3 +- tests/compExpr-old.test | 3 +- tests/compExpr.test | 3 +- tests/compile.test | 3 +- tests/concat.test | 3 +- tests/dcall.test | 3 +- tests/defs.tcl | 164 +++++++++----------- tests/dstring.test | 3 +- tests/encoding.test | 27 ++-- tests/env.test | 3 +- tests/error.test | 3 +- tests/eval.test | 3 +- tests/event.test | 3 +- tests/exec.test | 3 +- tests/execute.test | 3 +- tests/expr-old.test | 3 +- tests/expr.test | 3 +- tests/fCmd.test | 3 +- tests/fileName.test | 3 +- tests/for-old.test | 3 +- tests/for.test | 3 +- tests/foreach.test | 3 +- tests/format.test | 3 +- tests/get.test | 3 +- tests/history.test | 3 +- tests/http.test | 3 +- tests/httpold.test | 3 +- tests/if-old.test | 3 +- tests/if.test | 3 +- tests/incr-old.test | 3 +- tests/incr.test | 3 +- tests/indexObj.test | 3 +- tests/info.test | 3 +- tests/init.test | 3 +- tests/interp.test | 3 +- tests/io.test | 3 +- tests/ioCmd.test | 3 +- tests/ioUtil.test | 3 +- tests/join.test | 3 +- tests/lindex.test | 3 +- tests/link.test | 3 +- tests/linsert.test | 3 +- tests/list.test | 3 +- tests/listObj.test | 3 +- tests/llength.test | 3 +- tests/load.test | 3 +- tests/lrange.test | 3 +- tests/lreplace.test | 3 +- tests/lsearch.test | 3 +- tests/macFCmd.test | 3 +- tests/misc.test | 3 +- tests/msgcat.test | 3 +- tests/namespace-old.test | 3 +- tests/namespace.test | 3 +- tests/obj.test | 3 +- tests/opt.test | 3 +- tests/osa.test | 3 +- tests/parse.test | 3 +- tests/parseExpr.test | 3 +- tests/parseOld.test | 3 +- tests/pid.test | 3 +- tests/pkg.test | 3 +- tests/pkgMkIndex.test | 11 +- tests/platform.test | 1 + tests/proc-old.test | 3 +- tests/proc.test | 3 +- tests/pwd.test | 3 +- tests/reg.test | 3 +- tests/regexp.test | 3 +- tests/registry.test | 3 +- tests/remote.tcl | 3 +- tests/rename.test | 3 +- tests/resource.test | 3 +- tests/result.test | 1 + tests/safe.test | 3 +- tests/scan.test | 3 +- tests/security.test | 3 +- tests/set-old.test | 3 +- tests/set.test | 3 +- tests/socket.test | 3 +- tests/source.test | 3 +- tests/split.test | 3 +- tests/stack.test | 3 +- tests/string.test | 3 +- tests/stringObj.test | 3 +- tests/subst.test | 3 +- tests/switch.test | 3 +- tests/thread.test | 3 +- tests/timer.test | 3 +- tests/trace.test | 3 +- tests/unixFCmd.test | 5 +- tests/unixFile.test | 3 +- tests/unixInit.test | 3 +- tests/unixNotfy.test | 3 +- tests/unknown.test | 3 +- tests/uplevel.test | 3 +- tests/upvar.test | 3 +- tests/utf.test | 3 +- tests/util.test | 3 +- tests/var.test | 3 +- tests/while-old.test | 3 +- tests/while.test | 3 +- tests/winConsole.test | 3 +- tests/winFCmd.test | 3 +- tests/winFile.test | 3 +- tests/winNotify.test | 3 +- tests/winPipe.test | 3 +- 121 files changed, 548 insertions(+), 388 deletions(-) diff --git a/tests/README b/tests/README index 3df72ca..cb1ba35 100644 --- a/tests/README +++ b/tests/README @@ -1,10 +1,22 @@ README -- Tcl test suite design document. -RCS: @(#) $Id: README,v 1.1.2.3 1999/03/23 20:06:07 hershey Exp $ +RCS: @(#) $Id: README,v 1.1.2.4 1999/03/24 02:48:54 hershey Exp $ +Contents: +--------- -Introduction: -------------- + 1. Introduction + 2. Definitions file + 3. Writing a new test + 4. Constraints + 5. Adding a New Test File + 6. Test output + 7. Selecting tests for execution within a file + 8. Selecting files to be sourced by all.tcl + 9. Incompatibilities with prior Tcl versions + +1. Introduction: +---------------- This directory contains a set of validation tests for the Tcl commands and C Library procedures for Tcl. Each of the files whose name ends @@ -42,197 +54,124 @@ You can run the tests in three ways: (c) start up tcltest in this directory, then "source" the test file (for example, type "source parse.test"). To run all - of the tests, type "source all.tcl". + of the tests, type "source all.tcl". To use the options in + interactive mode, you can set their corresponding tcltest + namespace variables: + ::tcltest::matchingTests + ::tcltest::skippingTests + ::tcltest::testConfig(nonPortable) + ::tcltest::testConfig(knownBug) In all cases, no output will be generated if all goes well, except for -a listing of the tests. If there are errors then additional messages -will appear in the format described below. Note that some tests will -be skipped if you run as superuser. +a listing of the test files and a statical summary. If there are +errors then additional messages will appear in the format described +below. Note that some tests will be skipped if you run as superuser. This approach to testing was designed and initially implemented by Mary Ann May-Pumphrey of Sun Microsystems in the early 1990's. Many thanks to her for donating her work back to the public Tcl release. -The rest of this file provides additional information on the -features of the testing environment. - -Definitions file: ------------------ +2. Definitions file: +-------------------- -The file "defs.tcl" defines the "test" namespace which contains a +The file "defs.tcl" defines the "tcltest" namespace which contains a collection of procedures and variables used to run the tests. It is read in automatically by each of the .test files if needed, but once it has been read once it will not be read again by the .test files. -Currently, the following procedures are exported from the "test" +Currently, the following procedures are exported from the "tcltest" namespace and automatically imported: - test cleanupTests dotests saveState restoreState normalizeMsg - makeFile removeFile makeDirectory removeDirectory viewFile - safeFetch bytestring set_iso8859_1_locale restore_locale setTmpDir + test Run a test script. -Please refer to the defs.tcl file for documentation on these -procedures. + cleanupTests Print stats and remove files created by tests. -To keep tests from polluting the current working directory with -unwanted files, you can specify a temporary directory, which will -become the current working directory for the tests, by specifying --tmpdir on the command line or by calling the ::tcltest::setTmpDir -procedure (after sourcing the defs.tcl file). The default working dir -is the directory from which tcltest was called. Please note that when -you run the test suite by calling "make test", the working dir is no -longer automatically switched to "tests". + dotests Source a test file and run tests of the + specified pattern. + makeFile Create a file--the file will automatically + be removed by cleanupTests. -Test output: ------------- + removeFile Force a file to be removed. -For each test file, the number of tests passed, skipped, and failed is -printed to stdout. Aside from this statistical information, output -can be controlled on a per-test basis by the ::tcltest::verbose variable. + makeDirectory Create a directory--the directory will + automatically be removed by cleanupTests. -::tcltest::verbose can be set to any substring or permutation of "bps". -In the string "bps", the 'b' stands for a test's "body", the 'p' -stands for "passed" tests, and the 's' stands for "skipped" tests. -The default value of ::tcltest::verbose is "b". If 'b' is present, then -the entire body of the test is printed for each failed test, otherwise -only the test's name, desired output, and actual output, are printed -for each failed test. If 'p' is present, then a line is printed for -each passed test, otherwise no line is printed for passed tests. If -'s' is present, then a line (containing the consraints that cause the -test to be skipped) is printed for each skipped test, otherwise no -line is printed for skipped tests. + removeDirectory Force a directory to be removed. -You can set ::tcltest::verbose either interactively (after the defs.tcl -file has been sourced) or by the command line argument -verbose, for -example: + viewFile Returns the contents of a file. - tcltest socket.test -verbose bps + normalizeMsg Remove extra newlines from a string. + bytestring Construct a string that consists of the + requested sequence of bytes, as opposed to a + string of properly formed UTF-8 characters. -Selecting files to be sourced by all.tcl: ------------------------------------------ + set_iso8859_1_locale Set the locale to iso8859_1. -You can specify the files you want all.tcl to source on the command -line with the -file options. For example, if you call the -following: - - tcltest all.tcl -file "unix*.test" - -all files in "tests" directory that match the pattern unix*.test will -be sourced by the all.tcl file. Another useful example is if a -particular test hangs, say "get.test", and you just want to run the -remaining tests, then you can call the following: - - tcltest all.tcl -file "[h-z]*.test" - -Note that the argument to -file will be substituted relative to the -"tests" directory. + restore_locale Restore the locale to its original setting. + saveState Save the procedure and global variable names. -Selecting tests for execution within a file: --------------------------------------------- - -Normally, all the tests in a file are run whenever the file is -sourced. An individual test will be skipped if one of the following -conditions is met: + restoreState Restore the procedure and global variable names. - 1) the "name" of the tests does not match (using glob style - matching) one or more elements in the ::tcltest::matchingTests - variable - - 2) the "name" of the tests matches (using glob style matching) one - or more elements in the ::tcltest::skippingTests variable - - 3) the "constraints" argument to the "test" call, if given, - contains one or more false elements. - -You can set ::tcltest::matchingTests and/or ::tcltest::skippingTests either -interactively (after the defs.tcl file has been sourced), or by the -command line arguments -match and -skip, for example: - - tcltest socket.test -match "*2.* *4.*" -skip "*2.33*" - -The two predefined constraints (knownBug and nonPortable) can be -overridden either interactively (after the defs.tcl file has been -sourced) by setting the ::tcltest::testConfig() variable, or -by using the -constraints command line option with the name of the -constraint in the argument. The following example shows how to run -tests that are constrained by the knownBug and nonPortable -restricions: - - tcltest all.tcl -constraints "knownBug nonPortable" - -See the defs.tcl file for information about each of these constraints. -Other constraints can be added at any time. See the "Writing a new -test" section below for more details about using built-in constraints -and adding new ones. - -Adding a New Test File: ------------------------ - -If the file matches the tests/*.test pattern (as it should), then it -will automatically be run by the all.tcl file. Make sure your test -file can be run from any working directory by running the following -from several different working directories: - - tcltest tests/all.tcl - -Make sure no temporary files are left behind by your test file. Your -test file should call "::tcltest::cleanupTests" before returning. The -::tcltest::cleanupTests procedure prints statistics about the number of -tests that passed, skipped, and failed, and removes all files that -were created using the ::tcltest::makeFile and ::tcltest::makeDirectory +Please refer to the defs.tcl file for more documentation on these procedures. -Be sure your tests can run cross-platform in both the build -environment as well as the installation environment. If your test -file contains tests that should not be run in or more of those cases, -please use the constraints mechanism described in the next section to -skip those tests. +3. Writing a new test: +---------------------- + +The test procedure runs a test script and prints an error message if +the script's result does not match the expected result. The following +is the spec for the "test" command: -Writing a new test: -------------------- + test ??