path: root/tcllib/README.developer

RCS: @(#) $Id: README.developer,v 1.6 2009/06/02 22:49:55 andreas_kupries Exp $

Welcome to the tcllib, the Tcl Standard Library.
================================================

Introduction
------------

This README is intended to be a guide to the tools available to a

	Developer

working on Tcllib to help him with his tasks, i.e. making the tasks easier
to perform. It is our hope that this will improve the quality of even
non-released revisions of Tcllib, and make the work of the release
manager easier as well.

Audience
--------

The intended audience are, first and foremost, developers beginning to
work on Tcllib. To an experienced developer this document will be less
of a guide and more of a reference. Anybody else interested in working
on Tcllib is invited as well.


Directory hierarchy and file basics
------------------------------------

The main directories under the tcllib top directory are

	modules/
	examples/
and	apps/

Each directory FOO under modules/ represents one package, sometimes
more. In the case of the latter the packages are usually related in
some way. Examples are the base64, math, and struct modules, with
loose (base64) to strong (math) relations between the packages.

Examples associated with a module FOO, if there are any, are placed
into the directory

	examples/FOO

Any type of distributable application can be found under apps/,
together with their documentation, if any. Note that the apps/
directory is currently not split into sub-directories.

Regarding the files in Tcllib, the most common types found are

	.tcl	Tcl code for a package.

	.man	Documentation for a package, in doctools format.

	.test	Test suite for a package, or part of. Based on tcltest.

	.bench	Performance benchmarks for a package, or part of.
		Based on modules/bench

	.pcx	Syntax rules for TclDevKit's tclchecker. Using these
		rules allows tclchecker to check the use of commands
		of a Tcllib package X without having to scan the
		implementation of X, i.e. its .tcl files.


Adding a new module
-------------------

Assuming that FOO is the name of the new module, and T is the toplevel
directory of the Tcllib sources

(1)	Create the directory T/modules/FOO and put all the files of
	the module into it. Note:

	* The file 'pkgIndex.tcl' is required.

	* Implementation files should have the extension '.tcl',
	  naturally.

	* If available, documentation should be in doctools format,
          and the files should have the extension '.man' for SAK to
          recognize them.

	* If available the testsuite(s) should use 'tcltest' and the
	  general format as used by the other modules in Tcllib
	  (declaration of minimally needed Tcl, tcltest, supporting
	  packages, etc.). The file(s) should have the extension
	  '.test' for SAK to recognize them.

	  Note that an empty testsuite, or a testsuite which does not
	  perform any tests is less than useful and will not be
	  accepted.

	* If available the benchmark(s) should use 'bench' and the
          general format as used by the other modules in Tcllib. The
          file(s) should have the extension '.bench' for SAK to
          recognize them.

	* Other files can be named and placed as the module sees fit.

(2)	If the new module has an example application A which is
	polished enough for general use, put this application into the
	file "T/apps/A.tcl", and its documentation into the file
	"T/apps/A.man". While documentation for the application is
	optional, it is preferred.

	For examples which are not full-fledged applications, a
	skeleton, or not really polished for use, etc., create the
	directory T/examples/FOO/ and put them there.

	A key difference is what happens to them on installation, and
	what the target audience is.

	The examples are for developers using packages in Tcllib,
	whereas the applications are also for users of Tcllib which do
	not have an interest in developing for and with it. As such,
	they are installed as regular commands, accessible through the
	PATH, and example files are not installed.

(3)	To make Tcllib's installer aware of FOO, edit the file

		T/support/installation/modules.tcl

	Add a line 'Module FOO $impaction $docaction $exaction'. The
	various actions describe to the installer how to install the
	implementation files, the documentation, and the examples.

	Add a line 'Application A' for any application A which was
	added to T/apps for FOO.

	The following actions are available:

	Implementation

		_tcl - Copy all .tcl files in T/modules/FOO into the installation.
		_tcr - See above, does it for .tcl files in subdirectories as well.
		_tci - _tcl + Copying of a tclIndex - special to modules 'math', 'control'.
		_msg - _tcl + Copying of subdir 'msgs' - special to modules 'dns', 'log'.
		_doc - _tcl + Copying of subdir 'mpformats' - special to module 'doctools'.
		_tex - _tcl + Copying of .tex files - special to module 'textutil'.

		The _null action, see below, is available in principle
		too, but a module without implementation does not make
		sense.

	Documentation

		_null - Module has no documentation, do nothing.
		_man  - Process the .man files in T/modules/FOO and
			install the results (nroff and/or HTML) in the
			proper location, as given to the installer.

	Examples

		_null - Module has no examples, do nothing
		_exa  - Copy the directory T/examples/FOO
			(recursively) to the install location for
			examples.


Testing modules
---------------

To run the testsuite of a module FOO in tcllib use the 'test run'
argument of sak.tcl, like so:

	% pwd
	/the/tcllib/toplevel/directory

	% ./sak.tcl test run FOO
or	% ./sak.tcl test run modules/FOO

To run the testsuites of all modules either invoke 'test run' without a
module name, or use 'make test'. The latter assumes that configure was
run for Tcllib before, i.e.:

	% ./sak.tcl test run
or	% ./sak.tcl test run
	% make test

In all of the above cases the result will be a combination of progress
display and testsuite log, showing for each module the tests that pass
or failed and how many of each in a summary at the end.

To get a detailed log, it is necessary to invoke 'test run' with
additional options.

First example:
	% ./sak.tcl test run -l LOG FOO

This shows the same short log on the terminal, and writes a detailed
log to the file LOG.log, and excerpts to other files (LOG.summary,
LOG.failures, etc.).

Second example:
	% ./sak.tcl test run -v FOO
	% make test > LOG

This writes the detailed log to stdout, or to the file LOG, instead of
the short log. In all cases, the detailed log contains a list of all
test cases executed, which failed, and how they failed (expected
versus actual results).

Note:
The commands
	% make test
and	% make test > LOG

are able to generate different output (short vs long log) because the
Makefile target contains code which detects that stdout has been
redirected to a file and acts accordingly.

Non-developers should reports problems in Tcllib's bug tracker.
Information about its location and the relevant category can be found
in the section 'BUGS, IDEAS, FEEDBACK' of the manpage of the module
and/or package.

Module documentation
--------------------

The main format used for the documentation of packages in Tcllib is
'doctools', the support packages of which are part of Tcllib, see the
module 'doctools'.

To convert this documentation to HTML or nroff manpages, or some other
format use the 'doc' argument of sak.tcl, like so:

	% pwd
	/the/tcllib/toplevel/directory

	% ./sak.tcl doc html FOO
or	% ./sak.tcl doc html modules/FOO

The result of the conversion can be found in the newly-created 'doc'
directory in the current working directory.

The set of formats the documentation can be converted into can be
queried via

	% ./sak.tcl help doc


To convert the documentation of all modules either invoke 'test run'
without a module name, or use 'make html-doc', etc.. The latter
assumes that configure was run for Tcllib before, i.e.:

	% ./sak.tcl doc html
	% make html-doc

Note the special format 'validate'. Using this format does not convert
the documentation to anything (and the sub-directory 'doc' will not be
created), it just checks that the documentation is syntactically
correct. I.e.

	% ./sak.tcldoc validate modules/FOO
	% ./sak.tcldoc validate


Validating modules
------------------

Running the testsuite of a module, or checking the syntax of its
documentation (see the previous sections) are two forms of validation.

The 'validate' command of sak.tcl provides a few more. The online
documentation of this command is available via

	% ./sak.tcl help validate

The validated parts are man pages, testsuites, version information,
and syntax. The latter only if various static syntax checkers are
available on the PATH, like TclDevKit's tclchecker.

Note that testsuite validation is not the execution of the testsuites,
only if a package has a testsuite or not.

It is strongly recommended to validate a module before committing any
type of change made to it.

It is recommended to validate all modules before committing any type
of change made to one of them. We have package inter-dependencies
between packages in Tcllib, thus changing one package may break
others, and just validating the changed package will not catch such
problems.


Writing Tests
-------------

While a previous section talked about running the testsuite for a
module and the packages therein this has no meaning if the module in
question has no testsuites at all.

This section gives a very basic overview on methodologies for writing
tests and testsuites.

First there are "drudgery" tests. Written to check absolutely basic
assumptions which should never fail.

Example:

	For a command FOO taking two arguments, three tests calling it
	with zero, one, and three arguments. The basic checks that the
	command fails if it has not enough arguments, or too many.

After that come the tests checking things based on our knowledge of
the command, about its properties and assumptions. Some examples based
on the graph operations added during Google's Summer of Code 2009.

**	The BellmanFord command in struct::graph::ops takes a
	_startnode_ as argument, and this node should be a node of the
	graph. equals one test case checking the behavior when the
	specified node is not a node a graph.

	This often gives rise to code in the implementation which
	explicitly checks the assumption and throws a nice error.
	Instead of letting the algorithm fails later in some weird
	non-deterministic way.

	Such checks cannot be done always. The graph argument for
	example is just a command in itself, and while we expect it to
	exhibit a certain interface, i.e. set of sub-commands aka
	methods, we cannot check that it has them, except by actually
	trying to use them. That is done by the algorithm anyway, so
	an explicit check is just overhead we can get by without.

**	IIRC one of the distinguishing characteristic of either
	BellmanFord and/or Johnson is that they are able to handle
	negative weights. Whereas Dijkstra requires positive weights.

	This induces (at least) three testcases ... Graph with all
	positive weights, all negative, and a mix of positive and
	negative weights.

	Thinking further does the algorithm handle the weight '0' as
	well ? Another test case, or several, if we mix zero with
	positive and negative weights.

**	The two algorithms we are currently thinking about are about
	distances between nodes, and distance can be 'Inf'inity,
	i.e. nodes may not be connected. This means that good test
	cases are

	(1)	Strongly connected graph
	(2)	Connected graph
	(3)	Disconnected graph.

	At the extremes of (1) and (3) we have the fully connected
	graphs and graphs without edges, only nodes, i.e. completely
	disconnected.

**	IIRC both of the algorithms take weighted arcs, and fill in a
	default if arcs are left unweighted in the input graph.

	This also induces three test cases:

	(1)	Graph will all arcs with explicit weights.
	(2)	Graph without weights at all.
	(3)	Graph with mixture of weighted and unweighted graphs.


What was described above via examples is called 'black-box' testing.
Test cases are designed and written based on our knowledge of the
properties of the algorithm and its inputs, without referencing a
particular implementation.

Going further, a complement to 'black-box' testing is 'white-box'. For
this we know the implementation of the algorithm, we look at it and
design our tests cases so that they force the code through all
possible paths in the implementation. Wherever a decision is made we
have a test cases forcing a specific direction of the decision, for
all possible directions.

In practice I often hope that the black-box tests I have made are
enough to cover all the paths, obviating the need for white-box tests.

So, if you, dear reader, now believe that writing tests for an
algorithm takes at least as much time as coding the algorithm, and
often more time, then you are completely right. It does. Much more
time. See for example also http://sqlite.org/testing.html, a writeup
on how the Sqlite database engine is tested.



An interesting connection is to documentation. In one direction, the
properties you are checking with black-box testing are properties
which should be documented in the algorithm man page. And conversely,
if you have documentation of properties of an algorithm then this is a
good reference to base black-box tests on.

In practice test cases and documentation often get written together,
cross-influencing each other. And the actual writing of test cases is
a mix of black and white box, possibly influencing the implementation
while writing the tests. Like writing test for 'startnode not in input
graph' serving as reminder to put in a check for this into the code.