1 files changed, 467 insertions, 0 deletions
diff --git a/tcllib/apps/page.man b/tcllib/apps/page.man
new file mode 100644
index 0000000..bae424d
--- /dev/null
+++ b/tcllib/apps/page.man
@@ -0,0 +1,467 @@
+[comment {-*- tcl -*- doctools manpage}]
+[manpage_begin page n 1.0]
+[see_also page::pluginmgr]
+[keywords {parser generator}]
+[keywords {text processing}]
+[copyright {2005 Andreas Kupries <andreas_kupries@users.sourceforge.net>}]
+[titledesc {Parser Generator}]
+[moddesc   {Development Tools}]
+[category  {Page Parser Generator}]
+[description]
+[para]
+
+The application described by this document, [syscmd page], is actually
+not just a parser generator, as the name implies, but a generic tool
+for the execution of arbitrary transformations on texts.
+
+[para]
+
+Its genericity comes through the use of [term plugins] for reading,
+transforming, and writing data, and the predefined set of plugins
+provided by Tcllib is for the generation of memoizing recursive
+descent parsers (aka [term {packrat parsers}]) from grammar
+specifications ([term {Parsing Expression Grammars}]).
+
+[para]
+
+[syscmd page] is written on top of the package
+
+[package page::pluginmgr], wrapping its functionality into a command
+line based application. All the other [package page::*] packages are
+plugin and/or supporting packages for the generation of parsers. The
+parsers themselves are based on the packages [package grammar::peg],
+[package grammar::peg::interp], and [package grammar::mengine].
+
+[subsection {COMMAND LINE}]
+
+[list_begin definitions]
+
+[call [cmd page] [opt [arg options]...] [opt "[arg input] [opt [arg output]]"]]
+
+This is general form for calling [syscmd page]. The application will
+read the contents of the file [arg input], process them under the
+control of the specified [arg options], and then write the result to
+the file [arg output].
+
+[para]
+
+If [arg input] is the string [const -] the data to process will be
+read from [const stdin] instead of a file. Analogously the result will
+be written to [const stdout] instead of a file if [arg output] is the
+string [const -]. A missing output or input specification causes the
+application to assume [const -].
+
+[para]
+
+The detailed specifications of the recognized [arg options] are
+provided in section [sectref OPTIONS].
+
+[list_begin arguments]
+[arg_def path input in]
+
+This argument specifies the path to the file to be processed by the
+application, or [const -]. The last value causes the application to
+read the text from [const stdin]. Otherwise it has to exist, and be
+readable. If the argument is missing [const -] is assumed.
+
+[arg_def path output in]
+
+This argument specifies where to write the generated text. It can be
+the path to a file, or [const -]. The last value causes the
+application to write the generated documented to [const stdout].
+
+[para]
+
+If the file [arg output] does not exist then
+[lb]file dirname $output[rb] has to exist and must be a writable
+directory, as the application will create the fileto write to.
+
+[para]
+
+If the argument is missing [const -] is assumed.
+
+[list_end]
+[list_end]
+
+[subsection OPERATION]
+
+... reading ... transforming ... writing - plugins - pipeline ...
+
+[subsection OPTIONS]
+
+This section describes all the options available to the user of the
+application. Options are always processed in order. I.e. of both
+[option --help] and [option --version] are specified the option
+encountered first has precedence.
+
+[para]
+
+Unknown options specified before any of the options [option -rd],
+[option -wr], or [option -tr] will cause processing to abort with an
+error. Unknown options coming in between these options, or after the
+last of them are assumed to always take a single argument and are
+associated with the last plugin option coming before them. They will
+be checked after all the relevant plugins, and thus the options they
+understand, are known. I.e. such unknown options cause error if and
+only if the plugin option they are associated with does not understand
+them, and was not superceded by a plugin option coming after.
+
+[para]
+
+Default options are used if and only if the command line did not
+contain any options at all. They will set the application up as a
+PEG-based parser generator. The exact list of options is
+
+[para]
+[example {-c peg}]
+[para]
+
+And now the recognized options and their arguments, if they have any:
+
+[para]
+[list_begin options]
+
+[opt_def --help]
+[opt_def -h]
+[opt_def -?]
+
+When one of these options is found on the command line all arguments
+coming before or after are ignored. The application will print a short
+description of the recognized options and exit.
+
+[opt_def --version]
+[opt_def -V]
+
+When one of these options is found on the command line all arguments
+coming before or after are ignored. The application will print its
+own revision and exit.
+
+[opt_def -P]
+
+This option signals the application to activate visual feedback while
+reading the input.
+
+[opt_def -T]
+
+This option signals the application to collect statistics while
+reading the input and to print them after reading has completed,
+before processing started.
+
+[opt_def -D]
+
+This option signals the application to activate logging in the Safe
+base, for the debugging of problems with plugins.
+
+[opt_def -r parser]
+[opt_def -rd parser]
+[opt_def --reader parser]
+
+These options specify the plugin the application has to use for
+reading the [arg input]. If the options are used multiple times the
+last one will be used.
+
+[opt_def -w generator]
+[opt_def -wr generator]
+[opt_def --writer generator]
+
+These options specify the plugin the application has to use for
+generating and writing the final [arg output]. If the options are used
+multiple times the last one will be used.
+
+[opt_def -t process]
+[opt_def -tr process]
+[opt_def --transform process]
+
+These options specify a plugin to run on the input. In contrast to
+readers and writers each use will [emph not] supersede previous
+uses, but add each chosen plugin to a list of transformations, either
+at the front, or the end, per the last seen use of either option
+[option -p] or [option -a]. The initial default is to append the new
+transformations.
+
+[opt_def -a]
+[opt_def --append]
+
+These options signal the application that all following
+transformations should be added at the end of the list of
+transformations.
+
+[opt_def -p]
+[opt_def --prepend]
+
+These options signal the application that all following
+transformations should be added at the beginning of the list of
+transformations.
+
+[opt_def --reset]
+
+This option signals the application to clear the list of
+transformations. This is necessary to wipe out the default
+transformations used.
+
+[opt_def -c file]
+[opt_def --configuration file]
+
+This option causes the application to load a configuration file and/or
+plugin. This is a plugin which in essence provides a pre-defined set
+of commandline options. They are processed exactly as if they have
+been specified in place of the option and its arguments. This means
+that unknown options found at the beginning of the configuration file
+are associated with the last plugin, even if that plugin was specified
+before the configuration file itself. Conversely, unknown options
+coming after the configuration file can be associated with a plugin
+specified in the file.
+
+[para]
+
+If the argument is a file which cannot be loaded as a plugin the
+application will assume that its contents are a list of options and
+their arguments, separated by space, tabs, and newlines. Options and
+argumentes containing spaces can be quoted via double-quotes (") and
+quotes ('). The quote character can be specified within in a quoted
+string by doubling it. Newlines in a quoted string are accepted as is.
+
+[comment {"}]
+[list_end]
+
+[subsection PLUGINS]
+
+[syscmd page] makes use of four different types of plugins, namely:
+readers, writers, transformations, and configurations. Here we provide
+only a basic introduction on how to use them from [syscmd page]. The
+exact APIs provided to and expected from the plugins can be found in
+the documentation for [package page::pluginmgr], for those who wish to
+write their own plugins.
+
+[para]
+
+Plugins are specified as arguments to the options [option -r],
+[option -w], [option -t], [option -c], and their equivalent longer
+forms. See the section [sectref OPTIONS] for reference.
+
+[para]
+
+Each such argument will be first treated as the name of a file and
+this file is loaded as the plugin. If however there is no file with
+that name, then it will be translated into the name of a package, and
+this package is then loaded. For each type of plugins the package
+management searches not only the regular paths, but a set application-
+and type-specific paths as well. Please see the section
+[sectref {PLUGIN LOCATIONS}] for a listing of all paths and their
+sources.
+
+[para]
+
+[list_begin definitions]
+[def "[option -c] [arg name]"]
+
+Configurations. The name of the package for the plugin [arg name] is
+"page::config::[arg name]".
+
+[para]
+We have one predefined plugin:
+
+[list_begin definitions]
+[def [emph peg]]
+
+It sets the application up as a parser generator accepting parsing
+expression grammars and writing a packrat parser in Tcl. The actual
+arguments it specifies are:
+
+[para]
+[example {
+	--reset
+	--append
+	--reader    peg
+	--transform reach
+	--transform use
+	--writer    me
+}]
+[para]
+
+[list_end]
+
+[def "[option -r] [arg name]"]
+
+Readers. The name of the package for the plugin [arg name] is
+"page::reader::[arg name]".
+
+[para]
+We have five predefined plugins:
+
+[list_begin definitions]
+[def [emph peg]]
+
+Interprets the input as a parsing expression grammar ([term PEG]) and
+generates a tree representation for it. Both the syntax of PEGs and
+the structure of the tree representation are explained in their own
+manpages.
+
+[def [emph hb]]
+
+Interprets the input as Tcl code as generated by the writer plugin
+[emph hb] and generates its tree representation.
+
+[def [emph ser]]
+
+Interprets the input as the serialization of a PEG, as generated by
+the writer plugin [emph ser], using the package
+[package grammar::peg].
+
+[def [emph lemon]]
+
+Interprets the input as a grammar specification as understood by
+Richard Hipp's [term LEMON] parser generator and generates a tree
+representation for it. Both the input syntax and the structure of the
+tree representation are explained in their own manpages.
+
+[def [emph treeser]]
+
+Interprets the input as the serialization of a
+[package struct::tree]. It is validated as such,
+but nothing else. It is [emph not] assumed to
+be the tree representation of a grammar.
+[list_end]
+
+[def "[option -w] [arg name]"]
+
+Writers. The name of the package for the plugin [arg name] is
+"page::writer::[arg name]".
+
+[para]
+We have eight predefined plugins:
+
+[list_begin definitions]
+
+[def [emph identity]]
+
+Simply writes the incoming data as it is, without making any
+changes. This is good for inspecting the raw result of a reader or
+transformation.
+
+[def [emph null]]
+
+Generates nothing, and ignores the incoming data structure.
+
+[def [emph tree]]
+
+Assumes that the incoming data structure is a [package struct::tree]
+and generates an indented textual representation of all nodes, their
+parental relationships, and their attribute information.
+
+[def [emph peg]]
+
+Assumes that the incoming data structure is a tree representation of a
+[term PEG] or other other grammar and writes it out as a PEG. The
+result is nicely formatted and partially simplified (strings as
+sequences of characters). A pretty printer in essence, but can also be
+used to obtain a canonical representation of the input grammar.
+
+[def [emph tpc]]
+
+Assumes that the incoming data structure is a tree representation of a
+[term PEG] or other other grammar and writes out Tcl code defining a
+package which defines a [package grammar::peg] object containing the
+grammar when it is loaded into an interpreter.
+
+[def [emph hb]]
+
+This is like the writer plugin [emph tpc], but it writes only the
+statements which define stat expression and grammar rules. The code
+making the result a package is left out.
+
+[def [emph ser]]
+
+Assumes that the incoming data structure is a tree representation of a
+[term PEG] or other other grammar, transforms it internally into a
+[package grammar::peg] object and writes out its serialization.
+
+[def [emph me]]
+
+Assumes that the incoming data structure is a tree representation of a
+[term PEG] or other other grammar and writes out Tcl code defining a
+package which implements a memoizing recursive descent parser based on
+the match engine (ME) provided by the package [package grammar::mengine].
+
+[list_end]
+
+[def "[option -t] [arg name]"]
+
+Transformers. The name of the package for the plugin [arg name] is
+"page::transform::[arg name]".
+
+[para]
+We have two predefined plugins:
+
+[list_begin definitions]
+[def [emph reach]]
+
+Assumes that the incoming data structure is a tree representation of a
+[term PEG] or other other grammar. It determines which nonterminal
+symbols and rules are reachable from start-symbol/expression. All
+nonterminal symbols which were not reached are removed.
+
+[def [emph use]]
+
+Assumes that the incoming data structure is a tree representation of a
+[term PEG] or other other grammar. It determines which nonterminal
+symbols and rules are able to generate a [emph finite] sequences of
+terminal symbols (in the sense for a Context Free Grammar). All
+nonterminal symbols which were not deemed useful in this sense are
+removed.
+
+[list_end]
+[list_end]
+
+[subsection {PLUGIN LOCATIONS}]
+
+The application-specific paths searched by [syscmd page] either are,
+or come from:
+
+[para]
+
+[list_begin enumerated]
+[enum] The directory            [file ~/.page/plugin]
+[enum] The environment variable [term PAGE_PLUGINS]
+[enum] The registry entry       [term "HKEY_LOCAL_MACHINE\\SOFTWARE\\PAGE\\PLUGINS"]
+[enum] The registry entry       [term "HKEY_CURRENT_USER\\SOFTWARE\\PAGE\\PLUGINS"]
+[list_end]
+
+[para]
+
+The type-specific paths searched by [syscmd page] either are, or come
+from:
+
+[para]
+[list_begin enumerated]
+[enum] The directory            [file ~/.page/plugin/<TYPE>]
+[enum] The environment variable [term PAGE_<TYPE>_PLUGINS]
+[enum] The registry entry       [term "HKEY_LOCAL_MACHINE\\SOFTWARE\\PAGE\\<TYPE>\\PLUGINS"]
+[enum] The registry entry       [term "HKEY_CURRENT_USER\\SOFTWARE\\PAGE\\<TYPE>\\PLUGINS"]
+[list_end]
+
+[para]
+
+Where the placeholder [term <TYPE>] is always one of the values below,
+properly capitalized.
+
+[list_begin enumerated]
+[enum] reader
+[enum] writer
+[enum] transform
+[enum] config
+[list_end]
+[para]
+
+The registry entries are specific to the Windows(tm) platform, all
+other platforms will ignore them.
+
+[para]
+
+The contents of both environment variables and registry entries are
+interpreted as a list of paths, with the elements separated by either
+colon (Unix), or semicolon (Windows).
+
+[vset CATEGORY page]
+[include ../modules/doctools2base/include/feedback.inc]
+[manpage_end]