diff options
Diffstat (limited to 'tcllib/apps/page.man')
-rw-r--r-- | tcllib/apps/page.man | 467 |
1 files changed, 467 insertions, 0 deletions
diff --git a/tcllib/apps/page.man b/tcllib/apps/page.man new file mode 100644 index 0000000..bae424d --- /dev/null +++ b/tcllib/apps/page.man @@ -0,0 +1,467 @@ +[comment {-*- tcl -*- doctools manpage}] +[manpage_begin page n 1.0] +[see_also page::pluginmgr] +[keywords {parser generator}] +[keywords {text processing}] +[copyright {2005 Andreas Kupries <andreas_kupries@users.sourceforge.net>}] +[titledesc {Parser Generator}] +[moddesc {Development Tools}] +[category {Page Parser Generator}] +[description] +[para] + +The application described by this document, [syscmd page], is actually +not just a parser generator, as the name implies, but a generic tool +for the execution of arbitrary transformations on texts. + +[para] + +Its genericity comes through the use of [term plugins] for reading, +transforming, and writing data, and the predefined set of plugins +provided by Tcllib is for the generation of memoizing recursive +descent parsers (aka [term {packrat parsers}]) from grammar +specifications ([term {Parsing Expression Grammars}]). + +[para] + +[syscmd page] is written on top of the package + +[package page::pluginmgr], wrapping its functionality into a command +line based application. All the other [package page::*] packages are +plugin and/or supporting packages for the generation of parsers. The +parsers themselves are based on the packages [package grammar::peg], +[package grammar::peg::interp], and [package grammar::mengine]. + +[subsection {COMMAND LINE}] + +[list_begin definitions] + +[call [cmd page] [opt [arg options]...] [opt "[arg input] [opt [arg output]]"]] + +This is general form for calling [syscmd page]. The application will +read the contents of the file [arg input], process them under the +control of the specified [arg options], and then write the result to +the file [arg output]. + +[para] + +If [arg input] is the string [const -] the data to process will be +read from [const stdin] instead of a file. Analogously the result will +be written to [const stdout] instead of a file if [arg output] is the +string [const -]. A missing output or input specification causes the +application to assume [const -]. + +[para] + +The detailed specifications of the recognized [arg options] are +provided in section [sectref OPTIONS]. + +[list_begin arguments] +[arg_def path input in] + +This argument specifies the path to the file to be processed by the +application, or [const -]. The last value causes the application to +read the text from [const stdin]. Otherwise it has to exist, and be +readable. If the argument is missing [const -] is assumed. + +[arg_def path output in] + +This argument specifies where to write the generated text. It can be +the path to a file, or [const -]. The last value causes the +application to write the generated documented to [const stdout]. + +[para] + +If the file [arg output] does not exist then +[lb]file dirname $output[rb] has to exist and must be a writable +directory, as the application will create the fileto write to. + +[para] + +If the argument is missing [const -] is assumed. + +[list_end] +[list_end] + +[subsection OPERATION] + +... reading ... transforming ... writing - plugins - pipeline ... + +[subsection OPTIONS] + +This section describes all the options available to the user of the +application. Options are always processed in order. I.e. of both +[option --help] and [option --version] are specified the option +encountered first has precedence. + +[para] + +Unknown options specified before any of the options [option -rd], +[option -wr], or [option -tr] will cause processing to abort with an +error. Unknown options coming in between these options, or after the +last of them are assumed to always take a single argument and are +associated with the last plugin option coming before them. They will +be checked after all the relevant plugins, and thus the options they +understand, are known. I.e. such unknown options cause error if and +only if the plugin option they are associated with does not understand +them, and was not superceded by a plugin option coming after. + +[para] + +Default options are used if and only if the command line did not +contain any options at all. They will set the application up as a +PEG-based parser generator. The exact list of options is + +[para] +[example {-c peg}] +[para] + +And now the recognized options and their arguments, if they have any: + +[para] +[list_begin options] + +[opt_def --help] +[opt_def -h] +[opt_def -?] + +When one of these options is found on the command line all arguments +coming before or after are ignored. The application will print a short +description of the recognized options and exit. + +[opt_def --version] +[opt_def -V] + +When one of these options is found on the command line all arguments +coming before or after are ignored. The application will print its +own revision and exit. + +[opt_def -P] + +This option signals the application to activate visual feedback while +reading the input. + +[opt_def -T] + +This option signals the application to collect statistics while +reading the input and to print them after reading has completed, +before processing started. + +[opt_def -D] + +This option signals the application to activate logging in the Safe +base, for the debugging of problems with plugins. + +[opt_def -r parser] +[opt_def -rd parser] +[opt_def --reader parser] + +These options specify the plugin the application has to use for +reading the [arg input]. If the options are used multiple times the +last one will be used. + +[opt_def -w generator] +[opt_def -wr generator] +[opt_def --writer generator] + +These options specify the plugin the application has to use for +generating and writing the final [arg output]. If the options are used +multiple times the last one will be used. + +[opt_def -t process] +[opt_def -tr process] +[opt_def --transform process] + +These options specify a plugin to run on the input. In contrast to +readers and writers each use will [emph not] supersede previous +uses, but add each chosen plugin to a list of transformations, either +at the front, or the end, per the last seen use of either option +[option -p] or [option -a]. The initial default is to append the new +transformations. + +[opt_def -a] +[opt_def --append] + +These options signal the application that all following +transformations should be added at the end of the list of +transformations. + +[opt_def -p] +[opt_def --prepend] + +These options signal the application that all following +transformations should be added at the beginning of the list of +transformations. + +[opt_def --reset] + +This option signals the application to clear the list of +transformations. This is necessary to wipe out the default +transformations used. + +[opt_def -c file] +[opt_def --configuration file] + +This option causes the application to load a configuration file and/or +plugin. This is a plugin which in essence provides a pre-defined set +of commandline options. They are processed exactly as if they have +been specified in place of the option and its arguments. This means +that unknown options found at the beginning of the configuration file +are associated with the last plugin, even if that plugin was specified +before the configuration file itself. Conversely, unknown options +coming after the configuration file can be associated with a plugin +specified in the file. + +[para] + +If the argument is a file which cannot be loaded as a plugin the +application will assume that its contents are a list of options and +their arguments, separated by space, tabs, and newlines. Options and +argumentes containing spaces can be quoted via double-quotes (") and +quotes ('). The quote character can be specified within in a quoted +string by doubling it. Newlines in a quoted string are accepted as is. + +[comment {"}] +[list_end] + +[subsection PLUGINS] + +[syscmd page] makes use of four different types of plugins, namely: +readers, writers, transformations, and configurations. Here we provide +only a basic introduction on how to use them from [syscmd page]. The +exact APIs provided to and expected from the plugins can be found in +the documentation for [package page::pluginmgr], for those who wish to +write their own plugins. + +[para] + +Plugins are specified as arguments to the options [option -r], +[option -w], [option -t], [option -c], and their equivalent longer +forms. See the section [sectref OPTIONS] for reference. + +[para] + +Each such argument will be first treated as the name of a file and +this file is loaded as the plugin. If however there is no file with +that name, then it will be translated into the name of a package, and +this package is then loaded. For each type of plugins the package +management searches not only the regular paths, but a set application- +and type-specific paths as well. Please see the section +[sectref {PLUGIN LOCATIONS}] for a listing of all paths and their +sources. + +[para] + +[list_begin definitions] +[def "[option -c] [arg name]"] + +Configurations. The name of the package for the plugin [arg name] is +"page::config::[arg name]". + +[para] +We have one predefined plugin: + +[list_begin definitions] +[def [emph peg]] + +It sets the application up as a parser generator accepting parsing +expression grammars and writing a packrat parser in Tcl. The actual +arguments it specifies are: + +[para] +[example { + --reset + --append + --reader peg + --transform reach + --transform use + --writer me +}] +[para] + +[list_end] + +[def "[option -r] [arg name]"] + +Readers. The name of the package for the plugin [arg name] is +"page::reader::[arg name]". + +[para] +We have five predefined plugins: + +[list_begin definitions] +[def [emph peg]] + +Interprets the input as a parsing expression grammar ([term PEG]) and +generates a tree representation for it. Both the syntax of PEGs and +the structure of the tree representation are explained in their own +manpages. + +[def [emph hb]] + +Interprets the input as Tcl code as generated by the writer plugin +[emph hb] and generates its tree representation. + +[def [emph ser]] + +Interprets the input as the serialization of a PEG, as generated by +the writer plugin [emph ser], using the package +[package grammar::peg]. + +[def [emph lemon]] + +Interprets the input as a grammar specification as understood by +Richard Hipp's [term LEMON] parser generator and generates a tree +representation for it. Both the input syntax and the structure of the +tree representation are explained in their own manpages. + +[def [emph treeser]] + +Interprets the input as the serialization of a +[package struct::tree]. It is validated as such, +but nothing else. It is [emph not] assumed to +be the tree representation of a grammar. +[list_end] + +[def "[option -w] [arg name]"] + +Writers. The name of the package for the plugin [arg name] is +"page::writer::[arg name]". + +[para] +We have eight predefined plugins: + +[list_begin definitions] + +[def [emph identity]] + +Simply writes the incoming data as it is, without making any +changes. This is good for inspecting the raw result of a reader or +transformation. + +[def [emph null]] + +Generates nothing, and ignores the incoming data structure. + +[def [emph tree]] + +Assumes that the incoming data structure is a [package struct::tree] +and generates an indented textual representation of all nodes, their +parental relationships, and their attribute information. + +[def [emph peg]] + +Assumes that the incoming data structure is a tree representation of a +[term PEG] or other other grammar and writes it out as a PEG. The +result is nicely formatted and partially simplified (strings as +sequences of characters). A pretty printer in essence, but can also be +used to obtain a canonical representation of the input grammar. + +[def [emph tpc]] + +Assumes that the incoming data structure is a tree representation of a +[term PEG] or other other grammar and writes out Tcl code defining a +package which defines a [package grammar::peg] object containing the +grammar when it is loaded into an interpreter. + +[def [emph hb]] + +This is like the writer plugin [emph tpc], but it writes only the +statements which define stat expression and grammar rules. The code +making the result a package is left out. + +[def [emph ser]] + +Assumes that the incoming data structure is a tree representation of a +[term PEG] or other other grammar, transforms it internally into a +[package grammar::peg] object and writes out its serialization. + +[def [emph me]] + +Assumes that the incoming data structure is a tree representation of a +[term PEG] or other other grammar and writes out Tcl code defining a +package which implements a memoizing recursive descent parser based on +the match engine (ME) provided by the package [package grammar::mengine]. + +[list_end] + +[def "[option -t] [arg name]"] + +Transformers. The name of the package for the plugin [arg name] is +"page::transform::[arg name]". + +[para] +We have two predefined plugins: + +[list_begin definitions] +[def [emph reach]] + +Assumes that the incoming data structure is a tree representation of a +[term PEG] or other other grammar. It determines which nonterminal +symbols and rules are reachable from start-symbol/expression. All +nonterminal symbols which were not reached are removed. + +[def [emph use]] + +Assumes that the incoming data structure is a tree representation of a +[term PEG] or other other grammar. It determines which nonterminal +symbols and rules are able to generate a [emph finite] sequences of +terminal symbols (in the sense for a Context Free Grammar). All +nonterminal symbols which were not deemed useful in this sense are +removed. + +[list_end] +[list_end] + +[subsection {PLUGIN LOCATIONS}] + +The application-specific paths searched by [syscmd page] either are, +or come from: + +[para] + +[list_begin enumerated] +[enum] The directory [file ~/.page/plugin] +[enum] The environment variable [term PAGE_PLUGINS] +[enum] The registry entry [term "HKEY_LOCAL_MACHINE\\SOFTWARE\\PAGE\\PLUGINS"] +[enum] The registry entry [term "HKEY_CURRENT_USER\\SOFTWARE\\PAGE\\PLUGINS"] +[list_end] + +[para] + +The type-specific paths searched by [syscmd page] either are, or come +from: + +[para] +[list_begin enumerated] +[enum] The directory [file ~/.page/plugin/<TYPE>] +[enum] The environment variable [term PAGE_<TYPE>_PLUGINS] +[enum] The registry entry [term "HKEY_LOCAL_MACHINE\\SOFTWARE\\PAGE\\<TYPE>\\PLUGINS"] +[enum] The registry entry [term "HKEY_CURRENT_USER\\SOFTWARE\\PAGE\\<TYPE>\\PLUGINS"] +[list_end] + +[para] + +Where the placeholder [term <TYPE>] is always one of the values below, +properly capitalized. + +[list_begin enumerated] +[enum] reader +[enum] writer +[enum] transform +[enum] config +[list_end] +[para] + +The registry entries are specific to the Windows(tm) platform, all +other platforms will ignore them. + +[para] + +The contents of both environment variables and registry entries are +interpreted as a list of paths, with the elements separated by either +colon (Unix), or semicolon (Windows). + +[vset CATEGORY page] +[include ../modules/doctools2base/include/feedback.inc] +[manpage_end] |