11 files changed, 350 insertions, 0 deletions
diff --git a/tcllib/modules/httpd/demos/content.file.md b/tcllib/modules/httpd/demos/content.file.md
new file mode 100644
index 0000000..b2c1b3a
--- /dev/null
+++ b/tcllib/modules/httpd/demos/content.file.md
@@ -0,0 +1,55 @@
+httpd::content::file
+====================
+Back to: [Index](index.md) | [Package httpd::content](content.md)
+
+The **httpd::content::file** class implements a system for sharing a
+local file structure via http.
+
+## Special file handling
+
+### Directories
+
+When a directory path is requested, the system searches for one of the following (in order):
+
+* index.tml
+* index.md
+* index.html
+
+If one of these files is found, it is delivered as the response to the request. If no index file
+was found, the object will call the object's *DirectoryListing* method to
+deliver a dynamic listing of the files in the directory.
+
+### .md files
+
+Files with the .md extension are parsed using the *Markdown* package.
+
+### .tml files
+
+Files with the .tml extension are parsed using a call to *subst*.
+This allows them to deliver content in the same manner as tclhttpd. The
+contents of the *query_headers* are loaded as local variables prior to
+the *subst* call. NOTE: Unlike Tclhttpd, the substitution is performed
+inside of the reply object's namespace, not the local interpreter. Thus,
+template files can exercise the object's methods using the "my" command.
+
+## Dispatch Parameters
+
+Objects of this class needs additional information from the server in
+order to operate. These fields should be coded into the **add_root** call.
+
+### path *filepath*
+
+The **path** parameter specifies the root of the file path to be exposed.
+
+## Methods
+
+### DirectoryListing *local_file*
+
+Generates an HTML listing of a file path. The default implementation is a *very*
+rudimentary **glob --nocomplain [file join $local_path \*]**
+
+### FileName
+
+Converts the **REQUEST_URI** from query_headers into a local file path. This
+method searches first for the file name verbatim. If not found, it then searches
+for the same file name with a .md, .html, or .tml extension (in that order.)
diff --git a/tcllib/modules/httpd/demos/content.form.md b/tcllib/modules/httpd/demos/content.form.md
new file mode 100644
index 0000000..6a950fb
--- /dev/null
+++ b/tcllib/modules/httpd/demos/content.form.md
@@ -0,0 +1,17 @@
+httpd::content::form
+=============
+Back to: [Index](index.md) | [Package httpd::content](content.md)
+
+This class is intended as a mixin. It has no ancestors, nor does it contain
+any dispatch, content, or output methods.
+
+### Method Url_Decode *data*
+
+Translates a standard http query encoding string into a stream of key/value pairs.
+
+### Method ReadForm
+
+For GET requests, this method will convert the URI to key/value pairs.
+
+For POST requests, this method will read the body of the request and convert
+that block of text to a stream of key/value pairs.
+\ No newline at end of file
diff --git a/tcllib/modules/httpd/demos/content.md b/tcllib/modules/httpd/demos/content.md
new file mode 100644
index 0000000..abde776
--- /dev/null
+++ b/tcllib/modules/httpd/demos/content.md
@@ -0,0 +1,14 @@
+http::content
+=============
+[Back to Index](index.md)
+
+The **httpd::content** package is an extension to the core **httpd** package
+which provides all of the bread and butter functions needed to implement a
+basic webserver.
+
+* [Content Server](content.server.md)
+* [Form handler](content.form.md)
+* [File handler](content.file.md)
+* [SCGI handler](content.scgi.md)
+* [Proxy handler](content.proxy.md)
+    
+\ No newline at end of file
diff --git a/tcllib/modules/httpd/demos/content.proxy.md b/tcllib/modules/httpd/demos/content.proxy.md
new file mode 100644
index 0000000..32f757b
--- /dev/null
+++ b/tcllib/modules/httpd/demos/content.proxy.md
@@ -0,0 +1,20 @@
+httpd::content::proxy
+=============
+Back to: [Index](index.md) | [Package httpd::content](content.md)
+
+The proxy handler farms out the generation of content to an external process running
+on a known port. The external process is assumed to be a proxy server, and it is the job
+of this object to transform the query as received into a form that is understood by
+the external server.
+
+To implement a proxy handler, replace **proxy_info** with one that will return a list
+containing the following:
+
+    PROXYHOST PROXYPORT PROXYURI
+    
+* PROXYHOST - The hostname or IP address of the server running the process
+* PROXYPORT - The port to connect to
+* PROXYURI - The replacement GET/POST/etc request to make to the external process.
+
+The **proxy_info** method also makes a handly place to spawn a locally hosted process on demand.
+For an example of this, see the [docserver.tcl](docserver.tcl) Example.
+\ No newline at end of file
diff --git a/tcllib/modules/httpd/demos/content.scgi.md b/tcllib/modules/httpd/demos/content.scgi.md
new file mode 100644
index 0000000..07b7f11
--- /dev/null
+++ b/tcllib/modules/httpd/demos/content.scgi.md
@@ -0,0 +1,20 @@
+httpd::content::scgi
+=============
+Back to: [Index](index.md) | [Package httpd::content](content.md)
+
+The SCGI handler farms out the generation of content to an external process
+running at a known port. Because this process is persistent, the SCGI system
+avoids the overhead of spawning and spooling up an external process with
+every page view.
+
+To implement an SCGI handler, replace the **scgi_info** method with one
+that will return a list containing the following:
+
+   SCGIHOST SCGIPORT SCGISCRIPT
+   
+* SCGIHOST - The hostname or IP address of the server running the process
+* SCGIPORT - The port to connect to
+* SCGISCRIPT - The SCGISCRIPT parameter which will be passed to the external process via headers.
+
+The **scgi_info** method also makes a handly place to spawn a locally hosted process on demand.
+For an example of this, see the [docserver.tcl](docserver.tcl) Example.
+\ No newline at end of file
diff --git a/tcllib/modules/httpd/demos/content.server.md b/tcllib/modules/httpd/demos/content.server.md
new file mode 100644
index 0000000..fae985e
--- /dev/null
+++ b/tcllib/modules/httpd/demos/content.server.md
@@ -0,0 +1,42 @@
+http::content
+=============
+Back to: [Index](index.md) | [Package httpd::content](content.md)
+
+
+## Class: httpd::server::dispatch
+
+The **httpd::server::dispatch** adds additional functionality to the basic
+**httpd::server** class. It's *dispatch* method performs a pattern search
+based on url's registered via the *add_uri* method. That *add_uri* method
+allows the developer to specify which class will handle replies, as well as
+pass configuration information onto those objects.
+
+### Option doc_root
+
+Specifiying a *doc_root* will introduce a pattern search of last resort to
+find a matching URI as a file subordinate to the *doc_root*. Also, if the
+*doc_root* is specified, the system will search the root folder for the following
+templates:
+
+* notfound.tml - A site specific "404 File not found" template
+* internal_error.tml - A site specific "505 Internal Server Error" template
+
+### Method add_uri *pattern* *info*
+
+*add_uri* appends a new pattern to the server's internal pattern search dict.
+Patterns utilize **string match**, so any global characters or patterns for
+string match will work.
+
+Patterns are matched in the order in which they were given. In the example:
+
+<pre><code>SERVER add_uri /home* {...}
+SERVER add_uri /home/star/runner* {...}</code></pre>
+
+The pattern for /home/star/runner* will never be reached because /home* was specified first.
+
+The **info** argument contains a dict that will be passed by the *connect* method of the
+server to the *dispatch* method of the reply. Only two fields are reserved by the core of
+httpd:
+
+* class - The base class for the reply
+* mixin - The class to be mixed into the new object immediately prior to invoking the object's *dispatch* method.
diff --git a/tcllib/modules/httpd/demos/docserver.tcl b/tcllib/modules/httpd/demos/docserver.tcl
new file mode 100644
index 0000000..9bba988
--- /dev/null
+++ b/tcllib/modules/httpd/demos/docserver.tcl
@@ -0,0 +1,134 @@
+###
+# This script creates two toplevel domains:
+# * Hosting the tcllib embedded documentation as static content
+# * Hosting a local fossil mirror of the tcllib repository
+###
+set here [file dirname [file join [pwd] [info script]]]
+set DEMOROOT $here
+set tcllibroot [file normalize [file join $here .. .. ..]]
+set auto_path [linsert $auto_path 0 [file join $tcllibroot modules]]
+package require httpd
+package require httpd::content
+
+tool::class create ::docserver::reply::scgi_fossil {
+  superclass httpd::content::scgi
+
+  method scgi_info {} {
+    ###
+    # We could calculate this all out ahead of time
+    # but it's a nice demo to be able to launch the process
+    # and compute the parameters needed on the fly
+    ###
+    set uri    [my query_headers get REQUEST_URI]
+    set prefix [my query_headers get prefix]
+    set prefix [string trimright $prefix *]
+    set prefix [string trimright $prefix /]
+    set module tcllib
+    ###
+    # 
+    if {![info exists ::fossil_process($module)]} {
+      set info [exec fossil status]
+      set dbfile {}
+      foreach line [split $info \n] {
+        if {[lindex $line 0] eq "repository:"} {
+          set dbfile [string trim [string range $line 12 end]]
+          break
+        }
+      }
+      if {$dbfile eq {}} {
+        tailcall my error 505 "Could not locate fossil respository database"
+      }
+      puts [list LAUNCHING $module $dbfile]
+      package require processman
+      package require nettool
+      set port [::nettool::allocate_port 40000]
+      set handle fossil:$port
+      set mport [my <server> port_listening]
+      set cmd [list fossil server $dbfile --port $port --localhost --scgi --baseurl http://[my query_headers get HTTP_HOST]$prefix 2>/tmp/$module.err >/tmp/$module.log]
+      dict set ::fossil_process($module) port $port
+      dict set ::fossil_process($module) handle $handle
+      dict set ::fossil_process($module) cmd $cmd
+      dict set ::fossil_process($module) SCRIPT_NAME $prefix
+    }
+    dict with ::fossil_process($module) {}
+    if {![::processman::running $handle]} {
+      puts "LAUNCHING $module as $cmd"
+      set process [::processman::spawn $handle {*}$cmd]
+      puts "LAUNCHED"
+      my varname paused
+      after 500
+      puts "RESUMED"
+    }
+    return [list localhost $port $SCRIPT_NAME]
+  }
+}
+
+tool::class create ::docserver::reply::proxy_fossil {
+  superclass httpd::content::proxy
+
+  method proxy_info {} {
+    ###
+    # We could calculate this all out ahead of time
+    # but it's a nice demo to be able to launch the process
+    # and compute the parameters needed on the fly
+    ###
+    set uri    [my query_headers get REQUEST_URI]
+    set prefix [my query_headers get prefix]
+    set prefix [string trimright $prefix *]
+    set prefix [string trimright $prefix /]
+    set module tcllib.proxy
+    ###
+    # 
+    if {![info exists ::fossil_process($module)]} {
+      set info [exec fossil status]
+      set dbfile {}
+      foreach line [split $info \n] {
+        if {[lindex $line 0] eq "repository:"} {
+          set dbfile [string trim [string range $line 12 end]]
+          break
+        }
+      }
+      if {$dbfile eq {}} {
+        tailcall my error 505 "Could not locate fossil respository database"
+      }
+      puts [list LAUNCHING $module $dbfile]
+      package require processman
+      package require nettool
+      set port [::nettool::allocate_port 40000]
+      set handle fossil:$port
+      set mport [my <server> port_listening]
+      set cmd [list fossil server $dbfile --port $port --localhost --baseurl http://[my query_headers get HTTP_HOST]$prefix 2>/tmp/$module.err >/tmp/$module.log]
+      dict set ::fossil_process($module) port $port
+      dict set ::fossil_process($module) handle $handle
+      dict set ::fossil_process($module) cmd $cmd
+      dict set ::fossil_process($module) SCRIPT_NAME $prefix
+    }
+    dict with ::fossil_process($module) {}
+    if {![::processman::running $handle]} {
+      puts "LAUNCHING $module as $cmd"
+      set process [::processman::spawn $handle {*}$cmd]
+      puts "LAUNCHED"
+      my varname paused
+      after 500
+      puts "RESUMED"
+    }
+    set rawreq    [my query_headers get REQUEST_RAW]
+    set URI [string range [lindex $rawreq 1] [string length $prefix] end]
+    if {$URI eq {}} {
+      set URI /
+    }
+    set proxyuri "[lindex $rawreq 0] $URI [lindex $rawreq 2]"
+    return [list localhost $port $proxyuri]
+  }
+}
+
+tool::class create ::docserver::server {
+  superclass ::httpd::server::dispatch ::httpd::server
+}
+
+::docserver::server create appmain doc_root $DEMOROOT
+appmain add_uri /tcllib* [list mixin httpd::content::file path [file join $tcllibroot embedded www]]
+appmain add_uri /fossil* {mixin ::docserver::reply::scgi_fossil}
+appmain add_uri /proxy* {mixin ::docserver::reply::proxy_fossil}
+
+tool::main
diff --git a/tcllib/modules/httpd/demos/index.md b/tcllib/modules/httpd/demos/index.md
new file mode 100644
index 0000000..689f769
--- /dev/null
+++ b/tcllib/modules/httpd/demos/index.md
@@ -0,0 +1,18 @@
+Your test server works!
+
+* [Tcllib embedded docs](/tcllib/index.html)
+* [Tcllib's fossil repo (hosted via SCGI)](/fossil)
+* [Tcllib's fossil repo (hosted via proxy)](/proxy)
+
+Internal documentation for httpd:
+
+* [Operating Principals](operations.md)
+* [Program Listing for docserver.tcl](docserver.tcl)
+* [Class httpd::reply](reply.md)
+* [Class httpd::server](server.md)
+* [Class httpd::content](content.md)
+    * [Content Server](content.server.md)
+    * [Form handler](content.form.md)
+    * [File handler](content.file.md)
+    * [SCGI handler](content.scgi.md)
+    * [Proxy handler](content.proxy.md)
+\ No newline at end of file
diff --git a/tcllib/modules/httpd/demos/operations.md b/tcllib/modules/httpd/demos/operations.md
new file mode 100644
index 0000000..eb9b236
--- /dev/null
+++ b/tcllib/modules/httpd/demos/operations.md
@@ -0,0 +1,30 @@
+httpd: Operations Manual
+===============================
+[Back to Index](index.md)
+
+The httpd module is designed to be an http server which is embeddable within another project.
+
+1. When a reply socket is opened, the *connect* method is exercised.
+2. The *connect* method then populates a dict with basic information such as the REMOTE_IP and the URI.
+3. *connect* calls the Server's *dispatch* method with this new dict as a parameter.
+4. *dispatch* returns with a dict describing the response to this request, or an empty list to indicate that this is an invalid request.
+5. A new object is created to manage the reply.
+    * If a **class** field is present in the dispatch dict, a new object will be of that class.
+    * If no **class** was given, the new object will be of the class specified by the server's *reply_class* property.
+6. If the field *mixin* is present and non-empty, the new reply object will mixin the class specified.
+7. The server object will then call the reply object's *dispatch* process, with the complete reply description dict as a paramter.
+8. The server adds the object to a list of objects it is tracking. If the reply object does not destroy itself within 2 minutes, the server will destroy it.
+
+Once the *dispatch* method is called, it is the reply object's job to:
+
+1. Parse the HTTP headers of the incoming request
+2. Formulate a response
+3. Transmit that response back across the request socket.
+4. Destroy itself when finished.
+5. On destruction, unregister itself from the server object.
+
+The basic reply class perfoms the following:
+
+1. Reads the HTTP years
+2. Invokes the *content* class, which utilizes the *puts* method to populate an internal buffer.
+3. Invokes the *output* class which will prepare reply headers and output the reply buffer to the request socket.
diff --git a/tcllib/modules/httpd/demos/reply.md b/tcllib/modules/httpd/demos/reply.md
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tcllib/modules/httpd/demos/reply.md
diff --git a/tcllib/modules/httpd/demos/server.md b/tcllib/modules/httpd/demos/server.md
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tcllib/modules/httpd/demos/server.md