Merge http-bugfixes-2022H2 - several bugfixes and new helper commands.

1 files changed, 787 insertions, 197 deletions

diff --git a/doc/http.n b/doc/http.n
index d531995..89be995 100644
--- a/doc/http.n
+++ b/doc/http.n
@@ -13,7 +13,7 @@
 .SH NAME
 http \- Client-side implementation of the HTTP/1.1 protocol
 .SH SYNOPSIS
-\fBpackage require http\fI ?\fB2.10\fR?
+\fBpackage require http\fR ?\fB2.10\fR?
 .\" See Also -useragent option documentation in body!
 .sp
 \fB::http::config\fR ?\fI\-option value\fR ...?
@@ -32,36 +32,67 @@ http \- Client-side implementation of the HTTP/1.1 protocol
 .sp
 \fB::http::size \fItoken\fR
 .sp
-\fB::http::code \fItoken\fR
+\fB::http::error \fItoken\fR
 .sp
-\fB::http::ncode \fItoken\fR
+\fB::http::postError \fItoken\fR
+.sp
+\fB::http::cleanup \fItoken\fR
 .sp
-\fB::http::meta \fItoken\fR
+\fB::http::requestLine\fR \fItoken\fR
 .sp
-\fB::http::data \fItoken\fR
+\fB::http::requestHeaders\fR \fItoken\fR ?\fIheaderName\fR?
 .sp
-\fB::http::error \fItoken\fR
+\fB::http::requestHeaderValue\fR \fItoken\fR \fIheaderName\fR
 .sp
-\fB::http::cleanup \fItoken\fR
+\fB::http::responseLine\fR \fItoken\fR
+.sp
+\fB::http::responseCode\fR \fItoken\fR
+.sp
+\fB::http::reasonPhrase\fR \fIcode\fR
+.sp
+\fB::http::responseHeaders\fR \fItoken\fR ?\fIheaderName\fR?
+.sp
+\fB::http::responseHeaderValue\fR \fItoken\fR \fIheaderName\fR
+.sp
+\fB::http::responseInfo\fR \fItoken\fR
+.sp
+\fB::http::responseBody\fR \fItoken\fR
 .sp
 \fB::http::register \fIproto port command\fR
 .sp
 \fB::http::registerError \fIport\fR ?\fImessage\fR?
 .sp
 \fB::http::unregister \fIproto\fR
+.sp
+\fB::http::code \fItoken\fR
+.sp
+\fB::http::data \fItoken\fR
+.sp
+\fB::http::meta \fItoken\fR ?\fIheaderName\fR?
+.sp
+\fB::http::metaValue\fR \fItoken\fR \fIheaderName\fR
+.sp
+\fB::http::ncode \fItoken\fR
 .SH "EXPORTED COMMANDS"
 .PP
 Namespace \fBhttp\fR exports the commands \fBconfig\fR, \fBformatQuery\fR,
-\fBgeturl\fR, \fBquoteString\fR, \fBregister\fR, \fBregisterError\fR,
+\fBgeturl\fR, \fBpostError\fR, \fBquoteString\fR, \fBreasonPhrase\fR,
+\fBregister\fR,
+\fBregisterError\fR, \fBrequestHeaders\fR, \fBrequestHeaderValue\fR,
+\fBrequestLine\fR, \fBresponseBody\fR, \fBresponseCode\fR,
+\fBresponseHeaders\fR, \fBresponseHeaderValue\fR, \fBresponseInfo\fR,
+\fBresponseLine\fR,
 \fBreset\fR, \fBunregister\fR, and \fBwait\fR.
 .PP
 It does not export the commands \fBcleanup\fR, \fBcode\fR, \fBdata\fR,
-\fBerror\fR, \fBmeta\fR, \fBncode\fR, \fBsize\fR, or \fBstatus\fR.
+\fBerror\fR, \fBmeta\fR, \fBmetaValue\fR, \fBncode\fR,
+\fBsize\fR, or \fBstatus\fR.
 .BE
 .SH DESCRIPTION
 .PP
 The \fBhttp\fR package provides the client side of the HTTP/1.1
-protocol, as defined in RFC 7230 to RFC 7235, which supersede RFC 2616.
+protocol, as defined in RFC 9110 to 9112, which supersede RFC 7230
+to RFC 7235, which in turn supersede RFC 2616.
 The package implements the GET, POST, and HEAD operations
 of HTTP/1.1.  It allows configuration of a proxy host to get through
 firewalls.  The package is compatible with the \fBSafesock\fR security
@@ -74,14 +105,13 @@ The \fB::http::geturl\fR procedure does a HTTP transaction.
 Its \fIoptions \fR determine whether a GET, POST, or HEAD transaction
 is performed.
 The return value of \fB::http::geturl\fR is a token for the transaction.
-The value is also the name of an array in the ::http namespace
-that contains state information about the transaction.  The elements
-of this array are described in the \fBSTATE ARRAY\fR section.
+The token can be supplied as an argument to other commands, to manage the
+transaction and examine its results.
 .PP
 If the \fB\-command\fR option is specified, then
 the HTTP operation is done in the background.
 \fB::http::geturl\fR returns immediately after generating the
-HTTP request and the callback is invoked
+HTTP request and the \fB\-command\fR callback is invoked
 when the transaction completes.  For this to work, the Tcl event loop
 must be active.  In Tk applications this is always true.  For pure-Tcl
 applications, the caller can use \fB::http::wait\fR after calling
@@ -90,6 +120,15 @@ applications, the caller can use \fB::http::wait\fR after calling
 \fBNote:\fR The event queue is even used without the \fB\-command\fR option.
 As a side effect, arbitrary commands may be processed while \fBhttp::geturl\fR
 is running.
+.PP
+When the HTTP server has replied to the request, call the command
+\fB::http::responseInfo\fR, which
+returns a \fBdict\fR of metadata that is essential for identifying a
+successful transaction and making use of the response.  See
+section \fBMETADATA\fR for details of the information returned.
+The response itself is returned by command \fB::http::responseBody\fR,
+unless it has been redirected to a file by the \fI\-channel\fR option
+of \fB::http::geturl\fR.
 .SH COMMANDS
 .TP
 \fB::http::config\fR ?\fIoptions\fR?
@@ -175,7 +214,8 @@ default is 0.
 .TP
 \fB\-threadlevel\fR \fIlevel\fR
 .
-Specifies whether and how to use the \fBThread\fR package.  Possible values of \fIlevel\fR are 0, 1 or 2.
+Specifies whether and how to use the \fBThread\fR package.  Possible values
+of \fIlevel\fR are 0, 1 or 2.
 .RS
 .PP
 .DS
@@ -183,7 +223,11 @@ Specifies whether and how to use the \fBThread\fR package.  Possible values of \
 1 - use Thread if it is available, do not use it if it is unavailable
 2 - use Thread if it is available, raise an error if it is unavailable
 .DE
-The Tcl \fBsocket -async\fR command can block in adverse cases (e.g. a slow DNS lookup).  Using the Thread package works around this problem, for both HTTP and HTTPS transactions.  Values of \fIlevel\fR other than 0 are available only to the main interpreter in each thread.  See section \fBTHREADS\fR for more information.
+The Tcl \fBsocket -async\fR command can block in adverse cases (e.g. a slow
+DNS lookup).  Using the Thread package works around this problem, for both
+HTTP and HTTPS transactions.  Values of \fIlevel\fR other than 0 are
+available only to the main interpreter in each thread.  See
+section \fBTHREADS\fR for more information.
 .RE
 .TP
 \fB\-urlencoding\fR \fIencoding\fR
@@ -205,21 +249,22 @@ numbers of \fBhttp\fR and \fBTcl\fR.
 \fB\-zip\fR \fIboolean\fR
 .
 If the value is boolean \fBtrue\fR, then by default requests will send a header
-.QW "\fBAccept-Encoding: gzip,deflate,compress\fR" .
-If the value is boolean \fBfalse\fR, then by default this header will not be
-sent.  In either case the default can be overridden for an individual request by
+.QW "\fBAccept-Encoding: gzip,deflate\fR" .
+If the value is boolean \fBfalse\fR, then by default requests will send a header
+.QW "\fBAccept-Encoding: identity\fR" .
+In either case the default can be overridden for an individual request by
 supplying a custom \fBAccept-Encoding\fR header in the \fB\-headers\fR option
-of \fBhttp::geturl\fR. The default is 1.
+of \fBhttp::geturl\fR. The default value is 1.
 .RE
 .TP
 \fB::http::geturl\fR \fIurl\fR ?\fIoptions\fR?
 .
 The \fB::http::geturl\fR command is the main procedure in the package.
-The \fB\-query\fR option causes a POST operation and
+The \fB\-query\fR or \fB\-querychannel\fR option causes a POST operation and
 the \fB\-validate\fR option causes a HEAD operation;
 otherwise, a GET operation is performed.  The \fB::http::geturl\fR command
-returns a \fItoken\fR value that can be used to get
-information about the transaction.  See the \fBSTATE ARRAY\fR and
+returns a \fItoken\fR value that can be passed as an argument to other commands
+to get information about the transaction.  See the \fBMETADATA\fR and
 \fBERRORS\fR section for
 details.  The \fB::http::geturl\fR command blocks until the operation
 completes, unless the \fB\-command\fR option specifies a callback
@@ -231,7 +276,7 @@ that is invoked when the HTTP transaction completes.
 .
 Specifies whether to force interpreting the URL data as binary.  Normally
 this is auto-detected (anything not beginning with a \fBtext\fR content
-type or whose content encoding is \fBgzip\fR or \fBcompress\fR is
+type or whose content encoding is \fBgzip\fR or \fBdeflate\fR is
 considered binary data).
 .TP
 \fB\-blocksize\fR \fIsize\fR
@@ -243,13 +288,14 @@ At most \fIsize\fR bytes are read at once.  After each block, a call to the
 \fB\-channel\fR \fIname\fR
 .
 Copy the URL contents to channel \fIname\fR instead of saving it in
-\fBstate(body)\fR.
+a Tcl variable for retrieval by \fB::http::responseBody\fR.
 .TP
 \fB\-command\fR \fIcallback\fR
 .
-Invoke \fIcallback\fR after the HTTP transaction completes.
-This option causes \fB::http::geturl\fR to return immediately.
-The \fIcallback\fR gets an additional argument that is the \fItoken\fR returned
+The presence of this option causes \fB::http::geturl\fR to return immediately.
+After the HTTP transaction completes, the value of \fIcallback\fR is expanded,
+an additional argument is added, and the resulting command is evaluated.
+The additional argument is the \fItoken\fR returned
 from \fB::http::geturl\fR. This token is the name of an array that is
 described in the \fBSTATE ARRAY\fR section.  Here is a template for the
 callback:
@@ -257,8 +303,10 @@ callback:
 .PP
 .CS
 proc httpCallback {token} {
-    upvar #0 $token state
-    # Access state as a Tcl array
+    upvar 0 $token state
+    # Access state as a Tcl array defined in this proc
+    ...
+    return
 }
 .CE
 .PP
@@ -268,11 +316,30 @@ not call the \fBbgerror\fR handler.  See the \fBERRORS\fR section for
 details.
 .RE
 .TP
+\fB\-guesstype\fR \fIboolean\fR
+.
+Attempt to guess the \fBContent-Type\fR and character set when a misconfigured
+server provides no information.  The default value is \fIfalse\fR (do
+nothing).  If boolean \fItrue\fR then, if the server does not send a
+\fBContent-Type\fR header, or if it sends the value "application/octet-stream",
+\fBhttp::geturl\fR will attempt to guess appropriate values.  This is not
+intended to become a general-purpose tool, and currently it is limited to
+detecting XML documents that begin with an XML declaration.  In this case
+the \fBContent-Type\fR is changed to "application/xml", the binary flag
+state(binary) is changed to 0, and the character set is changed to
+the one specified by the "encoding" tag of the XML line, or to utf-8 if no
+encoding is specified.  Not used if a \fI\-channel\fR is specified.
+.TP
 \fB\-handler\fR \fIcallback\fR
 .
-Invoke \fIcallback\fR whenever HTTP data is available; if present, nothing
-else will be done with the HTTP data.  This procedure gets two additional
-arguments: the socket for the HTTP data and the \fItoken\fR returned from
+If this option is absent, \fBhttp::geturl\fR processes incoming data itself,
+either appending it to the state(body) variable or writing it to the -channel.
+But if the \fB\-handler\fR option is present, \fBhttp::geturl\fR does not do
+this processing and instead calls \fIcallback\fR.
+Whenever HTTP data is available, the value of \fIcallback\fR is expanded, an
+additional two arguments are added, and the resulting command is evaluated.
+The two additional
+arguments are: the socket for the HTTP data and the \fItoken\fR returned from
 \fB::http::geturl\fR.  The token is the name of a global array that is
 described in the \fBSTATE ARRAY\fR section.  The procedure is expected
 to return the number of bytes read from the socket.  Here is a
@@ -281,8 +348,8 @@ template for the callback:
 .PP
 .CS
 proc httpHandlerCallback {socket token} {
-    upvar #0 $token state
-    # Access socket, and state as a Tcl array
+    upvar 0 $token state
+    # Access socket, and state as a Tcl array defined in this proc
     # For example...
     ...
     set data [read $socket 1000]
@@ -295,8 +362,9 @@ proc httpHandlerCallback {socket token} {
 The \fBhttp::geturl\fR code for the \fB\-handler\fR option is not compatible
 with either compression or chunked transfer-encoding.  If \fB\-handler\fR is
 specified, then to work around these issues \fBhttp::geturl\fR will reduce the
-HTTP protocol to 1.0, and override the \fB\-zip\fR option (i.e. it will not
-send the header "\fBAccept-Encoding: gzip,deflate,compress\fR").
+HTTP protocol to 1.0, and override the \fB\-zip\fR option (i.e. it will
+send the header \fBAccept-Encoding: identity\fR instead
+of \fBAccept-Encoding: gzip,deflate\fR).
 .PP
 If options \fB\-handler\fR and \fB\-channel\fR are used together, the handler
 is responsible for copying the data from the HTTP socket to the specified
@@ -342,7 +410,10 @@ It is the caller's responsibility to ensure that the headers and request body
 (if any) conform to the requirements of the request method.  For example, if
 using \fB\-method\fR \fIPOST\fR to send a POST with an empty request body, the
 caller must also supply the option
-.QW "\-headers {Content-Length 0}" .
+.PP
+.CS
+\-headers {Content-Length 0}
+.CE
 .RE
 .TP
 \fB\-myaddr\fR \fIaddress\fR
@@ -352,18 +423,26 @@ multiple interfaces are available.
 .TP
 \fB\-progress\fR \fIcallback\fR
 .
-The \fIcallback\fR is made after each transfer of data from the URL.
-The callback gets three additional arguments: the \fItoken\fR from
+If the \fB\-progress\fR option is present, 
+then the \fIcallback\fR is made after each transfer of data from the URL.
+The value of \fIcallback\fR is expanded, an additional three arguments are
+added, and the resulting command is evaluated.
+The three additional arguments are: the \fItoken\fR returned from
 \fB::http::geturl\fR, the expected total size of the contents from the
-\fBContent-Length\fR meta-data, and the current number of bytes
-transferred so far.  The expected total size may be unknown, in which
+\fBContent-Length\fR response header, and the current number of bytes
+transferred so far.  The token is the name of a global array that is
+described in the \fBSTATE ARRAY\fR section.  The expected total size may
+be unknown, in which
 case zero is passed to the callback.  Here is a template for the
 progress callback:
 .RS
 .PP
 .CS
 proc httpProgress {token total current} {
-    upvar #0 $token state
+    upvar 0 $token state
+    # Access state as a Tcl array defined in this proc
+    ...
+    return
 }
 .CE
 .RE
@@ -407,20 +486,24 @@ This flag causes \fB::http::geturl\fR to do a POST request that passes the
 data contained in \fIchannelID\fR to the server. The data contained in
 \fIchannelID\fR must be an x-url-encoding
 formatted query unless the \fB\-type\fR option below is used.
-If a Content-Length header is not specified via the \fB\-headers\fR options,
-\fB::http::geturl\fR attempts to determine the size of the post data
+If a \fBContent-Length\fR header is not specified via the \fB\-headers\fR
+options, \fB::http::geturl\fR attempts to determine the size of the post data
 in order to create that header.  If it is
 unable to determine the size, it returns an error.
 .TP
 \fB\-queryprogress\fR \fIcallback\fR
 .
-The \fIcallback\fR is made after each transfer of data to the URL
-(i.e. POST) and acts exactly like the \fB\-progress\fR option (the
-callback format is the same).
+If the \fB\-queryprogress\fR option is present, 
+then the \fIcallback\fR is made after each transfer of data to the URL
+in a POST request (i.e. a call to \fB::http::geturl\fR with
+option \fB\-query\fR or \fB\-querychannel\fR) and acts exactly like
+the \fB\-progress\fR option (the callback format is the same).
 .TP
 \fB\-strict\fR \fIboolean\fR
 .
-Whether to enforce RFC 3986 URL validation on the request.  Default is 1.
+If true then the command will test that the URL complies with RFC 3986, i.e.
+that it has no characters that should be "x-url-encoded" (e.g. a space should
+be encoded to "%20").  Default value is 1.
 .TP
 \fB\-timeout\fR \fImilliseconds\fR
 .
@@ -428,7 +511,8 @@ If \fImilliseconds\fR is non-zero, then \fB::http::geturl\fR sets up a timeout
 to occur after the specified number of milliseconds.
 A timeout results in a call to \fB::http::reset\fR and to
 the \fB\-command\fR callback, if specified.
-The return value of \fB::http::status\fR is \fBtimeout\fR
+The return value of \fB::http::status\fR (and the value of the \fIstatus\fR key
+in the dictionary returned by \fB::http::responseInfo\fR) is \fBtimeout\fR
 after a timeout has occurred.
 .TP
 \fB\-type\fR \fImime-type\fR
@@ -440,10 +524,11 @@ POST operation.
 \fB\-validate\fR \fIboolean\fR
 .
 If \fIboolean\fR is non-zero, then \fB::http::geturl\fR does an HTTP HEAD
-request.  This request returns meta information about the URL, but the
-contents are not returned.  The meta information is available in the
-\fBstate(meta) \fR variable after the transaction.  See the
-\fBSTATE ARRAY\fR section for details.
+request.  This server returns the same status line and response headers as it
+would for a HTTP GET request, but omits the response entity
+(the URL "contents").  The response headers are available after the
+transaction using command \fB::http::responseHeaders\fR or, for selected
+information, \fB::http::responseInfo\fR.
 .RE
 .TP
 \fB::http::formatQuery\fR \fIkey value\fR ?\fIkey value\fR ...?
@@ -467,7 +552,7 @@ This sets the \fBstate(status)\fR value to \fIwhy\fR, which defaults to
 .TP
 \fB::http::wait\fR \fItoken\fR
 .
-This is a convenience procedure that blocks and waits for the
+This command blocks and waits for the
 transaction to complete.  This only works in trusted code because it
 uses \fBvwait\fR.  Also, it is not useful for the case where
 \fB::http::geturl\fR is called \fIwithout\fR the \fB\-command\fR option
@@ -475,54 +560,210 @@ because in this case the \fB::http::geturl\fR call does not return
 until the HTTP transaction is complete, and thus there is nothing to
 wait for.
 .TP
-\fB::http::data\fR \fItoken\fR
-.
-This is a convenience procedure that returns the \fBbody\fR element
-(i.e., the URL data) of the state array.
-.TP
-\fB::http::error\fR \fItoken\fR
-.
-This is a convenience procedure that returns the \fBerror\fR element
-of the state array.
-.TP
 \fB::http::status\fR \fItoken\fR
 .
-This is a convenience procedure that returns the \fBstatus\fR element of
-the state array.
-.TP
-\fB::http::code\fR \fItoken\fR
-.
-This is a convenience procedure that returns the \fBhttp\fR element of the
-state array.
+This command returns a description of the status of the HTTP transaction.
+The return value is the empty string until the HTTP transaction is
+completed; after completion it has one of the values ok, eof, error,
+timeout, and reset.  The meaning of these values is described in the
+section \fBERRORS\fR (below).
+.PP
+.RS
+The name "status" is not related to the terms "status line" and
+"status code" that are defined for a HTTP response.
+.RE
 .TP
-\fB::http::ncode\fR \fItoken\fR
+\fB::http::size\fR \fItoken\fR
 .
-This is a convenience procedure that returns just the numeric return
-code (200, 404, etc.) from the \fBhttp\fR element of the state array.
+This command returns the number of bytes
+received so far from the URL in the \fB::http::geturl\fR call.
 .TP
-\fB::http::size\fR \fItoken\fR
+\fB::http::error\fR \fItoken\fR
 .
-This is a convenience procedure that returns the \fBcurrentsize\fR
-element of the state array, which represents the number of bytes
-received from the URL in the \fB::http::geturl\fR call.
+This command returns the error information if the HTTP transaction failed,
+or the empty string if there was no error.  The information is a Tcl list of
+the error message, stack trace, and error code.
 .TP
-\fB::http::meta\fR \fItoken\fR
+\fB::http::postError\fR \fItoken\fR
 .
-This is a convenience procedure that returns the \fBmeta\fR
-element of the state array which contains the HTTP response
-headers. See below for an explanation of this element.
+A POST request is a call to \fB::http::geturl\fR with either
+the \fB\-query\fR or \fB\-querychannel\fR option.
+The \fB::http::postError\fR command returns the error information generated
+when a HTTP POST request sends its request-body to the server; or the empty
+string if there was no error.  The information is a Tcl list of the error
+message, stack trace, and error code.  When this type of error occurs,
+the \fB::http::geturl\fR command continues the transaction and attempts to
+receive a response from the server.
 .TP
 \fB::http::cleanup\fR \fItoken\fR
 .
 This procedure cleans up the state associated with the connection
 identified by \fItoken\fR.  After this call, the procedures
-like \fB::http::data\fR cannot be used to get information
+like \fB::http::responseBody\fR cannot be used to get information
 about the operation.  It is \fIstrongly\fR recommended that you call
 this function after you are done with a given HTTP request.  Not doing
 so will result in memory not being freed, and if your app calls
 \fB::http::geturl\fR enough times, the memory leak could cause a
 performance hit...or worse.
 .TP
+\fB::http::requestLine\fR \fItoken\fR
+.
+This command returns the "request line" sent to the server.
+The "request line" is the first line of a HTTP client request, and has three
+elements separated by spaces: the HTTP method, the URL relative to the server,
+and the HTTP version. Examples:
+.PP
+.DS
+.RS
+GET / HTTP/1.1
+GET /introduction.html?subject=plumbing HTTP/1.1
+POST /forms/order.html HTTP/1.1
+.RE
+.DE
+.TP
+\fB::http::requestHeaders\fR \fItoken\fR ?\fIheaderName\fR?
+.
+This command returns the HTTP request header names and values, in the
+order that they were sent to the server, as a Tcl list of the form
+?name value ...?  Header names are case-insensitive and are converted to lower
+case.  The return value is not a \fBdict\fR because some header names may occur
+more than once.  If one argument is supplied, all request headers
+are returned.  If two arguments are supplied, the
+second provides the value of a header name.  Only headers with the requested
+name (converted to lower case) are returned.  If no such headers are found,
+an empty list is returned.
+.TP
+\fB::http::requestHeaderValue\fR \fItoken\fR \fIheaderName\fR
+.
+This command returns the value of the HTTP request header named
+\fIheaderName\fR.  Header names are case-insensitive and are converted to
+lower case.  If no such header exists, the return value is the empty string.
+If there are multiple headers named \fIheaderName\fR, the result is obtained
+by joining the individual values with the string ", " (comma and space),
+preserving their order.
+.TP
+\fB::http::responseLine\fR \fItoken\fR
+.
+This command returns the first line of the server response: the
+HTTP "status line".  The "status line" has three
+elements separated by spaces: the HTTP version, a three-digit numerical
+"status code", and a "reason phrase".  Only the reason phrase may contain
+spaces.  Examples:
+.PP
+.DS
+.RS
+HTTP/1.1 200 OK
+HTTP/1.0 404 Not Found
+.RE
+.DE
+.RS
+The "status code" is a three-digit number in the range 100 to 599.
+A value of 200 is the normal return from a GET request, and its matching
+"reason phrase" is "OK".  Codes beginning with 4 or 5 indicate errors.
+Codes beginning with 3 are redirection errors.  In this case the
+\fBLocation\fR response header specifies a new URL that contains the
+requested information.
+.PP
+The "reason phrase" is a textual description of the "status code": it may
+vary from server to server,
+and can be changed without affecting the HTTP protocol.  The recommended
+values (RFC 7231 and IANA assignments) for each code are provided by the
+command \fB::http::reasonPhrase\fR.
+.RE
+.TP
+\fB::http::responseCode\fR \fItoken\fR
+.
+This command returns the "status code" (200, 404, etc.) of the server
+"status line".  If a three-digit code cannot be found, the full status
+line is returned.  See command \fB::http::responseLine\fR for more information
+on the "status line".
+.TP
+\fB::http::reasonPhrase\fR \fIcode\fR
+.
+This command returns the IANA recommended "reason phrase" for a particular
+"status code" returned by a HTTP server.  The argument \fIcode\fR is a valid
+status code, and therefore is an integer in the range 100 to 599 inclusive.
+For numbers in this range with no assigned meaning, the command returns the
+value "Unassigned".  Several status codes are used only in response to the
+methods defined by HTTP extensions such as WebDAV, and not in response to a
+HEAD, GET, or POST request method.
+.PP
+.RS
+The "reason phrase" returned by a HTTP server may differ from the recommended
+value, without affecting the HTTP protocol.  The value returned by
+\fB::http::geturl\fR can be obtained by calling either command
+\fB::http::responseLine\fR (which returns the full status line) or command
+\fB::http::responseInfo\fR (which returns a dictionary, with
+the "reason phrase" stored in key \fIreasonPhrase\fR).
+.PP
+A registry of valid status codes is maintained at
+https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
+.RE
+.TP
+\fB::http::responseHeaders\fR \fItoken\fR ?\fIheaderName\fR?
+.
+The response from a HTTP server includes metadata headers that describe the
+response body and the transaction itself.
+This command returns the HTTP response header names and values, in the
+order that they were received from the server, as a Tcl list of the form
+?name value ...?  Header names are case-insensitive and are converted to lower
+case.  The return value is not a \fBdict\fR because some header names may occur
+more than once, notably \fBSet-Cookie\fR.  If the second argument is not
+supplied, all response headers are returned.  If the second argument is
+supplied, it provides the value of a header name.  Only headers with the
+requested name (converted to lower case) are returned.  If no such headers
+are found, an empty list is returned.  See section \fBMETADATA\fR for more
+information.
+.TP
+\fB::http::responseHeaderValue\fR \fItoken\fR \fIheaderName\fR
+.
+This command returns the value of the HTTP response header named
+\fIheaderName\fR.  Header names are case-insensitive and are converted to
+lower case.  If no such header exists, the return value is the empty string.
+If there are multiple headers named \fIheaderName\fR, the result is obtained
+by joining the individual values with the string ", " (comma and space),
+preserving their order.  Multiple headers with the same name may be processed
+in this manner, except \fBSet-Cookie\fR which does not conform to the
+comma-separated-list syntax and cannot be combined into a single value.
+Each \fBSet-Cookie\fR header must be treated individually, e.g. by processing
+the return value of \fB::http::responseHeaders\fR \fItoken\fR \fBSet-Cookie\fR.
+.TP
+\fB::http::responseInfo\fR \fItoken\fR
+.
+This command returns a \fBdict\fR of selected response metadata that are
+essential for identifying a successful transaction and making use of the
+response, along with other metadata that are informational.  The keys of
+the \fBdict\fR are \fIstage\fR, \fIstatus\fR, \fIresponseCode\fR,
+\fIreasonPhrase\fR, \fIcontentType\fR, \fIbinary\fR, \fIredirection\fR,
+\fIupgrade\fR, \fIerror\fR, \fIpostError\fR, \fImethod\fR, \fIcharset\fR,
+\fIcompression\fR, \fIhttpRequest\fR, \fIhttpResponse\fR, \fIurl\fR,
+\fIconnectionRequest\fR, \fIconnectionResponse\fR, \fIconnectionActual\fR,
+\fItransferEncoding\fR, \fItotalPost\fR, \fIcurrentPost\fR, \fItotalSize\fR,
+and \fIcurrentSize\fR.  The meaning of these keys is described in the
+section \fBMETADATA\fR below.
+.RS
+.PP
+It is always worth checking the value of \fIbinary\fR after a HTTP transaction,
+to determine whether a misconfigured server has caused http to interpret a
+text resource as a binary, or vice versa.
+.PP
+After a POST transaction, check the value of \fIpostError\fR to verify that
+the request body was uploaded without error.
+.RE
+.TP
+\fB::http::responseBody\fR \fItoken\fR
+.
+This command returns the entity sent by the HTTP server (unless
+\fI-channel\fR was used, in which case the entity was delivered to the
+channel, and the command returns the empty string).
+.RS
+.PP
+Other terms for
+"entity", with varying precision, include "representation of resource",
+"resource", "response body after decoding", "payload",
+"message body after decoding", "content(s)", and "file".
+.RE
+.TP
 \fB::http::register\fR \fIproto port command\fR
 .
 This procedure allows one to provide custom HTTP transport types
@@ -558,18 +799,34 @@ registered via \fB::http::register\fR, returning a two-item list of
 the default port and handler command that was previously installed
 (via \fB::http::register\fR) if there was such a handler, and an error if
 there was no such handler.
+.TP
+\fB::http::code\fR \fItoken\fR
+.
+An alternative name for the command \fB::http::responseLine\fR
+.TP
+\fB::http::data\fR \fItoken\fR
+.
+An alternative name for the command \fB::http::responseBody\fR.
+.TP
+\fB::http::meta\fR \fItoken\fR ?\fIheaderName\fR?
+.
+An alternative name for the command \fB::http::responseHeaders\fR
+.TP
+\fB::http::ncode\fR \fItoken\fR
+.
+An alternative name for the command \fB::http::responseCode\fR
 .SH ERRORS
 The \fB::http::geturl\fR procedure will raise errors in the following cases:
 invalid command line options,
-an invalid URL,
-a URL on a non-existent host,
-or a URL at a bad port on an existing host.
+or an invalid URL.
 These errors mean that it
 cannot even start the network transaction.
-It will also raise an error if it gets an I/O error while
-writing out the HTTP request header.
 For synchronous \fB::http::geturl\fR calls (where \fB\-command\fR is
-not specified), it will raise an error if it gets an I/O error while
+not specified), it will raise an error if
+the URL is on a non-existent host
+or at a bad port on an existing host.
+It will also raise an error for any I/O errors while
+writing out the HTTP request line and headers, or
 reading the HTTP reply headers or data.  Because \fB::http::geturl\fR
 does not return a token in these cases, it does all the required
 cleanup and there is no issue of your app having to call
@@ -581,13 +838,12 @@ HTTP reply headers or data, no exception is thrown.  This is because
 after writing the HTTP headers, \fB::http::geturl\fR returns, and the
 rest of the HTTP transaction occurs in the background.  The command
 callback can check if any error occurred during the read by calling
-\fB::http::status\fR to check the status and if its \fIerror\fR,
-calling \fB::http::error\fR to get the error message.
+\fB::http::responseInfo\fR to check the transaction status.
 .PP
 Alternatively, if the main program flow reaches a point where it needs
 to know the result of the asynchronous HTTP request, it can call
 \fB::http::wait\fR and then check status and error, just as the
-callback does.
+synchronous call does.
 .PP
 The \fB::http::geturl\fR command runs the \fB\-command\fR, \fB\-handler\fR,
 and \fB\-proxyfilter\fR callbacks inside a \fBcatch\fR command.  Therefore
@@ -601,15 +857,17 @@ In any case, you must still call
 \fB::http::cleanup\fR to delete the state array when you are done.
 .PP
 There are other possible results of the HTTP transaction
-determined by examining the status from \fB::http::status\fR.
+determined by examining the status from \fB::http::status\fR (or the value
+of the \fIstatus\fR key in the dictionary returned
+by \fB::http::responseInfo\fR).
 These are described below.
 .TP
 \fBok\fR
 .
 If the HTTP transaction completes entirely, then status will be \fBok\fR.
-However, you should still check the \fB::http::code\fR value to get
-the HTTP status.  The \fB::http::ncode\fR procedure provides just
-the numeric error (e.g., 200, 404 or 500) while the \fB::http::code\fR
+However, you should still check the \fB::http::responseLine\fR value to get
+the HTTP status.  The \fB::http::responseCode\fR procedure provides just
+the numeric error (e.g., 200, 404 or 500) while the \fB::http::responseLine\fR
 procedure returns a value like
 .QW "HTTP 404 File not found" .
 .TP
@@ -620,147 +878,447 @@ is raised, but the status of the transaction will be \fBeof\fR.
 .TP
 \fBerror\fR
 .
-The error message will also be stored in the \fBerror\fR status
-array element, accessible via \fB::http::error\fR.
+The error message, stack trace, and error code are accessible
+via \fB::http::error\fR.  The error message is also provided by the value of
+the \fIerror\fR key in the dictionary returned by \fB::http::responseInfo\fR.
 .TP
 \fBtimeout\fR
 .
-A timeout occurred before the transaction could complete
+A timeout occurred before the transaction could complete.
 .TP
 \fBreset\fR
 .
-user-reset
-.PP
-Another error possibility is that \fB::http::geturl\fR is unable to
-write all the post query data to the server before the server
-responds and closes the socket.
-The error message is saved in the \fBposterror\fR status array
-element and then  \fB::http::geturl\fR attempts to complete the
-transaction.
-If it can read the server's response
-it will end up with an \fBok\fR status, otherwise it will have
-an \fBeof\fR status.
+The user has called \fB::http::reset\fR.
+.TP
+\fB""\fR
+.
+(empty string) The transaction has not yet finished.
+.PP
+Another error possibility is that \fB::http::geturl\fR failed to
+write the whole of the POST request body (\fB-query\fR or \fB-querychannel\fR
+data) to the server.  \fB::http::geturl\fR stores the error message for later
+retrieval by the \fB::http::postError\fR or \fB::http::responseInfo\fR
+commands, and then attempts to complete the transaction.
+If it can read the server's response the status will be \fBok\fR, but it is
+important to call \fB::http::postError\fR or \fB::http::responseInfo\fR after
+every POST to check that the data was sent in full.
+If the server has closed the connection the status will be \fBeof\fR.
+.SH "METADATA"
+.PP
+.SS "MOST USEFUL METADATA"
+When a HTTP server responds to a request, it supplies not only the entity
+requested, but also metadata.  This is provided by the first line (the
+"status line") of the response, and by a number of HTTP headers.  Further
+metadata relates to how \fB::http::geturl\fR has processed the response
+from the server.
+.PP
+The most important metadata can be accessed with the command
+\fB::http::responseInfo\fR.
+This command returns a \fBdict\fR of metadata that are essential for
+identifying a successful transaction and making use of the response,
+along with other metadata that are informational.  The keys of
+the \fBdict\fR are:
+.PP
+.RS
+.RS
+\fB===== Essential Values =====\fR
+.RE
+.RE
+.TP
+\fBstage\fR
+.
+This value, set by \fB::http::geturl\fR, describes the stage that the
+transaction has reached. Values, in order of the transaction lifecycle,
+are: "created", "connecting", "header", "body", and "complete".  The
+other \fBdict\fR keys will not be available until the value of \fBstage\fR
+is "body" or "complete".  The key \fBcurrentSize\fR has its final value only
+when \fBstage\fR is "complete".
+.TP
+\fBstatus\fR
+.
+This value, set by \fB::http::geturl\fR, is "ok" for a successful transaction;
+"eof", "error", "timeout", or "reset" for an unsuccessful transaction; or ""
+if the transaction is still in progress.  The value is the same as that
+returned by command \fB::http::status\fR. The meaning of these values is
+described in the section \fBERRORS\fR (above).
+.TP
+\fBresponseCode\fR
+.
+The "HTTP status code" sent by the server in the first line (the "status line")
+of the response.  If the value cannot be extracted from the status line, the
+full status line is returned.
+.TP
+\fBreasonPhrase\fR
+.
+The "reason phrase" sent by the server as a description of the HTTP status code.
+If the value cannot be extracted from the status line, the full status
+line is returned.
+.TP
+\fBcontentType\fR
+.
+The value of the \fBContent-Type\fR response header or, if the header was not
+supplied, the default value "application/octet-stream".
+.TP
+\fBbinary\fR
+.
+This boolean value, set by \fB::http::geturl\fR, describes how the command
+has interpreted the entity returned by the server (after decoding any
+compression specified by the \fBContent-Encoding\fR response header).
+This decoded entity is accessible as the return value of the
+command \fB::http::responseBody\fR.
+.PP
+.RS
+The value is \fBtrue\fR if http has interpreted the decoded entity as binary.
+The value returned by \fB::http::responseBody\fR is a Tcl binary string.
+This is a suitable format for image data, zip files, etc.
+\fB::http::geturl\fR chooses this value if the user has requested a binary
+interpretation by passing the option \fI\-binary\fR to the command, or if the
+server has supplied a binary content type in a \fBContent-Type\fR response
+header, or if the server has not supplied any \fBContent-Type\fR header.
+.PP
+The value is \fBfalse\fR in other cases, and this means that http has
+interpreted the decoded entity as text. The text has been converted, from the
+character set notified by the server, into Tcl's internal Unicode format;
+the value returned by \fB::http::responseBody\fR is an ordinary Tcl string.
+.PP
+It is always worth checking the value of "binary" after a HTTP transaction,
+to determine whether a misconfigured server has caused http to interpret a
+text resource as a binary, or vice versa.
+.RE
+.TP
+\fBredirection\fR
+.
+The URL that is the redirection target. The value is that of the \fBLocation\fR
+response header.  This header is sent when a response has status code
+3XX (redirection).
+.TP
+\fBupgrade\fR
+.
+If not empty, the value indicates the protocol(s) to which the server will
+switch after completion of this transaction, while continuing to use the
+same connection.  When the server intends to switch protocols, it will also
+send the value "101" as the status code (the \fBresponseCode\fR key), and the
+word "upgrade" as an element of the \fBConnection\fR response header (the
+\fBconnectionResponse\fR key), and it will not send a response body.
+See the section \fBPROTOCOL UPGRADES\fR for more information.
+.TP
+\fBerror\fR
+.
+The error message, if there is one.  Further information, including a stack
+trace and error code, are available from command \fB::http::error\fR.
+.TP
+\fBpostError\fR
+.
+The error message (if any) generated when a HTTP POST request sends its
+request-body to the server.  Further information, including a stack trace
+and error code, are available from command \fB::http::postError\fR.  A POST
+transaction may appear complete, according to the
+keys \fBstage\fR, \fBstatus\fR, and \fBresponseCode\fR, but it is important
+to check this \fBpostError\fR key in case an error occurred when uploading
+the request-body.
+.PP
+.RS
+.RS
+\fB===== Informational Values =====\fR
+.RE
+.RE
+.TP
+\fBmethod\fR
+.
+The HTTP method used in the request.
+.TP
+\fBcharset\fR
+.
+The value of the charset attribute of the \fBContent-Type\fR response header.
+The charset value is used only for a text resource.  If the server did not
+specify a charset, the value defaults to that of the
+variable \fB::http::defaultCharset\fR, which unless it has been deliberately
+modified by the caller is \fBiso8859-1\fR.  Incoming text data is automatically
+converted from the character set defined by \fBcharset\fR to Tcl's internal
+Unicode representation, i.e. to a Tcl string.
+.TP
+\fBcompression\fR
+.
+A copy of the \fBContent-Encoding\fR response-header value.
+.TP
+\fBhttpRequest\fR
+.
+The version of HTTP specified in the request (i.e. sent in the request line).
+The value is that of the option \fB\-protocol\fR supplied
+to \fB::http::geturl\fR (default value "1.1"), unless the command reduced the
+value to "1.0" because it was passed the \fB\-handler\fR option.
+.TP
+\fBhttpResponse\fR
+.
+The version of HTTP used by the server (obtained from the response
+"status line").  The server uses this version of HTTP in its response, but
+ensures that this response is compatible with the HTTP version specified in the
+client's request.  If the value cannot be extracted from the status line, the
+full status line is returned.
+.TP
+\fBurl\fR
+.
+The requested URL, typically the URL supplied as an argument
+to \fB::http::geturl\fR but without its "fragment" (the final part of the URL
+beginning with "#").
+.TP
+\fBconnectionRequest\fR
+.
+The value, if any, sent to the server in \fBConnection\fR request header(s).
+.TP
+\fBconnectionResponse\fR
+.
+The value, if any, received from the server in \fBConnection\fR response
+header(s).
+.TP
+\fBconnectionActual\fR
+.
+This value, set by \fB::http::geturl\fR, reports whether the connection was
+closed after the transaction (value "close"), or left open (value "keep-alive").
+.TP
+\fBtransferEncoding\fR
+.
+The value of the Transfer-Encoding response header, if it is present.
+The value is either "chunked" (indicating HTTP/1.1 "chunked encoding") or
+the empty string.
+.TP
+\fBtotalPost\fR
+.
+The total length of the request body in a POST request.
+.TP
+\fBcurrentPost\fR
+.
+The number of bytes of the POST request body sent to the server so far.
+The value is the same as that returned by command \fB::http::size\fR.
+.TP
+\fBtotalSize\fR
+.
+A copy of the \fBContent-Length\fR response-header value.
+The number of bytes specified in a \fBContent-Length\fR header, if one
+was sent.  If none was sent, the value is 0.  A correctly configured server
+omits this header if the transfer-encoding is "chunked", or (for older
+servers) if the server closes the connection when it reaches the end of
+the resource.
+.TP
+\fBcurrentSize\fR
+.
+The number of bytes fetched from the server so far.
+.PP
+.SS "MORE METADATA"
+The dictionary returned by \fB::http::responseInfo\fR is the most useful
+subset of the available metadata.  Other metadata include:
+.PP
+1. The full "status line" of the response, available as the return value
+of command \fB::http::responseLine\fR.
+.PP
+2. The full response headers, available as the return value of
+command \fB::http::responseHeaders\fR.  This return value is a list of the
+response-header names and values, in the order that they were received from
+the server.
+.PP
+The return value is not a \fBdict\fR because some header names may
+occur more than once, notably \fBSet-Cookie\fR. If the value is read
+into a \fBdict\fR or into an array (using array set), only the last header
+with each name will be preserved.
+.PP
+.RS
+Some of the header names (metadata keys) are listed below, but the HTTP
+standard defines several more, and servers are free to add their own.
+When a dictionary key is mentioned below, this refers to the \fBdict\fR
+value returned by command \fB::http::responseInfo\fR.
+.TP
+\fBContent-Type\fR
+.
+The content type of the URL contents.  Examples include \fBtext/html\fR,
+\fBimage/gif,\fR \fBapplication/postscript\fR and
+\fBapplication/x-tcl\fR.  Text values typically specify a character set, e.g.
+\fBtext/html; charset=UTF-8\fR.  Dictionary key \fIcontentType\fR.
+.TP
+\fBContent-Length\fR
+.
+The advertised size in bytes of the contents, available as dictionary
+key \fItotalSize\fR.  The actual number of bytes read by \fB::http::geturl\fR
+so far is available as dictionary key \fBcurrentSize\fR.
+.TP
+\fBContent-Encoding\fR
+.
+The compression algorithm used for the contents.
+Examples include \fBgzip\fR, \fBdeflate\fR.
+Dictionary key \fIcontent\fR.
+.TP
+\fBLocation\fR
+.
+This header is sent when a response has status code 3XX (redirection).
+It provides the URL that is the redirection target.
+Dictionary key \fIredirection\fR.
+.TP
+\fBSet-Cookie\fR
+.
+This header is sent to offer a cookie to the client.  Cookie management is
+done by the \fB::http::config\fR option \fI\-cookiejar\fR, and so
+the \fBSet-Cookie\fR headers need not be parsed by user scripts.
+See section \fBCOOKIE JAR PROTOCOL\fR.
+.TP
+\fBConnection\fR
+.
+The value can be supplied as a comma-separated list, or by multiple headers.
+The list often has only one element, either "close" or "keep-alive".
+The value "upgrade" indicates a successful upgrade request and is typically
+combined with the status code 101, an \fBUpgrade\fR response header, and no
+response body.  Dictionary key \fIconnectionResponse\fR.
+.TP
+\fBUpgrade\fR
+.
+The value indicates the protocol(s) to which the server will switch
+immediately after the empty line that terminates the 101 response headers.
+Dictionary key \fIupgrade\fR.
+.RE
+.PP
+.SS "EVEN MORE METADATA"
+.PP
+1. Details of the HTTP request.  The request is determined by the options
+supplied to \fB::http::geturl\fR and \fB::http::config\fR.  However, it is
+sometimes helpful to examine what \fB::http::geturl\fR actually sent to the
+server, and this information is available through
+commands \fB::http::requestHeaders\fR and \fB::http::requestLine\fR.
+.PP
+2. The state array: the internal variables of \fB::http::geturl\fR.
+It may sometimes be helpful to examine this array.
+Details are given in the next section.
 .SH "STATE ARRAY"
-The \fB::http::geturl\fR procedure returns a \fItoken\fR that can be used to
-get to the state of the HTTP transaction in the form of a Tcl array.
-Use this construct to create an easy-to-use array variable:
+The \fB::http::geturl\fR procedure returns a \fItoken\fR that can be used
+as an argument to other \fB::http::*\fR commands, which examine and manage
+the state of the HTTP transaction.  For most purposes these commands are
+sufficient.  The \fItoken\fR can also be used to access
+the internal state of the transaction, which is stored in a Tcl array.
+This facility is most useful when writing callback commands for the
+options \fB\-command\fR, \fB\-handler\fR, \fB\-progress\fR,
+or \fB\-queryprogress\fR.
+Use the following command inside the proc to define an easy-to-use
+array \fIstate\fR as a local variable within the proc
 .PP
 .CS
-upvar #0 $token state
+upvar 0 $token state
 .CE
 .PP
 Once the data associated with the URL is no longer needed, the state
 array should be unset to free up storage.
 The \fB::http::cleanup\fR procedure is provided for that purpose.
-The following elements of
-the array are supported:
+.PP
+The following elements of the array are supported, and are the origin of the
+values returned by commands as described below.  When a dictionary key is
+mentioned below, this refers to the \fBdict\fR value returned by
+command \fB::http::responseInfo\fR.
 .RS
 .TP
 \fBbinary\fR
 .
-This is boolean \fBtrue\fR if (after decoding any compression specified
-by the
-.QW "Content-Encoding"
-response header) the HTTP response is binary.  It is boolean \fBfalse\fR
-if the HTTP response is text.
+For dictionary key \fIbinary\fR.
 .TP
 \fBbody\fR
 .
-The contents of the URL.  This will be empty if the \fB\-channel\fR
-option has been specified.  This value is returned by the \fB::http::data\fR
-command.
+For command \fB::http::responseBody\fR.
 .TP
 \fBcharset\fR
 .
-The value of the charset attribute from the \fBContent-Type\fR meta-data
-value.  If none was specified, this defaults to the RFC standard
-\fBiso8859-1\fR, or the value of \fB$::http::defaultCharset\fR.  Incoming
-text data will be automatically converted from this charset to utf-8.
+For dictionary key \fIcharset\fR.
 .TP
 \fBcoding\fR
 .
-A copy of the \fBContent-Encoding\fR meta-data value.
+For dictionary key \fIcompression\fR.
+.TP
+\fBconnection\fR
+.
+For dictionary key \fIconnectionActual\fR.
 .TP
 \fBcurrentsize\fR
 .
-The current number of bytes fetched from the URL.
-This value is returned by the \fB::http::size\fR command.
+For command \fB::http::size\fR; and for dictionary key \fIcurrentSize\fR.
 .TP
 \fBerror\fR
 .
-If defined, this is the error string seen when the HTTP transaction
-was aborted.
+For command \fB::http::error\fR; part is used in dictionary key \fIerror\fR.
 .TP
 \fBhttp\fR
 .
-The HTTP status reply from the server.  This value
-is returned by the \fB::http::code\fR command.  The format of this value is:
-.RS
-.PP
-.CS
-\fIHTTP/1.1 code string\fR
-.CE
-.PP
-The \fIcode\fR is a three-digit number defined in the HTTP standard.
-A code of 200 is OK.  Codes beginning with 4 or 5 indicate errors.
-Codes beginning with 3 are redirection errors.  In this case the
-\fBLocation\fR meta-data specifies a new URL that contains the
-requested information.
-.RE
+For command \fB::http::responseLine\fR.
+.TP
+\fBhttpResponse\fR
+.
+For dictionary key \fIhttpResponse\fR.
 .TP
 \fBmeta\fR
 .
-The HTTP protocol returns meta-data that describes the URL contents.
-The \fBmeta\fR element of the state array is a list of the keys and
-values of the meta-data.  This is in a format useful for initializing
-an array that just contains the meta-data:
-.RS
-.PP
-.CS
-array set meta $state(meta)
-.CE
-.PP
-Some of the meta-data keys are listed below, but the HTTP standard defines
-more, and servers are free to add their own.
+For command \fB::http::responseHeaders\fR. Further discussion above in the
+section \fBMORE METADATA\fR.
 .TP
-\fBContent-Type\fR
+\fBmethod\fR
 .
-The type of the URL contents.  Examples include \fBtext/html\fR,
-\fBimage/gif,\fR \fBapplication/postscript\fR and
-\fBapplication/x-tcl\fR.
+For dictionary key \fImethod\fR.
 .TP
-\fBContent-Length\fR
+\fBposterror\fR
 .
-The advertised size of the contents.  The actual size obtained by
-\fB::http::geturl\fR is available as \fBstate(currentsize)\fR.
+For dictionary key \fIpostError\fR.
 .TP
-\fBLocation\fR
+\fBpostErrorFull\fR
 .
-An alternate URL that contains the requested data.
-.RE
+For command \fB::http::postError\fR.
 .TP
-\fBposterror\fR
+\fB\-protocol\fR
+.
+For dictionary key \fIhttpRequest\fR.
+.TP
+\fBquerylength\fR
+.
+For dictionary key \fItotalPost\fR.
+.TP
+\fBqueryoffset\fR
 .
-The error, if any, that occurred while writing
-the post query data to the server.
+For dictionary key \fIcurrentPost\fR.
+.TP
+\fBreasonPhrase\fR
+.
+For dictionary key \fIreasonPhrase\fR.
+.TP
+\fBrequestHeaders\fR
+.
+For command \fB::http::requestHeaders\fR.
+.TP
+\fBrequestLine\fR
+.
+For command \fB::http::requestLine\fR.
+.TP
+\fBresponseCode\fR
+.
+For dictionary key \fIresponseCode\fR.
+.TP
+\fBstate\fR
+.
+For dictionary key \fIstage\fR.
 .TP
 \fBstatus\fR
 .
-See description in the chapter \fBERRORS\fR above for a
-list and description of \fBstatus\fR.
-During the transaction this value is the empty string.
+For command \fB::http::status\fR; and for dictionary key \fIstatus\fR.
 .TP
 \fBtotalsize\fR
 .
-A copy of the \fBContent-Length\fR meta-data value.
+For dictionary key \fItotalSize\fR.
+.TP
+\fBtransfer\fR
+.
+For dictionary key \fItransferEncoding\fR.
 .TP
 \fBtype\fR
 .
-A copy of the \fBContent-Type\fR meta-data value.
+For dictionary key \fIcontentType\fR.
+.TP
+\fBupgrade\fR
+.
+For dictionary key \fIupgrade\fR.
 .TP
 \fBurl\fR
 .
-The requested URL.
+For dictionary key \fIurl\fR.
 .RE
 .SH "PERSISTENT CONNECTIONS"
 .PP
@@ -859,7 +1417,7 @@ that fails because it uses a persistent connection that the server has
 half-closed (an
 .QW "asynchronous close event" ).
 Subsequent GET and HEAD requests in a failed pipeline will also be retried.
-\fIThe \-repost option should be used only if the application understands
+\fIThe \fB\-repost\fI option should be used only if the application understands
 that the retry is appropriate\fR - specifically, the application must know
 that if the failed POST successfully modified the state of the server, a repeat
 POST would have no adverse effect.
@@ -967,22 +1525,25 @@ Other keys may always be ignored; they have no meaning in this protocol.
 .VE TIP406
 .SH "PROTOCOL UPGRADES"
 .PP
-The HTTP/1.1 \fBConnection\fR and \fBUpgrade\fR client headers inform the server
-that the client wishes to change the protocol used over the existing connection
-(RFC 7230).  This mechanism can be used to request a WebSocket (RFC 6455), a
+The HTTP/1.1 \fBConnection\fR and \fBUpgrade\fR request headers inform the
+server that the client wishes to change the protocol used over the existing
+connection (RFC 7230).
+This mechanism can be used to request a WebSocket (RFC 6455), a
 higher version of the HTTP protocol (HTTP 2), or TLS encryption.  If the
 server accepts the upgrade request, its response code will be 101.
 .PP
-To request a protocol upgrade when calling \fBhttp::geturl\fR, the \fB\-headers\fR
-option must supply appropriate values for \fBConnection\fR and \fBUpgrade\fR, and
+To request a protocol upgrade when calling \fBhttp::geturl\fR,
+the \fB\-headers\fR option must supply appropriate values for \fBConnection\fR
+and \fBUpgrade\fR, and
 the \fB\-command\fR option must supply a command that implements the requested
 protocol and can also handle the server response if the server refuses the
 protocol upgrade.  For upgrade requests \fBhttp::geturl\fR ignores the value of
 option \fB\-keepalive\fR, and always uses the value \fB0\fR so that the upgrade
-request is not made over a connection that is intended for multiple HTTP requests.
+request is not made over a connection that is intended for multiple HTTP
+requests.
 .PP
-The Tcllib library \fBwebsocket\fR implements WebSockets, and makes the necessary
-calls to commands in the \fBhttp\fR package.
+The Tcllib library \fBwebsocket\fR implements WebSockets, and makes the
+necessary calls to commands in the \fBhttp\fR package.
 .PP
 There is currently no native Tcl client library for HTTP/2.
 .PP
@@ -993,30 +1554,59 @@ protocols such as Internet Printing Protocol (IPP) that are built on top of
 traffic.
 .PP
 In browsers, opportunistic encryption is instead implemented by the
-\fBUpgrade-Insecure-Requests\fR client header.  If a secure service is available,
-the server response code is a 307 redirect, and the response header
-\fBLocation\fR specifies the target URL.  The browser must call \fBhttp::geturl\fR
-again in order to fetch this URL.
+\fBUpgrade-Insecure-Requests\fR client header.  If a secure service is
+available, the server response code is a 307 redirect, and the response header
+\fBLocation\fR specifies the target URL.  The browser must
+call \fBhttp::geturl\fR again in order to fetch this URL.
 See https://w3c.github.io/webappsec-upgrade-insecure-requests/
 .PP
 .SH THREADS
 .PP
 .SS "PURPOSE"
 .PP
-Command \fB::http::geturl\fR uses the Tcl \fB::socket\fR command with the \-async option to connect to a remote server, but the return from this command can be delayed in adverse cases (e.g. a slow DNS lookup), preventing the event loop from processing other events.  This delay is avoided if the \fB::socket\fR command is evaluated in another thread.  The Thread package is not part of Tcl but is provided in "Batteries Included" distributions.  Instead of the \fB::socket\fR command, the http package uses \fB::http::socket\fR which makes connections in the manner specified by the value of \-threadlevel and the availability of package Thread.
+Command \fB::http::geturl\fR uses the Tcl \fB::socket\fR command with
+the \fI\-async\fR option to connect to a remote server, but the return from
+this command can be delayed in adverse cases (e.g. a slow DNS lookup),
+preventing the event loop from processing other events.
+This delay is avoided if the \fB::socket\fR command is evaluated in another
+thread.  The Thread package is not part of Tcl but is provided in
+"Batteries Included" distributions.  Instead of the \fB::socket\fR command,
+the http package uses \fB::http::socket\fR which makes connections in the
+manner specified by the value of \fI\-threadlevel\fR and the availability
+of package Thread.
 .PP
 .SS "WITH TLS (HTTPS)"
 .PP
-The same \-threadlevel configuration applies to both HTTP and HTTPS connections. HTTPS is enabled by using the \fBhttp::register\fR command, typically by specifying the \fB::tls::socket\fR command of the tls package to handle TLS cryptography.  The \fB::tls::socket\fR command connects to the remote server by using the command specified by the value of variable \fB::tls::socketCmd\fR, and this value defaults to "::socket".  If http::geturl finds that \fB::tls::socketCmd\fR has this value, it replaces it with the value "::http::socket".  If \fB::tls::socketCmd\fR has a value other than "::socket", i.e. if the script or the Tcl installation has replaced the value "::socket" with the name of a different command, then http does not change the value.  The script or installation that modified \fB::tls::socketCmd\fR is responsible for integrating \fB::http::socket\fR into its own replacement command.
+The same \fI\-threadlevel\fR configuration applies to both HTTP and HTTPS
+connections.
+HTTPS is enabled by using the \fBhttp::register\fR command, typically by
+specifying the \fB::tls::socket\fR command of the tls package to handle TLS
+cryptography.  The \fB::tls::socket\fR command connects to the remote server by
+using the command specified by the value of variable \fB::tls::socketCmd\fR, and
+this value defaults to "::socket".  If http::geturl finds
+that \fB::tls::socketCmd\fR has this value, it replaces it with the value
+"::http::socket".  If \fB::tls::socketCmd\fR has a value other than "::socket",
+i.e. if the script or the Tcl installation has replaced the value "::socket"
+with the name of a different command, then http does not change the value.
+The script or installation that modified \fB::tls::socketCmd\fR is responsible
+for integrating \fR::http::socket\fR into its own replacement command.
 .PP
 .SS "WITH A CHILD INTERPRETER"
 .PP
-The peer thread can transfer the socket only to the main interpreter of the script's thread.  Therefore the thread-based \fB::http::socket\fR works with non-zero \-threadlevel values only if the script runs in the main interpreter.  A child interpreter must use \-threadlevel 0 unless the parent interpreter has provided alternative facilities.  The main parent interpreter may grant full \-threadlevel facilities to a child interpreter, for example by aliasing, to \fB::http::socket\fR in the child, a command that runs \fBhttp::socket\fR in the parent, and then transfers the socket to the child.
+The peer thread can transfer the socket only to the main interpreter of the 
+script's thread.  Therefore the thread-based \fB::http::socket\fR works with
+non-zero \fI\-threadlevel\fR values only if the script runs in the main
+interpreter.  A child interpreter must use \fI\-threadlevel 0\fR unless the
+parent interpreter has provided alternative facilities.  The main parent
+interpreter may grant full \fI\-threadlevel\fR facilities to a child
+interpreter, for example by aliasing, to \fB::http::socket\fR in the child,
+a command that runs \fBhttp::socket\fR in the parent, and then transfers
+the socket to the child.
 .PP
 .SH EXAMPLE
 .PP
 This example creates a procedure to copy a URL to a file while printing a
-progress meter, and prints the meta-data associated with the URL.
+progress meter, and prints the response headers associated with the URL.
 .PP
 .CS
 proc httpcopy { url file {chunk 4096} } {
@@ -1028,7 +1618,7 @@ proc httpcopy { url file {chunk 4096} } {
     # This ends the line started by httpCopyProgress
     puts stderr ""
 
-    upvar #0 $token state
+    upvar 0 $token state
     set max 0
     foreach {name value} $state(meta) {
         if {[string length $name] > $max} {