diff options
Diffstat (limited to 'doc/http.n')
| -rw-r--r-- | doc/http.n | 984 |
1 files changed, 787 insertions, 197 deletions
@@ -13,7 +13,7 @@ .SH NAME http \- Client-side implementation of the HTTP/1.1 protocol .SH SYNOPSIS -\fBpackage require http\fI ?\fB2.10\fR? +\fBpackage require http\fR ?\fB2.10\fR? .\" See Also -useragent option documentation in body! .sp \fB::http::config\fR ?\fI\-option value\fR ...? @@ -32,36 +32,67 @@ http \- Client-side implementation of the HTTP/1.1 protocol .sp \fB::http::size \fItoken\fR .sp -\fB::http::code \fItoken\fR +\fB::http::error \fItoken\fR .sp -\fB::http::ncode \fItoken\fR +\fB::http::postError \fItoken\fR +.sp +\fB::http::cleanup \fItoken\fR .sp -\fB::http::meta \fItoken\fR +\fB::http::requestLine\fR \fItoken\fR .sp -\fB::http::data \fItoken\fR +\fB::http::requestHeaders\fR \fItoken\fR ?\fIheaderName\fR? .sp -\fB::http::error \fItoken\fR +\fB::http::requestHeaderValue\fR \fItoken\fR \fIheaderName\fR .sp -\fB::http::cleanup \fItoken\fR +\fB::http::responseLine\fR \fItoken\fR +.sp +\fB::http::responseCode\fR \fItoken\fR +.sp +\fB::http::reasonPhrase\fR \fIcode\fR +.sp +\fB::http::responseHeaders\fR \fItoken\fR ?\fIheaderName\fR? +.sp +\fB::http::responseHeaderValue\fR \fItoken\fR \fIheaderName\fR +.sp +\fB::http::responseInfo\fR \fItoken\fR +.sp +\fB::http::responseBody\fR \fItoken\fR .sp \fB::http::register \fIproto port command\fR .sp \fB::http::registerError \fIport\fR ?\fImessage\fR? .sp \fB::http::unregister \fIproto\fR +.sp +\fB::http::code \fItoken\fR +.sp +\fB::http::data \fItoken\fR +.sp +\fB::http::meta \fItoken\fR ?\fIheaderName\fR? +.sp +\fB::http::metaValue\fR \fItoken\fR \fIheaderName\fR +.sp +\fB::http::ncode \fItoken\fR .SH "EXPORTED COMMANDS" .PP Namespace \fBhttp\fR exports the commands \fBconfig\fR, \fBformatQuery\fR, -\fBgeturl\fR, \fBquoteString\fR, \fBregister\fR, \fBregisterError\fR, +\fBgeturl\fR, \fBpostError\fR, \fBquoteString\fR, \fBreasonPhrase\fR, +\fBregister\fR, +\fBregisterError\fR, \fBrequestHeaders\fR, \fBrequestHeaderValue\fR, +\fBrequestLine\fR, \fBresponseBody\fR, \fBresponseCode\fR, +\fBresponseHeaders\fR, \fBresponseHeaderValue\fR, \fBresponseInfo\fR, +\fBresponseLine\fR, \fBreset\fR, \fBunregister\fR, and \fBwait\fR. .PP It does not export the commands \fBcleanup\fR, \fBcode\fR, \fBdata\fR, -\fBerror\fR, \fBmeta\fR, \fBncode\fR, \fBsize\fR, or \fBstatus\fR. +\fBerror\fR, \fBmeta\fR, \fBmetaValue\fR, \fBncode\fR, +\fBsize\fR, or \fBstatus\fR. .BE .SH DESCRIPTION .PP The \fBhttp\fR package provides the client side of the HTTP/1.1 -protocol, as defined in RFC 7230 to RFC 7235, which supersede RFC 2616. +protocol, as defined in RFC 9110 to 9112, which supersede RFC 7230 +to RFC 7235, which in turn supersede RFC 2616. The package implements the GET, POST, and HEAD operations of HTTP/1.1. It allows configuration of a proxy host to get through firewalls. The package is compatible with the \fBSafesock\fR security @@ -74,14 +105,13 @@ The \fB::http::geturl\fR procedure does a HTTP transaction. Its \fIoptions \fR determine whether a GET, POST, or HEAD transaction is performed. The return value of \fB::http::geturl\fR is a token for the transaction. -The value is also the name of an array in the ::http namespace -that contains state information about the transaction. The elements -of this array are described in the \fBSTATE ARRAY\fR section. +The token can be supplied as an argument to other commands, to manage the +transaction and examine its results. .PP If the \fB\-command\fR option is specified, then the HTTP operation is done in the background. \fB::http::geturl\fR returns immediately after generating the -HTTP request and the callback is invoked +HTTP request and the \fB\-command\fR callback is invoked when the transaction completes. For this to work, the Tcl event loop must be active. In Tk applications this is always true. For pure-Tcl applications, the caller can use \fB::http::wait\fR after calling @@ -90,6 +120,15 @@ applications, the caller can use \fB::http::wait\fR after calling \fBNote:\fR The event queue is even used without the \fB\-command\fR option. As a side effect, arbitrary commands may be processed while \fBhttp::geturl\fR is running. +.PP +When the HTTP server has replied to the request, call the command +\fB::http::responseInfo\fR, which +returns a \fBdict\fR of metadata that is essential for identifying a +successful transaction and making use of the response. See +section \fBMETADATA\fR for details of the information returned. +The response itself is returned by command \fB::http::responseBody\fR, +unless it has been redirected to a file by the \fI\-channel\fR option +of \fB::http::geturl\fR. .SH COMMANDS .TP \fB::http::config\fR ?\fIoptions\fR? @@ -175,7 +214,8 @@ default is 0. .TP \fB\-threadlevel\fR \fIlevel\fR . -Specifies whether and how to use the \fBThread\fR package. Possible values of \fIlevel\fR are 0, 1 or 2. +Specifies whether and how to use the \fBThread\fR package. Possible values +of \fIlevel\fR are 0, 1 or 2. .RS .PP .DS @@ -183,7 +223,11 @@ Specifies whether and how to use the \fBThread\fR package. Possible values of \ 1 - use Thread if it is available, do not use it if it is unavailable 2 - use Thread if it is available, raise an error if it is unavailable .DE -The Tcl \fBsocket -async\fR command can block in adverse cases (e.g. a slow DNS lookup). Using the Thread package works around this problem, for both HTTP and HTTPS transactions. Values of \fIlevel\fR other than 0 are available only to the main interpreter in each thread. See section \fBTHREADS\fR for more information. +The Tcl \fBsocket -async\fR command can block in adverse cases (e.g. a slow +DNS lookup). Using the Thread package works around this problem, for both +HTTP and HTTPS transactions. Values of \fIlevel\fR other than 0 are +available only to the main interpreter in each thread. See +section \fBTHREADS\fR for more information. .RE .TP \fB\-urlencoding\fR \fIencoding\fR @@ -205,21 +249,22 @@ numbers of \fBhttp\fR and \fBTcl\fR. \fB\-zip\fR \fIboolean\fR . If the value is boolean \fBtrue\fR, then by default requests will send a header -.QW "\fBAccept-Encoding: gzip,deflate,compress\fR" . -If the value is boolean \fBfalse\fR, then by default this header will not be -sent. In either case the default can be overridden for an individual request by +.QW "\fBAccept-Encoding: gzip,deflate\fR" . +If the value is boolean \fBfalse\fR, then by default requests will send a header +.QW "\fBAccept-Encoding: identity\fR" . +In either case the default can be overridden for an individual request by supplying a custom \fBAccept-Encoding\fR header in the \fB\-headers\fR option -of \fBhttp::geturl\fR. The default is 1. +of \fBhttp::geturl\fR. The default value is 1. .RE .TP \fB::http::geturl\fR \fIurl\fR ?\fIoptions\fR? . The \fB::http::geturl\fR command is the main procedure in the package. -The \fB\-query\fR option causes a POST operation and +The \fB\-query\fR or \fB\-querychannel\fR option causes a POST operation and the \fB\-validate\fR option causes a HEAD operation; otherwise, a GET operation is performed. The \fB::http::geturl\fR command -returns a \fItoken\fR value that can be used to get -information about the transaction. See the \fBSTATE ARRAY\fR and +returns a \fItoken\fR value that can be passed as an argument to other commands +to get information about the transaction. See the \fBMETADATA\fR and \fBERRORS\fR section for details. The \fB::http::geturl\fR command blocks until the operation completes, unless the \fB\-command\fR option specifies a callback @@ -231,7 +276,7 @@ that is invoked when the HTTP transaction completes. . Specifies whether to force interpreting the URL data as binary. Normally this is auto-detected (anything not beginning with a \fBtext\fR content -type or whose content encoding is \fBgzip\fR or \fBcompress\fR is +type or whose content encoding is \fBgzip\fR or \fBdeflate\fR is considered binary data). .TP \fB\-blocksize\fR \fIsize\fR @@ -243,13 +288,14 @@ At most \fIsize\fR bytes are read at once. After each block, a call to the \fB\-channel\fR \fIname\fR . Copy the URL contents to channel \fIname\fR instead of saving it in -\fBstate(body)\fR. +a Tcl variable for retrieval by \fB::http::responseBody\fR. .TP \fB\-command\fR \fIcallback\fR . -Invoke \fIcallback\fR after the HTTP transaction completes. -This option causes \fB::http::geturl\fR to return immediately. -The \fIcallback\fR gets an additional argument that is the \fItoken\fR returned +The presence of this option causes \fB::http::geturl\fR to return immediately. +After the HTTP transaction completes, the value of \fIcallback\fR is expanded, +an additional argument is added, and the resulting command is evaluated. +The additional argument is the \fItoken\fR returned from \fB::http::geturl\fR. This token is the name of an array that is described in the \fBSTATE ARRAY\fR section. Here is a template for the callback: @@ -257,8 +303,10 @@ callback: .PP .CS proc httpCallback {token} { - upvar #0 $token state - # Access state as a Tcl array + upvar 0 $token state + # Access state as a Tcl array defined in this proc + ... + return } .CE .PP @@ -268,11 +316,30 @@ not call the \fBbgerror\fR handler. See the \fBERRORS\fR section for details. .RE .TP +\fB\-guesstype\fR \fIboolean\fR +. +Attempt to guess the \fBContent-Type\fR and character set when a misconfigured +server provides no information. The default value is \fIfalse\fR (do +nothing). If boolean \fItrue\fR then, if the server does not send a +\fBContent-Type\fR header, or if it sends the value "application/octet-stream", +\fBhttp::geturl\fR will attempt to guess appropriate values. This is not +intended to become a general-purpose tool, and currently it is limited to +detecting XML documents that begin with an XML declaration. In this case +the \fBContent-Type\fR is changed to "application/xml", the binary flag +state(binary) is changed to 0, and the character set is changed to +the one specified by the "encoding" tag of the XML line, or to utf-8 if no +encoding is specified. Not used if a \fI\-channel\fR is specified. +.TP \fB\-handler\fR \fIcallback\fR . -Invoke \fIcallback\fR whenever HTTP data is available; if present, nothing -else will be done with the HTTP data. This procedure gets two additional -arguments: the socket for the HTTP data and the \fItoken\fR returned from +If this option is absent, \fBhttp::geturl\fR processes incoming data itself, +either appending it to the state(body) variable or writing it to the -channel. +But if the \fB\-handler\fR option is present, \fBhttp::geturl\fR does not do +this processing and instead calls \fIcallback\fR. +Whenever HTTP data is available, the value of \fIcallback\fR is expanded, an +additional two arguments are added, and the resulting command is evaluated. +The two additional +arguments are: the socket for the HTTP data and the \fItoken\fR returned from \fB::http::geturl\fR. The token is the name of a global array that is described in the \fBSTATE ARRAY\fR section. The procedure is expected to return the number of bytes read from the socket. Here is a @@ -281,8 +348,8 @@ template for the callback: .PP .CS proc httpHandlerCallback {socket token} { - upvar #0 $token state - # Access socket, and state as a Tcl array + upvar 0 $token state + # Access socket, and state as a Tcl array defined in this proc # For example... ... set data [read $socket 1000] @@ -295,8 +362,9 @@ proc httpHandlerCallback {socket token} { The \fBhttp::geturl\fR code for the \fB\-handler\fR option is not compatible with either compression or chunked transfer-encoding. If \fB\-handler\fR is specified, then to work around these issues \fBhttp::geturl\fR will reduce the -HTTP protocol to 1.0, and override the \fB\-zip\fR option (i.e. it will not -send the header "\fBAccept-Encoding: gzip,deflate,compress\fR"). +HTTP protocol to 1.0, and override the \fB\-zip\fR option (i.e. it will +send the header \fBAccept-Encoding: identity\fR instead +of \fBAccept-Encoding: gzip,deflate\fR). .PP If options \fB\-handler\fR and \fB\-channel\fR are used together, the handler is responsible for copying the data from the HTTP socket to the specified @@ -342,7 +410,10 @@ It is the caller's responsibility to ensure that the headers and request body (if any) conform to the requirements of the request method. For example, if using \fB\-method\fR \fIPOST\fR to send a POST with an empty request body, the caller must also supply the option -.QW "\-headers {Content-Length 0}" . +.PP +.CS +\-headers {Content-Length 0} +.CE .RE .TP \fB\-myaddr\fR \fIaddress\fR @@ -352,18 +423,26 @@ multiple interfaces are available. .TP \fB\-progress\fR \fIcallback\fR . -The \fIcallback\fR is made after each transfer of data from the URL. -The callback gets three additional arguments: the \fItoken\fR from +If the \fB\-progress\fR option is present, +then the \fIcallback\fR is made after each transfer of data from the URL. +The value of \fIcallback\fR is expanded, an additional three arguments are +added, and the resulting command is evaluated. +The three additional arguments are: the \fItoken\fR returned from \fB::http::geturl\fR, the expected total size of the contents from the -\fBContent-Length\fR meta-data, and the current number of bytes -transferred so far. The expected total size may be unknown, in which +\fBContent-Length\fR response header, and the current number of bytes +transferred so far. The token is the name of a global array that is +described in the \fBSTATE ARRAY\fR section. The expected total size may +be unknown, in which case zero is passed to the callback. Here is a template for the progress callback: .RS .PP .CS proc httpProgress {token total current} { - upvar #0 $token state + upvar 0 $token state + # Access state as a Tcl array defined in this proc + ... + return } .CE .RE @@ -407,20 +486,24 @@ This flag causes \fB::http::geturl\fR to do a POST request that passes the data contained in \fIchannelID\fR to the server. The data contained in \fIchannelID\fR must be an x-url-encoding formatted query unless the \fB\-type\fR option below is used. -If a Content-Length header is not specified via the \fB\-headers\fR options, -\fB::http::geturl\fR attempts to determine the size of the post data +If a \fBContent-Length\fR header is not specified via the \fB\-headers\fR +options, \fB::http::geturl\fR attempts to determine the size of the post data in order to create that header. If it is unable to determine the size, it returns an error. .TP \fB\-queryprogress\fR \fIcallback\fR . -The \fIcallback\fR is made after each transfer of data to the URL -(i.e. POST) and acts exactly like the \fB\-progress\fR option (the -callback format is the same). +If the \fB\-queryprogress\fR option is present, +then the \fIcallback\fR is made after each transfer of data to the URL +in a POST request (i.e. a call to \fB::http::geturl\fR with +option \fB\-query\fR or \fB\-querychannel\fR) and acts exactly like +the \fB\-progress\fR option (the callback format is the same). .TP \fB\-strict\fR \fIboolean\fR . -Whether to enforce RFC 3986 URL validation on the request. Default is 1. +If true then the command will test that the URL complies with RFC 3986, i.e. +that it has no characters that should be "x-url-encoded" (e.g. a space should +be encoded to "%20"). Default value is 1. .TP \fB\-timeout\fR \fImilliseconds\fR . @@ -428,7 +511,8 @@ If \fImilliseconds\fR is non-zero, then \fB::http::geturl\fR sets up a timeout to occur after the specified number of milliseconds. A timeout results in a call to \fB::http::reset\fR and to the \fB\-command\fR callback, if specified. -The return value of \fB::http::status\fR is \fBtimeout\fR +The return value of \fB::http::status\fR (and the value of the \fIstatus\fR key +in the dictionary returned by \fB::http::responseInfo\fR) is \fBtimeout\fR after a timeout has occurred. .TP \fB\-type\fR \fImime-type\fR @@ -440,10 +524,11 @@ POST operation. \fB\-validate\fR \fIboolean\fR . If \fIboolean\fR is non-zero, then \fB::http::geturl\fR does an HTTP HEAD -request. This request returns meta information about the URL, but the -contents are not returned. The meta information is available in the -\fBstate(meta) \fR variable after the transaction. See the -\fBSTATE ARRAY\fR section for details. +request. This server returns the same status line and response headers as it +would for a HTTP GET request, but omits the response entity +(the URL "contents"). The response headers are available after the +transaction using command \fB::http::responseHeaders\fR or, for selected +information, \fB::http::responseInfo\fR. .RE .TP \fB::http::formatQuery\fR \fIkey value\fR ?\fIkey value\fR ...? @@ -467,7 +552,7 @@ This sets the \fBstate(status)\fR value to \fIwhy\fR, which defaults to .TP \fB::http::wait\fR \fItoken\fR . -This is a convenience procedure that blocks and waits for the +This command blocks and waits for the transaction to complete. This only works in trusted code because it uses \fBvwait\fR. Also, it is not useful for the case where \fB::http::geturl\fR is called \fIwithout\fR the \fB\-command\fR option @@ -475,54 +560,210 @@ because in this case the \fB::http::geturl\fR call does not return until the HTTP transaction is complete, and thus there is nothing to wait for. .TP -\fB::http::data\fR \fItoken\fR -. -This is a convenience procedure that returns the \fBbody\fR element -(i.e., the URL data) of the state array. -.TP -\fB::http::error\fR \fItoken\fR -. -This is a convenience procedure that returns the \fBerror\fR element -of the state array. -.TP \fB::http::status\fR \fItoken\fR . -This is a convenience procedure that returns the \fBstatus\fR element of -the state array. -.TP -\fB::http::code\fR \fItoken\fR -. -This is a convenience procedure that returns the \fBhttp\fR element of the -state array. +This command returns a description of the status of the HTTP transaction. +The return value is the empty string until the HTTP transaction is +completed; after completion it has one of the values ok, eof, error, +timeout, and reset. The meaning of these values is described in the +section \fBERRORS\fR (below). +.PP +.RS +The name "status" is not related to the terms "status line" and +"status code" that are defined for a HTTP response. +.RE .TP -\fB::http::ncode\fR \fItoken\fR +\fB::http::size\fR \fItoken\fR . -This is a convenience procedure that returns just the numeric return -code (200, 404, etc.) from the \fBhttp\fR element of the state array. +This command returns the number of bytes +received so far from the URL in the \fB::http::geturl\fR call. .TP -\fB::http::size\fR \fItoken\fR +\fB::http::error\fR \fItoken\fR . -This is a convenience procedure that returns the \fBcurrentsize\fR -element of the state array, which represents the number of bytes -received from the URL in the \fB::http::geturl\fR call. +This command returns the error information if the HTTP transaction failed, +or the empty string if there was no error. The information is a Tcl list of +the error message, stack trace, and error code. .TP -\fB::http::meta\fR \fItoken\fR +\fB::http::postError\fR \fItoken\fR . -This is a convenience procedure that returns the \fBmeta\fR -element of the state array which contains the HTTP response -headers. See below for an explanation of this element. +A POST request is a call to \fB::http::geturl\fR with either +the \fB\-query\fR or \fB\-querychannel\fR option. +The \fB::http::postError\fR command returns the error information generated +when a HTTP POST request sends its request-body to the server; or the empty +string if there was no error. The information is a Tcl list of the error +message, stack trace, and error code. When this type of error occurs, +the \fB::http::geturl\fR command continues the transaction and attempts to +receive a response from the server. .TP \fB::http::cleanup\fR \fItoken\fR . This procedure cleans up the state associated with the connection identified by \fItoken\fR. After this call, the procedures -like \fB::http::data\fR cannot be used to get information +like \fB::http::responseBody\fR cannot be used to get information about the operation. It is \fIstrongly\fR recommended that you call this function after you are done with a given HTTP request. Not doing so will result in memory not being freed, and if your app calls \fB::http::geturl\fR enough times, the memory leak could cause a performance hit...or worse. .TP +\fB::http::requestLine\fR \fItoken\fR +. +This command returns the "request line" sent to the server. +The "request line" is the first line of a HTTP client request, and has three +elements separated by spaces: the HTTP method, the URL relative to the server, +and the HTTP version. Examples: +.PP +.DS +.RS +GET / HTTP/1.1 +GET /introduction.html?subject=plumbing HTTP/1.1 +POST /forms/order.html HTTP/1.1 +.RE +.DE +.TP +\fB::http::requestHeaders\fR \fItoken\fR ?\fIheaderName\fR? +. +This command returns the HTTP request header names and values, in the +order that they were sent to the server, as a Tcl list of the form +?name value ...? Header names are case-insensitive and are converted to lower +case. The return value is not a \fBdict\fR because some header names may occur +more than once. If one argument is supplied, all request headers +are returned. If two arguments are supplied, the +second provides the value of a header name. Only headers with the requested +name (converted to lower case) are returned. If no such headers are found, +an empty list is returned. +.TP +\fB::http::requestHeaderValue\fR \fItoken\fR \fIheaderName\fR +. +This command returns the value of the HTTP request header named +\fIheaderName\fR. Header names are case-insensitive and are converted to +lower case. If no such header exists, the return value is the empty string. +If there are multiple headers named \fIheaderName\fR, the result is obtained +by joining the individual values with the string ", " (comma and space), +preserving their order. +.TP +\fB::http::responseLine\fR \fItoken\fR +. +This command returns the first line of the server response: the +HTTP "status line". The "status line" has three +elements separated by spaces: the HTTP version, a three-digit numerical +"status code", and a "reason phrase". Only the reason phrase may contain +spaces. Examples: +.PP +.DS +.RS +HTTP/1.1 200 OK +HTTP/1.0 404 Not Found +.RE +.DE +.RS +The "status code" is a three-digit number in the range 100 to 599. +A value of 200 is the normal return from a GET request, and its matching +"reason phrase" is "OK". Codes beginning with 4 or 5 indicate errors. +Codes beginning with 3 are redirection errors. In this case the +\fBLocation\fR response header specifies a new URL that contains the +requested information. +.PP +The "reason phrase" is a textual description of the "status code": it may +vary from server to server, +and can be changed without affecting the HTTP protocol. The recommended +values (RFC 7231 and IANA assignments) for each code are provided by the +command \fB::http::reasonPhrase\fR. +.RE +.TP +\fB::http::responseCode\fR \fItoken\fR +. +This command returns the "status code" (200, 404, etc.) of the server +"status line". If a three-digit code cannot be found, the full status +line is returned. See command \fB::http::responseLine\fR for more information +on the "status line". +.TP +\fB::http::reasonPhrase\fR \fIcode\fR +. +This command returns the IANA recommended "reason phrase" for a particular +"status code" returned by a HTTP server. The argument \fIcode\fR is a valid +status code, and therefore is an integer in the range 100 to 599 inclusive. +For numbers in this range with no assigned meaning, the command returns the +value "Unassigned". Several status codes are used only in response to the +methods defined by HTTP extensions such as WebDAV, and not in response to a +HEAD, GET, or POST request method. +.PP +.RS +The "reason phrase" returned by a HTTP server may differ from the recommended +value, without affecting the HTTP protocol. The value returned by +\fB::http::geturl\fR can be obtained by calling either command +\fB::http::responseLine\fR (which returns the full status line) or command +\fB::http::responseInfo\fR (which returns a dictionary, with +the "reason phrase" stored in key \fIreasonPhrase\fR). +.PP +A registry of valid status codes is maintained at +https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml +.RE +.TP +\fB::http::responseHeaders\fR \fItoken\fR ?\fIheaderName\fR? +. +The response from a HTTP server includes metadata headers that describe the +response body and the transaction itself. +This command returns the HTTP response header names and values, in the +order that they were received from the server, as a Tcl list of the form +?name value ...? Header names are case-insensitive and are converted to lower +case. The return value is not a \fBdict\fR because some header names may occur +more than once, notably \fBSet-Cookie\fR. If the second argument is not +supplied, all response headers are returned. If the second argument is +supplied, it provides the value of a header name. Only headers with the +requested name (converted to lower case) are returned. If no such headers +are found, an empty list is returned. See section \fBMETADATA\fR for more +information. +.TP +\fB::http::responseHeaderValue\fR \fItoken\fR \fIheaderName\fR +. +This command returns the value of the HTTP response header named +\fIheaderName\fR. Header names are case-insensitive and are converted to +lower case. If no such header exists, the return value is the empty string. +If there are multiple headers named \fIheaderName\fR, the result is obtained +by joining the individual values with the string ", " (comma and space), +preserving their order. Multiple headers with the same name may be processed +in this manner, except \fBSet-Cookie\fR which does not conform to the +comma-separated-list syntax and cannot be combined into a single value. +Each \fBSet-Cookie\fR header must be treated individually, e.g. by processing +the return value of \fB::http::responseHeaders\fR \fItoken\fR \fBSet-Cookie\fR. +.TP +\fB::http::responseInfo\fR \fItoken\fR +. +This command returns a \fBdict\fR of selected response metadata that are +essential for identifying a successful transaction and making use of the +response, along with other metadata that are informational. The keys of +the \fBdict\fR are \fIstage\fR, \fIstatus\fR, \fIresponseCode\fR, +\fIreasonPhrase\fR, \fIcontentType\fR, \fIbinary\fR, \fIredirection\fR, +\fIupgrade\fR, \fIerror\fR, \fIpostError\fR, \fImethod\fR, \fIcharset\fR, +\fIcompression\fR, \fIhttpRequest\fR, \fIhttpResponse\fR, \fIurl\fR, +\fIconnectionRequest\fR, \fIconnectionResponse\fR, \fIconnectionActual\fR, +\fItransferEncoding\fR, \fItotalPost\fR, \fIcurrentPost\fR, \fItotalSize\fR, +and \fIcurrentSize\fR. The meaning of these keys is described in the +section \fBMETADATA\fR below. +.RS +.PP +It is always worth checking the value of \fIbinary\fR after a HTTP transaction, +to determine whether a misconfigured server has caused http to interpret a +text resource as a binary, or vice versa. +.PP +After a POST transaction, check the value of \fIpostError\fR to verify that +the request body was uploaded without error. +.RE +.TP +\fB::http::responseBody\fR \fItoken\fR +. +This command returns the entity sent by the HTTP server (unless +\fI-channel\fR was used, in which case the entity was delivered to the +channel, and the command returns the empty string). +.RS +.PP +Other terms for +"entity", with varying precision, include "representation of resource", +"resource", "response body after decoding", "payload", +"message body after decoding", "content(s)", and "file". +.RE +.TP \fB::http::register\fR \fIproto port command\fR . This procedure allows one to provide custom HTTP transport types @@ -558,18 +799,34 @@ registered via \fB::http::register\fR, returning a two-item list of the default port and handler command that was previously installed (via \fB::http::register\fR) if there was such a handler, and an error if there was no such handler. +.TP +\fB::http::code\fR \fItoken\fR +. +An alternative name for the command \fB::http::responseLine\fR +.TP +\fB::http::data\fR \fItoken\fR +. +An alternative name for the command \fB::http::responseBody\fR. +.TP +\fB::http::meta\fR \fItoken\fR ?\fIheaderName\fR? +. +An alternative name for the command \fB::http::responseHeaders\fR +.TP +\fB::http::ncode\fR \fItoken\fR +. +An alternative name for the command \fB::http::responseCode\fR .SH ERRORS The \fB::http::geturl\fR procedure will raise errors in the following cases: invalid command line options, -an invalid URL, -a URL on a non-existent host, -or a URL at a bad port on an existing host. +or an invalid URL. These errors mean that it cannot even start the network transaction. -It will also raise an error if it gets an I/O error while -writing out the HTTP request header. For synchronous \fB::http::geturl\fR calls (where \fB\-command\fR is -not specified), it will raise an error if it gets an I/O error while +not specified), it will raise an error if +the URL is on a non-existent host +or at a bad port on an existing host. +It will also raise an error for any I/O errors while +writing out the HTTP request line and headers, or reading the HTTP reply headers or data. Because \fB::http::geturl\fR does not return a token in these cases, it does all the required cleanup and there is no issue of your app having to call @@ -581,13 +838,12 @@ HTTP reply headers or data, no exception is thrown. This is because after writing the HTTP headers, \fB::http::geturl\fR returns, and the rest of the HTTP transaction occurs in the background. The command callback can check if any error occurred during the read by calling -\fB::http::status\fR to check the status and if its \fIerror\fR, -calling \fB::http::error\fR to get the error message. +\fB::http::responseInfo\fR to check the transaction status. .PP Alternatively, if the main program flow reaches a point where it needs to know the result of the asynchronous HTTP request, it can call \fB::http::wait\fR and then check status and error, just as the -callback does. +synchronous call does. .PP The \fB::http::geturl\fR command runs the \fB\-command\fR, \fB\-handler\fR, and \fB\-proxyfilter\fR callbacks inside a \fBcatch\fR command. Therefore @@ -601,15 +857,17 @@ In any case, you must still call \fB::http::cleanup\fR to delete the state array when you are done. .PP There are other possible results of the HTTP transaction -determined by examining the status from \fB::http::status\fR. +determined by examining the status from \fB::http::status\fR (or the value +of the \fIstatus\fR key in the dictionary returned +by \fB::http::responseInfo\fR). These are described below. .TP \fBok\fR . If the HTTP transaction completes entirely, then status will be \fBok\fR. -However, you should still check the \fB::http::code\fR value to get -the HTTP status. The \fB::http::ncode\fR procedure provides just -the numeric error (e.g., 200, 404 or 500) while the \fB::http::code\fR +However, you should still check the \fB::http::responseLine\fR value to get +the HTTP status. The \fB::http::responseCode\fR procedure provides just +the numeric error (e.g., 200, 404 or 500) while the \fB::http::responseLine\fR procedure returns a value like .QW "HTTP 404 File not found" . .TP @@ -620,147 +878,447 @@ is raised, but the status of the transaction will be \fBeof\fR. .TP \fBerror\fR . -The error message will also be stored in the \fBerror\fR status -array element, accessible via \fB::http::error\fR. +The error message, stack trace, and error code are accessible +via \fB::http::error\fR. The error message is also provided by the value of +the \fIerror\fR key in the dictionary returned by \fB::http::responseInfo\fR. .TP \fBtimeout\fR . -A timeout occurred before the transaction could complete +A timeout occurred before the transaction could complete. .TP \fBreset\fR . -user-reset -.PP -Another error possibility is that \fB::http::geturl\fR is unable to -write all the post query data to the server before the server -responds and closes the socket. -The error message is saved in the \fBposterror\fR status array -element and then \fB::http::geturl\fR attempts to complete the -transaction. -If it can read the server's response -it will end up with an \fBok\fR status, otherwise it will have -an \fBeof\fR status. +The user has called \fB::http::reset\fR. +.TP +\fB""\fR +. +(empty string) The transaction has not yet finished. +.PP +Another error possibility is that \fB::http::geturl\fR failed to +write the whole of the POST request body (\fB-query\fR or \fB-querychannel\fR +data) to the server. \fB::http::geturl\fR stores the error message for later +retrieval by the \fB::http::postError\fR or \fB::http::responseInfo\fR +commands, and then attempts to complete the transaction. +If it can read the server's response the status will be \fBok\fR, but it is +important to call \fB::http::postError\fR or \fB::http::responseInfo\fR after +every POST to check that the data was sent in full. +If the server has closed the connection the status will be \fBeof\fR. +.SH "METADATA" +.PP +.SS "MOST USEFUL METADATA" +When a HTTP server responds to a request, it supplies not only the entity +requested, but also metadata. This is provided by the first line (the +"status line") of the response, and by a number of HTTP headers. Further +metadata relates to how \fB::http::geturl\fR has processed the response +from the server. +.PP +The most important metadata can be accessed with the command +\fB::http::responseInfo\fR. +This command returns a \fBdict\fR of metadata that are essential for +identifying a successful transaction and making use of the response, +along with other metadata that are informational. The keys of +the \fBdict\fR are: +.PP +.RS +.RS +\fB===== Essential Values =====\fR +.RE +.RE +.TP +\fBstage\fR +. +This value, set by \fB::http::geturl\fR, describes the stage that the +transaction has reached. Values, in order of the transaction lifecycle, +are: "created", "connecting", "header", "body", and "complete". The +other \fBdict\fR keys will not be available until the value of \fBstage\fR +is "body" or "complete". The key \fBcurrentSize\fR has its final value only +when \fBstage\fR is "complete". +.TP +\fBstatus\fR +. +This value, set by \fB::http::geturl\fR, is "ok" for a successful transaction; +"eof", "error", "timeout", or "reset" for an unsuccessful transaction; or "" +if the transaction is still in progress. The value is the same as that +returned by command \fB::http::status\fR. The meaning of these values is +described in the section \fBERRORS\fR (above). +.TP +\fBresponseCode\fR +. +The "HTTP status code" sent by the server in the first line (the "status line") +of the response. If the value cannot be extracted from the status line, the +full status line is returned. +.TP +\fBreasonPhrase\fR +. +The "reason phrase" sent by the server as a description of the HTTP status code. +If the value cannot be extracted from the status line, the full status +line is returned. +.TP +\fBcontentType\fR +. +The value of the \fBContent-Type\fR response header or, if the header was not +supplied, the default value "application/octet-stream". +.TP +\fBbinary\fR +. +This boolean value, set by \fB::http::geturl\fR, describes how the command +has interpreted the entity returned by the server (after decoding any +compression specified by the \fBContent-Encoding\fR response header). +This decoded entity is accessible as the return value of the +command \fB::http::responseBody\fR. +.PP +.RS +The value is \fBtrue\fR if http has interpreted the decoded entity as binary. +The value returned by \fB::http::responseBody\fR is a Tcl binary string. +This is a suitable format for image data, zip files, etc. +\fB::http::geturl\fR chooses this value if the user has requested a binary +interpretation by passing the option \fI\-binary\fR to the command, or if the +server has supplied a binary content type in a \fBContent-Type\fR response +header, or if the server has not supplied any \fBContent-Type\fR header. +.PP +The value is \fBfalse\fR in other cases, and this means that http has +interpreted the decoded entity as text. The text has been converted, from the +character set notified by the server, into Tcl's internal Unicode format; +the value returned by \fB::http::responseBody\fR is an ordinary Tcl string. +.PP +It is always worth checking the value of "binary" after a HTTP transaction, +to determine whether a misconfigured server has caused http to interpret a +text resource as a binary, or vice versa. +.RE +.TP +\fBredirection\fR +. +The URL that is the redirection target. The value is that of the \fBLocation\fR +response header. This header is sent when a response has status code +3XX (redirection). +.TP +\fBupgrade\fR +. +If not empty, the value indicates the protocol(s) to which the server will +switch after completion of this transaction, while continuing to use the +same connection. When the server intends to switch protocols, it will also +send the value "101" as the status code (the \fBresponseCode\fR key), and the +word "upgrade" as an element of the \fBConnection\fR response header (the +\fBconnectionResponse\fR key), and it will not send a response body. +See the section \fBPROTOCOL UPGRADES\fR for more information. +.TP +\fBerror\fR +. +The error message, if there is one. Further information, including a stack +trace and error code, are available from command \fB::http::error\fR. +.TP +\fBpostError\fR +. +The error message (if any) generated when a HTTP POST request sends its +request-body to the server. Further information, including a stack trace +and error code, are available from command \fB::http::postError\fR. A POST +transaction may appear complete, according to the +keys \fBstage\fR, \fBstatus\fR, and \fBresponseCode\fR, but it is important +to check this \fBpostError\fR key in case an error occurred when uploading +the request-body. +.PP +.RS +.RS +\fB===== Informational Values =====\fR +.RE +.RE +.TP +\fBmethod\fR +. +The HTTP method used in the request. +.TP +\fBcharset\fR +. +The value of the charset attribute of the \fBContent-Type\fR response header. +The charset value is used only for a text resource. If the server did not +specify a charset, the value defaults to that of the +variable \fB::http::defaultCharset\fR, which unless it has been deliberately +modified by the caller is \fBiso8859-1\fR. Incoming text data is automatically +converted from the character set defined by \fBcharset\fR to Tcl's internal +Unicode representation, i.e. to a Tcl string. +.TP +\fBcompression\fR +. +A copy of the \fBContent-Encoding\fR response-header value. +.TP +\fBhttpRequest\fR +. +The version of HTTP specified in the request (i.e. sent in the request line). +The value is that of the option \fB\-protocol\fR supplied +to \fB::http::geturl\fR (default value "1.1"), unless the command reduced the +value to "1.0" because it was passed the \fB\-handler\fR option. +.TP +\fBhttpResponse\fR +. +The version of HTTP used by the server (obtained from the response +"status line"). The server uses this version of HTTP in its response, but +ensures that this response is compatible with the HTTP version specified in the +client's request. If the value cannot be extracted from the status line, the +full status line is returned. +.TP +\fBurl\fR +. +The requested URL, typically the URL supplied as an argument +to \fB::http::geturl\fR but without its "fragment" (the final part of the URL +beginning with "#"). +.TP +\fBconnectionRequest\fR +. +The value, if any, sent to the server in \fBConnection\fR request header(s). +.TP +\fBconnectionResponse\fR +. +The value, if any, received from the server in \fBConnection\fR response +header(s). +.TP +\fBconnectionActual\fR +. +This value, set by \fB::http::geturl\fR, reports whether the connection was +closed after the transaction (value "close"), or left open (value "keep-alive"). +.TP +\fBtransferEncoding\fR +. +The value of the Transfer-Encoding response header, if it is present. +The value is either "chunked" (indicating HTTP/1.1 "chunked encoding") or +the empty string. +.TP +\fBtotalPost\fR +. +The total length of the request body in a POST request. +.TP +\fBcurrentPost\fR +. +The number of bytes of the POST request body sent to the server so far. +The value is the same as that returned by command \fB::http::size\fR. +.TP +\fBtotalSize\fR +. +A copy of the \fBContent-Length\fR response-header value. +The number of bytes specified in a \fBContent-Length\fR header, if one +was sent. If none was sent, the value is 0. A correctly configured server +omits this header if the transfer-encoding is "chunked", or (for older +servers) if the server closes the connection when it reaches the end of +the resource. +.TP +\fBcurrentSize\fR +. +The number of bytes fetched from the server so far. +.PP +.SS "MORE METADATA" +The dictionary returned by \fB::http::responseInfo\fR is the most useful +subset of the available metadata. Other metadata include: +.PP +1. The full "status line" of the response, available as the return value +of command \fB::http::responseLine\fR. +.PP +2. The full response headers, available as the return value of +command \fB::http::responseHeaders\fR. This return value is a list of the +response-header names and values, in the order that they were received from +the server. +.PP +The return value is not a \fBdict\fR because some header names may +occur more than once, notably \fBSet-Cookie\fR. If the value is read +into a \fBdict\fR or into an array (using array set), only the last header +with each name will be preserved. +.PP +.RS +Some of the header names (metadata keys) are listed below, but the HTTP +standard defines several more, and servers are free to add their own. +When a dictionary key is mentioned below, this refers to the \fBdict\fR +value returned by command \fB::http::responseInfo\fR. +.TP +\fBContent-Type\fR +. +The content type of the URL contents. Examples include \fBtext/html\fR, +\fBimage/gif,\fR \fBapplication/postscript\fR and +\fBapplication/x-tcl\fR. Text values typically specify a character set, e.g. +\fBtext/html; charset=UTF-8\fR. Dictionary key \fIcontentType\fR. +.TP +\fBContent-Length\fR +. +The advertised size in bytes of the contents, available as dictionary +key \fItotalSize\fR. The actual number of bytes read by \fB::http::geturl\fR +so far is available as dictionary key \fBcurrentSize\fR. +.TP +\fBContent-Encoding\fR +. +The compression algorithm used for the contents. +Examples include \fBgzip\fR, \fBdeflate\fR. +Dictionary key \fIcontent\fR. +.TP +\fBLocation\fR +. +This header is sent when a response has status code 3XX (redirection). +It provides the URL that is the redirection target. +Dictionary key \fIredirection\fR. +.TP +\fBSet-Cookie\fR +. +This header is sent to offer a cookie to the client. Cookie management is +done by the \fB::http::config\fR option \fI\-cookiejar\fR, and so +the \fBSet-Cookie\fR headers need not be parsed by user scripts. +See section \fBCOOKIE JAR PROTOCOL\fR. +.TP +\fBConnection\fR +. +The value can be supplied as a comma-separated list, or by multiple headers. +The list often has only one element, either "close" or "keep-alive". +The value "upgrade" indicates a successful upgrade request and is typically +combined with the status code 101, an \fBUpgrade\fR response header, and no +response body. Dictionary key \fIconnectionResponse\fR. +.TP +\fBUpgrade\fR +. +The value indicates the protocol(s) to which the server will switch +immediately after the empty line that terminates the 101 response headers. +Dictionary key \fIupgrade\fR. +.RE +.PP +.SS "EVEN MORE METADATA" +.PP +1. Details of the HTTP request. The request is determined by the options +supplied to \fB::http::geturl\fR and \fB::http::config\fR. However, it is +sometimes helpful to examine what \fB::http::geturl\fR actually sent to the +server, and this information is available through +commands \fB::http::requestHeaders\fR and \fB::http::requestLine\fR. +.PP +2. The state array: the internal variables of \fB::http::geturl\fR. +It may sometimes be helpful to examine this array. +Details are given in the next section. .SH "STATE ARRAY" -The \fB::http::geturl\fR procedure returns a \fItoken\fR that can be used to -get to the state of the HTTP transaction in the form of a Tcl array. -Use this construct to create an easy-to-use array variable: +The \fB::http::geturl\fR procedure returns a \fItoken\fR that can be used +as an argument to other \fB::http::*\fR commands, which examine and manage +the state of the HTTP transaction. For most purposes these commands are +sufficient. The \fItoken\fR can also be used to access +the internal state of the transaction, which is stored in a Tcl array. +This facility is most useful when writing callback commands for the +options \fB\-command\fR, \fB\-handler\fR, \fB\-progress\fR, +or \fB\-queryprogress\fR. +Use the following command inside the proc to define an easy-to-use +array \fIstate\fR as a local variable within the proc .PP .CS -upvar #0 $token state +upvar 0 $token state .CE .PP Once the data associated with the URL is no longer needed, the state array should be unset to free up storage. The \fB::http::cleanup\fR procedure is provided for that purpose. -The following elements of -the array are supported: +.PP +The following elements of the array are supported, and are the origin of the +values returned by commands as described below. When a dictionary key is +mentioned below, this refers to the \fBdict\fR value returned by +command \fB::http::responseInfo\fR. .RS .TP \fBbinary\fR . -This is boolean \fBtrue\fR if (after decoding any compression specified -by the -.QW "Content-Encoding" -response header) the HTTP response is binary. It is boolean \fBfalse\fR -if the HTTP response is text. +For dictionary key \fIbinary\fR. .TP \fBbody\fR . -The contents of the URL. This will be empty if the \fB\-channel\fR -option has been specified. This value is returned by the \fB::http::data\fR -command. +For command \fB::http::responseBody\fR. .TP \fBcharset\fR . -The value of the charset attribute from the \fBContent-Type\fR meta-data -value. If none was specified, this defaults to the RFC standard -\fBiso8859-1\fR, or the value of \fB$::http::defaultCharset\fR. Incoming -text data will be automatically converted from this charset to utf-8. +For dictionary key \fIcharset\fR. .TP \fBcoding\fR . -A copy of the \fBContent-Encoding\fR meta-data value. +For dictionary key \fIcompression\fR. +.TP +\fBconnection\fR +. +For dictionary key \fIconnectionActual\fR. .TP \fBcurrentsize\fR . -The current number of bytes fetched from the URL. -This value is returned by the \fB::http::size\fR command. +For command \fB::http::size\fR; and for dictionary key \fIcurrentSize\fR. .TP \fBerror\fR . -If defined, this is the error string seen when the HTTP transaction -was aborted. +For command \fB::http::error\fR; part is used in dictionary key \fIerror\fR. .TP \fBhttp\fR . -The HTTP status reply from the server. This value -is returned by the \fB::http::code\fR command. The format of this value is: -.RS -.PP -.CS -\fIHTTP/1.1 code string\fR -.CE -.PP -The \fIcode\fR is a three-digit number defined in the HTTP standard. -A code of 200 is OK. Codes beginning with 4 or 5 indicate errors. -Codes beginning with 3 are redirection errors. In this case the -\fBLocation\fR meta-data specifies a new URL that contains the -requested information. -.RE +For command \fB::http::responseLine\fR. +.TP +\fBhttpResponse\fR +. +For dictionary key \fIhttpResponse\fR. .TP \fBmeta\fR . -The HTTP protocol returns meta-data that describes the URL contents. -The \fBmeta\fR element of the state array is a list of the keys and -values of the meta-data. This is in a format useful for initializing -an array that just contains the meta-data: -.RS -.PP -.CS -array set meta $state(meta) -.CE -.PP -Some of the meta-data keys are listed below, but the HTTP standard defines -more, and servers are free to add their own. +For command \fB::http::responseHeaders\fR. Further discussion above in the +section \fBMORE METADATA\fR. .TP -\fBContent-Type\fR +\fBmethod\fR . -The type of the URL contents. Examples include \fBtext/html\fR, -\fBimage/gif,\fR \fBapplication/postscript\fR and -\fBapplication/x-tcl\fR. +For dictionary key \fImethod\fR. .TP -\fBContent-Length\fR +\fBposterror\fR . -The advertised size of the contents. The actual size obtained by -\fB::http::geturl\fR is available as \fBstate(currentsize)\fR. +For dictionary key \fIpostError\fR. .TP -\fBLocation\fR +\fBpostErrorFull\fR . -An alternate URL that contains the requested data. -.RE +For command \fB::http::postError\fR. .TP -\fBposterror\fR +\fB\-protocol\fR +. +For dictionary key \fIhttpRequest\fR. +.TP +\fBquerylength\fR +. +For dictionary key \fItotalPost\fR. +.TP +\fBqueryoffset\fR . -The error, if any, that occurred while writing -the post query data to the server. +For dictionary key \fIcurrentPost\fR. +.TP +\fBreasonPhrase\fR +. +For dictionary key \fIreasonPhrase\fR. +.TP +\fBrequestHeaders\fR +. +For command \fB::http::requestHeaders\fR. +.TP +\fBrequestLine\fR +. +For command \fB::http::requestLine\fR. +.TP +\fBresponseCode\fR +. +For dictionary key \fIresponseCode\fR. +.TP +\fBstate\fR +. +For dictionary key \fIstage\fR. .TP \fBstatus\fR . -See description in the chapter \fBERRORS\fR above for a -list and description of \fBstatus\fR. -During the transaction this value is the empty string. +For command \fB::http::status\fR; and for dictionary key \fIstatus\fR. .TP \fBtotalsize\fR . -A copy of the \fBContent-Length\fR meta-data value. +For dictionary key \fItotalSize\fR. +.TP +\fBtransfer\fR +. +For dictionary key \fItransferEncoding\fR. .TP \fBtype\fR . -A copy of the \fBContent-Type\fR meta-data value. +For dictionary key \fIcontentType\fR. +.TP +\fBupgrade\fR +. +For dictionary key \fIupgrade\fR. .TP \fBurl\fR . -The requested URL. +For dictionary key \fIurl\fR. .RE .SH "PERSISTENT CONNECTIONS" .PP @@ -859,7 +1417,7 @@ that fails because it uses a persistent connection that the server has half-closed (an .QW "asynchronous close event" ). Subsequent GET and HEAD requests in a failed pipeline will also be retried. -\fIThe \-repost option should be used only if the application understands +\fIThe \fB\-repost\fI option should be used only if the application understands that the retry is appropriate\fR - specifically, the application must know that if the failed POST successfully modified the state of the server, a repeat POST would have no adverse effect. @@ -967,22 +1525,25 @@ Other keys may always be ignored; they have no meaning in this protocol. .VE TIP406 .SH "PROTOCOL UPGRADES" .PP -The HTTP/1.1 \fBConnection\fR and \fBUpgrade\fR client headers inform the server -that the client wishes to change the protocol used over the existing connection -(RFC 7230). This mechanism can be used to request a WebSocket (RFC 6455), a +The HTTP/1.1 \fBConnection\fR and \fBUpgrade\fR request headers inform the +server that the client wishes to change the protocol used over the existing +connection (RFC 7230). +This mechanism can be used to request a WebSocket (RFC 6455), a higher version of the HTTP protocol (HTTP 2), or TLS encryption. If the server accepts the upgrade request, its response code will be 101. .PP -To request a protocol upgrade when calling \fBhttp::geturl\fR, the \fB\-headers\fR -option must supply appropriate values for \fBConnection\fR and \fBUpgrade\fR, and +To request a protocol upgrade when calling \fBhttp::geturl\fR, +the \fB\-headers\fR option must supply appropriate values for \fBConnection\fR +and \fBUpgrade\fR, and the \fB\-command\fR option must supply a command that implements the requested protocol and can also handle the server response if the server refuses the protocol upgrade. For upgrade requests \fBhttp::geturl\fR ignores the value of option \fB\-keepalive\fR, and always uses the value \fB0\fR so that the upgrade -request is not made over a connection that is intended for multiple HTTP requests. +request is not made over a connection that is intended for multiple HTTP +requests. .PP -The Tcllib library \fBwebsocket\fR implements WebSockets, and makes the necessary -calls to commands in the \fBhttp\fR package. +The Tcllib library \fBwebsocket\fR implements WebSockets, and makes the +necessary calls to commands in the \fBhttp\fR package. .PP There is currently no native Tcl client library for HTTP/2. .PP @@ -993,30 +1554,59 @@ protocols such as Internet Printing Protocol (IPP) that are built on top of traffic. .PP In browsers, opportunistic encryption is instead implemented by the -\fBUpgrade-Insecure-Requests\fR client header. If a secure service is available, -the server response code is a 307 redirect, and the response header -\fBLocation\fR specifies the target URL. The browser must call \fBhttp::geturl\fR -again in order to fetch this URL. +\fBUpgrade-Insecure-Requests\fR client header. If a secure service is +available, the server response code is a 307 redirect, and the response header +\fBLocation\fR specifies the target URL. The browser must +call \fBhttp::geturl\fR again in order to fetch this URL. See https://w3c.github.io/webappsec-upgrade-insecure-requests/ .PP .SH THREADS .PP .SS "PURPOSE" .PP -Command \fB::http::geturl\fR uses the Tcl \fB::socket\fR command with the \-async option to connect to a remote server, but the return from this command can be delayed in adverse cases (e.g. a slow DNS lookup), preventing the event loop from processing other events. This delay is avoided if the \fB::socket\fR command is evaluated in another thread. The Thread package is not part of Tcl but is provided in "Batteries Included" distributions. Instead of the \fB::socket\fR command, the http package uses \fB::http::socket\fR which makes connections in the manner specified by the value of \-threadlevel and the availability of package Thread. +Command \fB::http::geturl\fR uses the Tcl \fB::socket\fR command with +the \fI\-async\fR option to connect to a remote server, but the return from +this command can be delayed in adverse cases (e.g. a slow DNS lookup), +preventing the event loop from processing other events. +This delay is avoided if the \fB::socket\fR command is evaluated in another +thread. The Thread package is not part of Tcl but is provided in +"Batteries Included" distributions. Instead of the \fB::socket\fR command, +the http package uses \fB::http::socket\fR which makes connections in the +manner specified by the value of \fI\-threadlevel\fR and the availability +of package Thread. .PP .SS "WITH TLS (HTTPS)" .PP -The same \-threadlevel configuration applies to both HTTP and HTTPS connections. HTTPS is enabled by using the \fBhttp::register\fR command, typically by specifying the \fB::tls::socket\fR command of the tls package to handle TLS cryptography. The \fB::tls::socket\fR command connects to the remote server by using the command specified by the value of variable \fB::tls::socketCmd\fR, and this value defaults to "::socket". If http::geturl finds that \fB::tls::socketCmd\fR has this value, it replaces it with the value "::http::socket". If \fB::tls::socketCmd\fR has a value other than "::socket", i.e. if the script or the Tcl installation has replaced the value "::socket" with the name of a different command, then http does not change the value. The script or installation that modified \fB::tls::socketCmd\fR is responsible for integrating \fB::http::socket\fR into its own replacement command. +The same \fI\-threadlevel\fR configuration applies to both HTTP and HTTPS +connections. +HTTPS is enabled by using the \fBhttp::register\fR command, typically by +specifying the \fB::tls::socket\fR command of the tls package to handle TLS +cryptography. The \fB::tls::socket\fR command connects to the remote server by +using the command specified by the value of variable \fB::tls::socketCmd\fR, and +this value defaults to "::socket". If http::geturl finds +that \fB::tls::socketCmd\fR has this value, it replaces it with the value +"::http::socket". If \fB::tls::socketCmd\fR has a value other than "::socket", +i.e. if the script or the Tcl installation has replaced the value "::socket" +with the name of a different command, then http does not change the value. +The script or installation that modified \fB::tls::socketCmd\fR is responsible +for integrating \fR::http::socket\fR into its own replacement command. .PP .SS "WITH A CHILD INTERPRETER" .PP -The peer thread can transfer the socket only to the main interpreter of the script's thread. Therefore the thread-based \fB::http::socket\fR works with non-zero \-threadlevel values only if the script runs in the main interpreter. A child interpreter must use \-threadlevel 0 unless the parent interpreter has provided alternative facilities. The main parent interpreter may grant full \-threadlevel facilities to a child interpreter, for example by aliasing, to \fB::http::socket\fR in the child, a command that runs \fBhttp::socket\fR in the parent, and then transfers the socket to the child. +The peer thread can transfer the socket only to the main interpreter of the +script's thread. Therefore the thread-based \fB::http::socket\fR works with +non-zero \fI\-threadlevel\fR values only if the script runs in the main +interpreter. A child interpreter must use \fI\-threadlevel 0\fR unless the +parent interpreter has provided alternative facilities. The main parent +interpreter may grant full \fI\-threadlevel\fR facilities to a child +interpreter, for example by aliasing, to \fB::http::socket\fR in the child, +a command that runs \fBhttp::socket\fR in the parent, and then transfers +the socket to the child. .PP .SH EXAMPLE .PP This example creates a procedure to copy a URL to a file while printing a -progress meter, and prints the meta-data associated with the URL. +progress meter, and prints the response headers associated with the URL. .PP .CS proc httpcopy { url file {chunk 4096} } { @@ -1028,7 +1618,7 @@ proc httpcopy { url file {chunk 4096} } { # This ends the line started by httpCopyProgress puts stderr "" - upvar #0 $token state + upvar 0 $token state set max 0 foreach {name value} $state(meta) { if {[string length $name] > $max} { |