diff options
Diffstat (limited to 'doc/http.n')
| -rw-r--r-- | doc/http.n | 299 |
1 files changed, 289 insertions, 10 deletions
@@ -6,22 +6,24 @@ '\" See the file "license.terms" for information on usage and redistribution '\" of this file, and for a DISCLAIMER OF ALL WARRANTIES. '\" -.TH "http" n 2.7 http "Tcl Bundled Packages" +.TH "http" n 2.9 http "Tcl Bundled Packages" .so man.macros .BS '\" Note: do not modify the .SH NAME line immediately below! .SH NAME http \- Client-side implementation of the HTTP/1.1 protocol .SH SYNOPSIS -\fBpackage require http ?2.7?\fR +\fBpackage require http\fI ?\fB2.8\fR? .\" See Also -useragent option documentation in body! .sp -\fB::http::config ?\fI\-option value\fR ...? +\fB::http::config\fR ?\fI\-option value\fR ...? .sp \fB::http::geturl \fIurl\fR ?\fI\-option value\fR ...? .sp \fB::http::formatQuery\fR \fIkey value\fR ?\fIkey value\fR ...? .sp +\fB::http::quoteString\fR \fIvalue\fR +.sp \fB::http::reset\fR \fItoken\fR ?\fIwhy\fR? .sp \fB::http::wait \fItoken\fR @@ -44,12 +46,14 @@ http \- Client-side implementation of the HTTP/1.1 protocol .sp \fB::http::register \fIproto port command\fR .sp +\fB::http::registerError \fIport\fR ?\fImessage\fR? +.sp \fB::http::unregister \fIproto\fR .BE .SH DESCRIPTION .PP The \fBhttp\fR package provides the client side of the HTTP/1.1 -protocol, as defined in RFC 2616. +protocol, as defined in RFC 7230 to RFC 7235, which supersede RFC 2616. The package implements the GET, POST, and HEAD operations of HTTP/1.1. It allows configuration of a proxy host to get through firewalls. The package is compatible with the \fBSafesock\fR security @@ -95,6 +99,28 @@ comma-separated list of mime type patterns that you are willing to receive. For example, .QW "image/gif, image/jpeg, text/*" . .TP +\fB\-cookiejar\fR \fIcommand\fR +.VS TIP406 +The cookie store for the package to use to manage HTTP cookies. +\fIcommand\fR is a command prefix list; if the empty list (the +default value) is used, no cookies will be sent by requests or stored +from responses. The command indicated by \fIcommand\fR, if supplied, +must obey the \fBCOOKIE JAR PROTOCOL\fR described below. +.VE TIP406 +.TP +\fB\-pipeline\fR \fIboolean\fR +. +Specifies whether HTTP/1.1 transactions on a persistent socket will be +pipelined. See the \fBPERSISTENT SOCKETS\fR section for details. The default +is 1. +.TP +\fB\-postfresh\fR \fIboolean\fR +. +Specifies whether requests that use the \fBPOST\fR method will always use a +fresh socket, overriding the \fB-keepalive\fR option of +command \fBhttp::geturl\fR. See the \fBPERSISTENT SOCKETS\fR section for details. +The default is 0. +.TP \fB\-proxyhost\fR \fIhostname\fR . The name of the proxy host, if any. If this value is the @@ -116,20 +142,47 @@ an empty list. The default filter returns the values of the \fB\-proxyhost\fR and \fB\-proxyport\fR settings if they are non-empty. .TP +\fB\-repost\fR \fIboolean\fR +. +Specifies what to do if a POST request over a persistent connection fails +because the server has half-closed the connection. If boolean \fBtrue\fR, the +request +will be automatically retried; if boolean \fBfalse\fR it will not, and the +application +that uses \fBhttp::geturl\fR is expected to seek user confirmation before +retrying the POST. The value \fBtrue\fR should be used only under certain +conditions. See the \fBPERSISTENT SOCKETS\fR section for details. The +default is 0. +.TP \fB\-urlencoding\fR \fIencoding\fR . The \fIencoding\fR used for creating the x-url-encoded URLs with -\fB::http::formatQuery\fR. The default is \fButf-8\fR, as specified by RFC +\fB::http::formatQuery\fR and \fB::http::quoteString\fR. +The default is \fButf-8\fR, as specified by RFC 2718. Prior to http 2.5 this was unspecified, and that behavior can be returned by specifying the empty string (\fB{}\fR), although \fIiso8859-1\fR is recommended to restore similar behavior but without the -\fB::http::formatQuery\fR throwing an error processing non-latin-1 -characters. +\fB::http::formatQuery\fR or \fB::http::quoteString\fR +throwing an error processing non-latin-1 characters. .TP \fB\-useragent\fR \fIstring\fR . -The value of the User-Agent header in the HTTP request. The default is -.QW "\fBTcl http client package 2.7\fR" . +The value of the User-Agent header in the HTTP request. In an unsafe +interpreter, the default value depends upon the operating system, and +the version numbers of \fBhttp\fR and \fBTcl\fR, and is (for example) +.QW "\fBMozilla/5.0 (Windows; U; Windows NT 10.0) http/2.8.12 Tcl/8.6.8\fR" . +A safe interpreter cannot determine its operating system, and so the default +in a safe interpreter is to use a Windows 10 value with the current version +numbers of \fBhttp\fR and \fBTcl\fR. +.TP +\fB\-zip\fR \fIboolean\fR +. +If the value is boolean \fBtrue\fR, then by default requests will send a header +.QW "\fBAccept-Encoding: gzip,deflate,compress\fR" . +If the value is boolean \fBfalse\fR, then by default this header will not be sent. +In either case the default can be overridden for an individual request by +supplying a custom \fBAccept-Encoding\fR header in the \fB-headers\fR option +of \fBhttp::geturl\fR. The default is 1. .RE .TP \fB::http::geturl\fR \fIurl\fR ?\fIoptions\fR? @@ -227,7 +280,7 @@ Pragma: no-cache .TP \fB\-keepalive\fR \fIboolean\fR . -If true, attempt to keep the connection open for servicing +If boolean \fBtrue\fR, attempt to keep the connection open for servicing multiple requests. Default is 0. .TP \fB\-method\fR \fItype\fR @@ -334,6 +387,11 @@ encodes the keys and values, and generates one string that has the proper & and = separators. The result is suitable for the \fB\-query\fR value passed to \fB::http::geturl\fR. .TP +\fB::http::quoteString\fR \fIvalue\fR +. +This procedure does x-url-encoding of string. It takes a single argument and +encodes it. +.TP \fB::http::reset\fR \fItoken\fR ?\fIwhy\fR? . This command resets the HTTP transaction identified by \fItoken\fR, if any. @@ -415,6 +473,17 @@ set token [::http::geturl https://my.secure.site/] .CE .RE .TP +\fB::http::registerError\fR \fIport\fR ?\fImessage\fR? +. +This procedure allows a registered protocol handler to deliver an error +message for use by \fBhttp\fR. Calling this command does not raise an +error. The command is useful when a registered protocol detects an problem +(for example, an invalid TLS certificate) that will cause an error to +propagate to \fBhttp\fR. The command allows \fBhttp\fR to provide a +precise error message rather than a general one. The command returns the +value provided by the last call with argument \fImessage\fR, or the empty +string if no such call has been made. +.TP \fB::http::unregister\fR \fIproto\fR . This procedure unregisters a protocol handler that was previously @@ -504,6 +573,14 @@ The following elements of the array are supported: .RS .TP +\fBbinary\fR +. +This is boolean \fBtrue\fR if (after decoding any compression specified +by the +.QW "Content-Encoding" +response header) the HTTP response is binary. It is boolean \fBfalse\fR +if the HTTP response is text. +.TP \fBbody\fR . The contents of the URL. This will be empty if the \fB\-channel\fR @@ -602,6 +679,208 @@ A copy of the \fBContent-Type\fR meta-data value. . The requested URL. .RE +.SH "PERSISTENT CONNECTIONS" +.PP +.SS "BASICS" +.PP +See RFC 7230 Sec 6, which supersedes RFC 2616 Sec 8.1. +.PP +A persistent connection allows multiple HTTP/1.1 transactions to be +carried over the same TCP connection. Pipelining allows a +client to make multiple requests over a persistent connection without +waiting for each response. The server sends responses in the same order +that the requests were received. +.PP +If a POST request fails to complete, typically user confirmation is +needed before sending the request again. The user may wish to verify +whether the server was modified by the failed POST request, before +sending the same request again. +.PP +A HTTP request will use a persistent socket if the call to +\fBhttp::geturl\fR has the option \fB-keepalive true\fR. It will use +pipelining where permitted if the \fBhttp::config\fR option +\fB-pipeline\fR is boolean \fBtrue\fR (its default value). +.PP +The http package maintains no more than one persistent connection to each +server (i.e. each value of +.QW "domain:port" ). +If \fBhttp::geturl\fR is called to make a request over a persistent +connection while the connection is busy with another request, the new +request will be held in a queue until the connection is free. +.PP +The http package does not support HTTP/1.0 persistent connections +controlled by the \fBKeep-Alive\fR header. +.SS "SPECIAL CASES" +.PP +This subsection discusses issues related to closure of the +persistent connection by the server, automatic retry of failed requests, +the special treatment necessary for POST requests, and the options for +dealing with these cases. +.PP +In accordance with RFC 7230, \fBhttp::geturl\fR does not pipeline +requests that use the POST method. If a POST uses a persistent +connection and is not the first request on that connection, +\fBhttp::geturl\fR waits until it has received the response for the previous +request; or (if \fBhttp::config\fR option \fB-postfresh\fR is boolean \fBtrue\fR) it +uses a new connection for each POST. +.PP +If the server is processing a number of pipelined requests, and sends a +response header +.QW "\fBConnection: close\fR" +with one of the responses (other than the last), then subsequent responses +are unfulfilled. \fBhttp::geturl\fR will send the unfulfilled requests again +over a new connection. +.PP +A difficulty arises when a HTTP client sends a request over a persistent +connection that has been idle for a while. The HTTP server may +half-close an apparently idle connection while the client is sending a +request, but before the request arrives at the server: in this case (an +.QW "asynchronous close event" ) +the request will fail. The difficulty arises because the client cannot +be certain whether the POST modified the state of the server. For HEAD or +GET requests, \fBhttp::geturl\fR opens another connection and retransmits +the failed request. However, if the request was a POST, RFC 7230 forbids +automatic retry by default, suggesting either user confirmation, or +confirmation by user-agent software that has semantic understanding of +the application. The \fBhttp::config\fR option \fB-repost\fR allows for +either possibility. +.PP +Asynchronous close events can occur only in a short interval of time. The +\fBhttp\fR package monitors each persistent connection for closure by the +server. Upon detection, the connection is also closed at the client end, +and subsequent requests will use a fresh connection. +.PP +If the \fBhttp::geturl\fR command is called with option \fB-keepalive true\fR, +then it will both try to use an existing persistent connection +(if one is available), and it will send the server a +.QW "\fBConnection: keep-alive\fR" +request header asking to keep the connection open for future requests. +.PP +The \fBhttp::config\fR options \fB-pipeline\fR, \fB-postfresh\fR, and +\fB-repost\fR relate to persistent connections. +.PP +Option \fB-pipeline\fR, if boolean \fBtrue\fR, will pipeline GET and HEAD requests +made +over a persistent connection. POST requests will not be pipelined - if the +POST is not the first transaction on the connection, its request will not +be sent until the previous response has finished. GET and HEAD requests +made after a POST will not be sent until the POST response has been +delivered, and will not be sent if the POST fails. +.PP +Option \fB-postfresh\fR, if boolean \fBtrue\fR, will override the \fBhttp::geturl\fR option +\fB-keepalive\fR, and always open a fresh connection for a POST request. +.PP +Option \fB-repost\fR, if \fBtrue\fR, permits automatic retry of a POST request +that fails because it uses a persistent connection that the server has +half-closed (an +.QW "asynchronous close event" ). +Subsequent GET and HEAD requests in a failed pipeline will also be retried. +\fIThe -repost option should be used only if the application understands +that the retry is appropriate\fR - specifically, the application must know +that if the failed POST successfully modified the state of the server, a repeat POST +would have no adverse effect. +.VS TIP406 +.SH "COOKIE JAR PROTOCOL" +.PP +Cookies are short key-value pairs used to implement sessions within the +otherwise-stateless HTTP protocol. (See RFC 6265 for details; Tcl does not +implement the Cookie2 protocol as that is rarely seen in the wild.) +.PP +Cookie storage managment commands \(em +.QW "cookie jars" +\(em must support these subcommands which form the HTTP cookie storage +management protocol. Note that \fIcookieJar\fR below does not have to be a +command name; it is properly a command prefix (a Tcl list of words that will +be expanded in place) and admits many possible implementations. +.PP +Though not formally part of the protocol, it is expected that particular +values of \fIcookieJar\fR will correspond to sessions; it is up to the caller +of \fB::http::config\fR to decide what session applies and to manage the +deletion of said sessions when they are no longer desired (which should be +when they not configured as the current cookie jar). +.TP +\fIcookieJar \fBgetCookies \fIprotocol host requestPath\fR +. +This command asks the cookie jar what cookies should be supplied for a +particular request. It should take the \fIprotocol\fR (typically \fBhttp\fR or +\fBhttps\fR), \fIhost\fR name and \fIrequestPath\fR (parsed from the \fIurl\fR +argument to \fB::http::geturl\fR) and return a list of cookie keys and values +that describe the cookies to supply to the remote host. The list must have an +even number of elements. +.RS +.PP +There should only ever be at most one cookie with a particular key for any +request (typically the one with the most specific \fIhost\fR/domain match and +most specific \fIrequestPath\fR/path match), but there may be many cookies +with different names in any request. +.RE +.TP +\fIcookieJar \fBstoreCookie \fIcookieDictionary\fR +. +This command asks the cookie jar to store a particular cookie that was +returned by a request; the result of this command is ignored. The cookie +(which will have been parsed by the http package) is described by a +dictionary, \fIcookieDictionary\fR, that may have the following keys: +.RS +.TP +\fBdomain\fR +. +This is always present. Its value describes the domain hostname \fIor +prefix\fR that the cookie should be returned for. The checking of the domain +against the origin (below) should be careful since sites that issue cookies +should only do so for domains related to themselves. Cookies that do not obey +a relevant origin matching rule should be ignored. +.TP +\fBexpires\fR +. +This is optional. If present, the cookie is intended to be a persistent cookie +and the value of the option is the Tcl timestamp (in seconds from the same +base as \fBclock seconds\fR) of when the cookie expires (which may be in the +past, which should result in the cookie being deleted immediately). If absent, +the cookie is intended to be a session cookie that should be not persisted +beyond the lifetime of the cookie jar. +.TP +\fBhostonly\fR +. +This is always present. Its value is a boolean that describes whether the +cookie is a single host cookie (true) or a domain-level cookie (false). +.TP +\fBhttponly\fR +. +This is always present. Its value is a boolean that is true when the site +wishes the cookie to only ever be used with HTTP (or HTTPS) traffic. +.TP +\fBkey\fR +. +This is always present. Its value is the \fIkey\fR of the cookie, which is +part of the information that must be return when sending this cookie back in a +future request. +.TP +\fBorigin\fR +. +This is always present. Its value describes where the http package believes it +received the cookie from, which may be useful for checking whether the +cookie's domain is valid. +.TP +\fBpath\fR +. +This is always present. Its value describes the path prefix of requests to the +cookie domain where the cookie should be returned. +.TP +\fBsecure\fR +. +This is always present. Its value is a boolean that is true when the cookie +should only used on requests sent over secure channels (typically HTTPS). +.TP +\fBvalue\fR +. +This is always present. Its value is the value of the cookie, which is part of +the information that must be return when sending this cookie back in a future +request. +.PP +Other keys may always be ignored; they have no meaning in this protocol. +.RE +.VE TIP406 .SH EXAMPLE .PP This example creates a procedure to copy a URL to a file while printing a |