This is my first CGI script

Error

name:", form["name"].value + print "

addr:", form["addr"].value + ...further form processing here... + +Here the fields, accessed through ``form[key]``, are themselves instances of +:class:`FieldStorage` (or :class:`MiniFieldStorage`, depending on the form +encoding). The :attr:`value` attribute of the instance yields the string value +of the field. The :meth:`getvalue` method returns this string value directly; +it also accepts an optional second argument as a default to return if the +requested key is not present. + +If the submitted form data contains more than one field with the same name, the +object retrieved by ``form[key]`` is not a :class:`FieldStorage` or +:class:`MiniFieldStorage` instance but a list of such instances. Similarly, in +this situation, ``form.getvalue(key)`` would return a list of strings. If you +expect this possibility (when your HTML form contains multiple fields with the +same name), use the :func:`getlist` function, which always returns a list of +values (so that you do not need to special-case the single item case). For +example, this code concatenates any number of username fields, separated by +commas:: + + value = form.getlist("username") + usernames = ",".join(value) + +If a field represents an uploaded file, accessing the value via the +:attr:`value` attribute or the :func:`getvalue` method reads the entire file in +memory as a string. This may not be what you want. You can test for an uploaded +file by testing either the :attr:`filename` attribute or the :attr:`file` +attribute. You can then read the data at leisure from the :attr:`file` +attribute:: + + fileitem = form["userfile"] + if fileitem.file: + # It's an uploaded file; count lines + linecount = 0 + while 1: + line = fileitem.file.readline() + if not line: break + linecount = linecount + 1 + +The file upload draft standard entertains the possibility of uploading multiple +files from one field (using a recursive :mimetype:`multipart/\*` encoding). +When this occurs, the item will be a dictionary-like :class:`FieldStorage` item. +This can be determined by testing its :attr:`type` attribute, which should be +:mimetype:`multipart/form-data` (or perhaps another MIME type matching +:mimetype:`multipart/\*`). In this case, it can be iterated over recursively +just like the top-level form object. + +When a form is submitted in the "old" format (as the query string or as a single +data part of type :mimetype:`application/x-www-form-urlencoded`), the items will +actually be instances of the class :class:`MiniFieldStorage`. In this case, the +:attr:`list`, :attr:`file`, and :attr:`filename` attributes are always ``None``. + + +Higher Level Interface +---------------------- + +.. versionadded:: 2.2 + +The previous section explains how to read CGI form data using the +:class:`FieldStorage` class. This section describes a higher level interface +which was added to this class to allow one to do it in a more readable and +intuitive way. The interface doesn't make the techniques described in previous +sections obsolete --- they are still useful to process file uploads efficiently, +for example. + +.. % XXX: Is this true ? + +The interface consists of two simple methods. Using the methods you can process +form data in a generic way, without the need to worry whether only one or more +values were posted under one name. + +In the previous section, you learned to write following code anytime you +expected a user to post more than one value under one name:: + + item = form.getvalue("item") + if isinstance(item, list): + # The user is requesting more than one item. + else: + # The user is requesting only one item. + +This situation is common for example when a form contains a group of multiple +checkboxes with the same name:: + + + + +In most situations, however, there's only one form control with a particular +name in a form and then you expect and need only one value associated with this +name. So you write a script containing for example this code:: + + user = form.getvalue("user").upper() + +The problem with the code is that you should never expect that a client will +provide valid input to your scripts. For example, if a curious user appends +another ``user=foo`` pair to the query string, then the script would crash, +because in this situation the ``getvalue("user")`` method call returns a list +instead of a string. Calling the :meth:`toupper` method on a list is not valid +(since lists do not have a method of this name) and results in an +:exc:`AttributeError` exception. + +Therefore, the appropriate way to read form data values was to always use the +code which checks whether the obtained value is a single value or a list of +values. That's annoying and leads to less readable scripts. + +A more convenient approach is to use the methods :meth:`getfirst` and +:meth:`getlist` provided by this higher level interface. + + +.. method:: FieldStorage.getfirst(name[, default]) + + This method always returns only one value associated with form field *name*. + The method returns only the first value in case that more values were posted + under such name. Please note that the order in which the values are received + may vary from browser to browser and should not be counted on. [#]_ If no such + form field or value exists then the method returns the value specified by the + optional parameter *default*. This parameter defaults to ``None`` if not + specified. + + +.. method:: FieldStorage.getlist(name) + + This method always returns a list of values associated with form field *name*. + The method returns an empty list if no such form field or value exists for + *name*. It returns a list consisting of one item if only one such value exists. + +Using these methods you can write nice compact code:: + + import cgi + form = cgi.FieldStorage() + user = form.getfirst("user", "").upper() # This way it's safe. + for item in form.getlist("item"): + do_something(item) + + +Old classes +----------- + +These classes, present in earlier versions of the :mod:`cgi` module, are still +supported for backward compatibility. New applications should use the +:class:`FieldStorage` class. + +:class:`SvFormContentDict` stores single value form content as dictionary; it +assumes each field name occurs in the form only once. + +:class:`FormContentDict` stores multiple value form content as a dictionary (the +form items are lists of values). Useful if your form contains multiple fields +with the same name. + +Other classes (:class:`FormContent`, :class:`InterpFormContentDict`) are present +for backwards compatibility with really old applications only. If you still use +these and would be inconvenienced when they disappeared from a next version of +this module, drop me a note. + + +.. _functions-in-cgi-module: + +Functions +--------- + +These are useful if you want more control, or if you want to employ some of the +algorithms implemented in this module in other circumstances. + + +.. function:: parse(fp[, keep_blank_values[, strict_parsing]]) + + Parse a query in the environment or from a file (the file defaults to + ``sys.stdin``). The *keep_blank_values* and *strict_parsing* parameters are + passed to :func:`parse_qs` unchanged. + + +.. function:: parse_qs(qs[, keep_blank_values[, strict_parsing]]) + + Parse a query string given as a string argument (data of type + :mimetype:`application/x-www-form-urlencoded`). Data are returned as a + dictionary. The dictionary keys are the unique query variable names and the + values are lists of values for each name. + + The optional argument *keep_blank_values* is a flag indicating whether blank + values in URL encoded queries should be treated as blank strings. A true value + indicates that blanks should be retained as blank strings. The default false + value indicates that blank values are to be ignored and treated as if they were + not included. + + The optional argument *strict_parsing* is a flag indicating what to do with + parsing errors. If false (the default), errors are silently ignored. If true, + errors raise a :exc:`ValueError` exception. + + Use the :func:`urllib.urlencode` function to convert such dictionaries into + query strings. + + +.. function:: parse_qsl(qs[, keep_blank_values[, strict_parsing]]) + + Parse a query string given as a string argument (data of type + :mimetype:`application/x-www-form-urlencoded`). Data are returned as a list of + name, value pairs. + + The optional argument *keep_blank_values* is a flag indicating whether blank + values in URL encoded queries should be treated as blank strings. A true value + indicates that blanks should be retained as blank strings. The default false + value indicates that blank values are to be ignored and treated as if they were + not included. + + The optional argument *strict_parsing* is a flag indicating what to do with + parsing errors. If false (the default), errors are silently ignored. If true, + errors raise a :exc:`ValueError` exception. + + Use the :func:`urllib.urlencode` function to convert such lists of pairs into + query strings. + + +.. function:: parse_multipart(fp, pdict) + + Parse input of type :mimetype:`multipart/form-data` (for file uploads). + Arguments are *fp* for the input file and *pdict* for a dictionary containing + other parameters in the :mailheader:`Content-Type` header. + + Returns a dictionary just like :func:`parse_qs` keys are the field names, each + value is a list of values for that field. This is easy to use but not much good + if you are expecting megabytes to be uploaded --- in that case, use the + :class:`FieldStorage` class instead which is much more flexible. + + Note that this does not parse nested multipart parts --- use + :class:`FieldStorage` for that. + + +.. function:: parse_header(string) + + Parse a MIME header (such as :mailheader:`Content-Type`) into a main value and a + dictionary of parameters. + + +.. function:: test() + + Robust test CGI script, usable as main program. Writes minimal HTTP headers and + formats all information provided to the script in HTML form. + + +.. function:: print_environ() + + Format the shell environment in HTML. + + +.. function:: print_form(form) + + Format a form in HTML. + + +.. function:: print_directory() + + Format the current directory in HTML. + + +.. function:: print_environ_usage() + + Print a list of useful (used by CGI) environment variables in HTML. + + +.. function:: escape(s[, quote]) + + Convert the characters ``'&'``, ``'<'`` and ``'>'`` in string *s* to HTML-safe + sequences. Use this if you need to display text that might contain such + characters in HTML. If the optional flag *quote* is true, the quotation mark + character (``'"'``) is also translated; this helps for inclusion in an HTML + attribute value, as in ````. If the value to be quoted might + include single- or double-quote characters, or both, consider using the + :func:`quoteattr` function in the :mod:`xml.sax.saxutils` module instead. + + +.. _cgi-security: + +Caring about security +--------------------- + +.. index:: pair: CGI; security + +There's one important rule: if you invoke an external program (via the +:func:`os.system` or :func:`os.popen` functions. or others with similar +functionality), make very sure you don't pass arbitrary strings received from +the client to the shell. This is a well-known security hole whereby clever +hackers anywhere on the Web can exploit a gullible CGI script to invoke +arbitrary shell commands. Even parts of the URL or field names cannot be +trusted, since the request doesn't have to come from your form! + +To be on the safe side, if you must pass a string gotten from a form to a shell +command, you should make sure the string contains only alphanumeric characters, +dashes, underscores, and periods. + + +Installing your CGI script on a Unix system +------------------------------------------- + +Read the documentation for your HTTP server and check with your local system +administrator to find the directory where CGI scripts should be installed; +usually this is in a directory :file:`cgi-bin` in the server tree. + +Make sure that your script is readable and executable by "others"; the Unix file +mode should be ``0755`` octal (use ``chmod 0755 filename``). Make sure that the +first line of the script contains ``#!`` starting in column 1 followed by the +pathname of the Python interpreter, for instance:: + + #!/usr/local/bin/python + +Make sure the Python interpreter exists and is executable by "others". + +Make sure that any files your script needs to read or write are readable or +writable, respectively, by "others" --- their mode should be ``0644`` for +readable and ``0666`` for writable. This is because, for security reasons, the +HTTP server executes your script as user "nobody", without any special +privileges. It can only read (write, execute) files that everybody can read +(write, execute). The current directory at execution time is also different (it +is usually the server's cgi-bin directory) and the set of environment variables +is also different from what you get when you log in. In particular, don't count +on the shell's search path for executables (:envvar:`PATH`) or the Python module +search path (:envvar:`PYTHONPATH`) to be set to anything interesting. + +If you need to load modules from a directory which is not on Python's default +module search path, you can change the path in your script, before importing +other modules. For example:: + + import sys + sys.path.insert(0, "/usr/home/joe/lib/python") + sys.path.insert(0, "/usr/local/lib/python") + +(This way, the directory inserted last will be searched first!) + +Instructions for non-Unix systems will vary; check your HTTP server's +documentation (it will usually have a section on CGI scripts). + + +Testing your CGI script +----------------------- + +Unfortunately, a CGI script will generally not run when you try it from the +command line, and a script that works perfectly from the command line may fail +mysteriously when run from the server. There's one reason why you should still +test your script from the command line: if it contains a syntax error, the +Python interpreter won't execute it at all, and the HTTP server will most likely +send a cryptic error to the client. + +Assuming your script has no syntax errors, yet it does not work, you have no +choice but to read the next section. + + +Debugging CGI scripts +--------------------- + +.. index:: pair: CGI; debugging + +First of all, check for trivial installation errors --- reading the section +above on installing your CGI script carefully can save you a lot of time. If +you wonder whether you have understood the installation procedure correctly, try +installing a copy of this module file (:file:`cgi.py`) as a CGI script. When +invoked as a script, the file will dump its environment and the contents of the +form in HTML form. Give it the right mode etc, and send it a request. If it's +installed in the standard :file:`cgi-bin` directory, it should be possible to +send it a request by entering a URL into your browser of the form:: + + http://yourhostname/cgi-bin/cgi.py?name=Joe+Blow&addr=At+Home + +If this gives an error of type 404, the server cannot find the script -- perhaps +you need to install it in a different directory. If it gives another error, +there's an installation problem that you should fix before trying to go any +further. If you get a nicely formatted listing of the environment and form +content (in this example, the fields should be listed as "addr" with value "At +Home" and "name" with value "Joe Blow"), the :file:`cgi.py` script has been +installed correctly. If you follow the same procedure for your own script, you +should now be able to debug it. + +The next step could be to call the :mod:`cgi` module's :func:`test` function +from your script: replace its main code with the single statement :: + + cgi.test() + +This should produce the same results as those gotten from installing the +:file:`cgi.py` file itself. + +When an ordinary Python script raises an unhandled exception (for whatever +reason: of a typo in a module name, a file that can't be opened, etc.), the +Python interpreter prints a nice traceback and exits. While the Python +interpreter will still do this when your CGI script raises an exception, most +likely the traceback will end up in one of the HTTP server's log files, or be +discarded altogether. + +Fortunately, once you have managed to get your script to execute *some* code, +you can easily send tracebacks to the Web browser using the :mod:`cgitb` module. +If you haven't done so already, just add the line:: + + import cgitb; cgitb.enable() + +to the top of your script. Then try running it again; when a problem occurs, +you should see a detailed report that will likely make apparent the cause of the +crash. + +If you suspect that there may be a problem in importing the :mod:`cgitb` module, +you can use an even more robust approach (which only uses built-in modules):: + + import sys + sys.stderr = sys.stdout + print "Content-Type: text/plain" + print + ...your code here... + +This relies on the Python interpreter to print the traceback. The content type +of the output is set to plain text, which disables all HTML processing. If your +script works, the raw HTML will be displayed by your client. If it raises an +exception, most likely after the first two lines have been printed, a traceback +will be displayed. Because no HTML interpretation is going on, the traceback +will be readable. + + +Common problems and solutions +----------------------------- + +* Most HTTP servers buffer the output from CGI scripts until the script is + completed. This means that it is not possible to display a progress report on + the client's display while the script is running. + +* Check the installation instructions above. + +* Check the HTTP server's log files. (``tail -f logfile`` in a separate window + may be useful!) + +* Always check a script for syntax errors first, by doing something like + ``python script.py``. + +* If your script does not have any syntax errors, try adding ``import cgitb; + cgitb.enable()`` to the top of the script. + +* When invoking external programs, make sure they can be found. Usually, this + means using absolute path names --- :envvar:`PATH` is usually not set to a very + useful value in a CGI script. + +* When reading or writing external files, make sure they can be read or written + by the userid under which your CGI script will be running: this is typically the + userid under which the web server is running, or some explicitly specified + userid for a web server's ``suexec`` feature. + +* Don't try to give a CGI script a set-uid mode. This doesn't work on most + systems, and is a security liability as well. + +.. rubric:: Footnotes + +.. [#] Note that some recent versions of the HTML specification do state what order the + field values should be supplied in, but knowing whether a request was + received from a conforming browser, or even from a browser at all, is tedious + and error-prone. + diff --git a/Doc/library/cgihttpserver.rst b/Doc/library/cgihttpserver.rst new file mode 100644 index 0000000..4f27627 --- /dev/null +++ b/Doc/library/cgihttpserver.rst @@ -0,0 +1,73 @@ + +:mod:`CGIHTTPServer` --- CGI-capable HTTP request handler +========================================================= + +.. module:: CGIHTTPServer + :synopsis: This module provides a request handler for HTTP servers which can run CGI + scripts. +.. sectionauthor:: Moshe Zadka + + +The :mod:`CGIHTTPServer` module defines a request-handler class, interface +compatible with :class:`BaseHTTPServer.BaseHTTPRequestHandler` and inherits +behavior from :class:`SimpleHTTPServer.SimpleHTTPRequestHandler` but can also +run CGI scripts. + +.. note:: + + This module can run CGI scripts on Unix and Windows systems; on Mac OS it will + only be able to run Python scripts within the same process as itself. + +.. note:: + + CGI scripts run by the :class:`CGIHTTPRequestHandler` class cannot execute + redirects (HTTP code 302), because code 200 (script output follows) is sent + prior to execution of the CGI script. This pre-empts the status code. + +The :mod:`CGIHTTPServer` module defines the following class: + + +.. class:: CGIHTTPRequestHandler(request, client_address, server) + + This class is used to serve either files or output of CGI scripts from the + current directory and below. Note that mapping HTTP hierarchic structure to + local directory structure is exactly as in + :class:`SimpleHTTPServer.SimpleHTTPRequestHandler`. + + The class will however, run the CGI script, instead of serving it as a file, if + it guesses it to be a CGI script. Only directory-based CGI are used --- the + other common server configuration is to treat special extensions as denoting CGI + scripts. + + The :func:`do_GET` and :func:`do_HEAD` functions are modified to run CGI scripts + and serve the output, instead of serving files, if the request leads to + somewhere below the ``cgi_directories`` path. + +The :class:`CGIHTTPRequestHandler` defines the following data member: + + +.. attribute:: CGIHTTPRequestHandler.cgi_directories + + This defaults to ``['/cgi-bin', '/htbin']`` and describes directories to treat + as containing CGI scripts. + +The :class:`CGIHTTPRequestHandler` defines the following methods: + + +.. method:: CGIHTTPRequestHandler.do_POST() + + This method serves the ``'POST'`` request type, only allowed for CGI scripts. + Error 501, "Can only POST to CGI scripts", is output when trying to POST to a + non-CGI url. + +Note that CGI scripts will be run with UID of user nobody, for security reasons. +Problems with the CGI script will be translated to error 403. + +For example usage, see the implementation of the :func:`test` function. + + +.. seealso:: + + Module :mod:`BaseHTTPServer` + Base class implementation for Web server and request handler. + diff --git a/Doc/library/cgitb.rst b/Doc/library/cgitb.rst new file mode 100644 index 0000000..327cd17 --- /dev/null +++ b/Doc/library/cgitb.rst @@ -0,0 +1,64 @@ + +:mod:`cgitb` --- Traceback manager for CGI scripts +================================================== + +.. module:: cgitb + :synopsis: Configurable traceback handler for CGI scripts. +.. moduleauthor:: Ka-Ping Yee +.. sectionauthor:: Fred L. Drake, Jr. + + +.. versionadded:: 2.2 + +.. index:: + single: CGI; exceptions + single: CGI; tracebacks + single: exceptions; in CGI scripts + single: tracebacks; in CGI scripts + +The :mod:`cgitb` module provides a special exception handler for Python scripts. +(Its name is a bit misleading. It was originally designed to display extensive +traceback information in HTML for CGI scripts. It was later generalized to also +display this information in plain text.) After this module is activated, if an +uncaught exception occurs, a detailed, formatted report will be displayed. The +report includes a traceback showing excerpts of the source code for each level, +as well as the values of the arguments and local variables to currently running +functions, to help you debug the problem. Optionally, you can save this +information to a file instead of sending it to the browser. + +To enable this feature, simply add one line to the top of your CGI script:: + + import cgitb; cgitb.enable() + +The options to the :func:`enable` function control whether the report is +displayed in the browser and whether the report is logged to a file for later +analysis. + + +.. function:: enable([display[, logdir[, context[, format]]]]) + + .. index:: single: excepthook() (in module sys) + + This function causes the :mod:`cgitb` module to take over the interpreter's + default handling for exceptions by setting the value of :attr:`sys.excepthook`. + + The optional argument *display* defaults to ``1`` and can be set to ``0`` to + suppress sending the traceback to the browser. If the argument *logdir* is + present, the traceback reports are written to files. The value of *logdir* + should be a directory where these files will be placed. The optional argument + *context* is the number of lines of context to display around the current line + of source code in the traceback; this defaults to ``5``. If the optional + argument *format* is ``"html"``, the output is formatted as HTML. Any other + value forces plain text output. The default value is ``"html"``. + + +.. function:: handler([info]) + + This function handles an exception using the default settings (that is, show a + report in the browser, but don't log to a file). This can be used when you've + caught an exception and want to report it using :mod:`cgitb`. The optional + *info* argument should be a 3-tuple containing an exception type, exception + value, and traceback object, exactly like the tuple returned by + :func:`sys.exc_info`. If the *info* argument is not supplied, the current + exception is obtained from :func:`sys.exc_info`. + diff --git a/Doc/library/chunk.rst b/Doc/library/chunk.rst new file mode 100644 index 0000000..2e1798d --- /dev/null +++ b/Doc/library/chunk.rst @@ -0,0 +1,130 @@ + +:mod:`chunk` --- Read IFF chunked data +====================================== + +.. module:: chunk + :synopsis: Module to read IFF chunks. +.. moduleauthor:: Sjoerd Mullender +.. sectionauthor:: Sjoerd Mullender + + +.. index:: + single: Audio Interchange File Format + single: AIFF + single: AIFF-C + single: Real Media File Format + single: RMFF + +This module provides an interface for reading files that use EA IFF 85 chunks. +[#]_ This format is used in at least the Audio Interchange File Format +(AIFF/AIFF-C) and the Real Media File Format (RMFF). The WAVE audio file format +is closely related and can also be read using this module. + +A chunk has the following structure: + ++---------+--------+-------------------------------+ +| Offset | Length | Contents | ++=========+========+===============================+ +| 0 | 4 | Chunk ID | ++---------+--------+-------------------------------+ +| 4 | 4 | Size of chunk in big-endian | +| | | byte order, not including the | +| | | header | ++---------+--------+-------------------------------+ +| 8 | *n* | Data bytes, where *n* is the | +| | | size given in the preceding | +| | | field | ++---------+--------+-------------------------------+ +| 8 + *n* | 0 or 1 | Pad byte needed if *n* is odd | +| | | and chunk alignment is used | ++---------+--------+-------------------------------+ + +The ID is a 4-byte string which identifies the type of chunk. + +The size field (a 32-bit value, encoded using big-endian byte order) gives the +size of the chunk data, not including the 8-byte header. + +Usually an IFF-type file consists of one or more chunks. The proposed usage of +the :class:`Chunk` class defined here is to instantiate an instance at the start +of each chunk and read from the instance until it reaches the end, after which a +new instance can be instantiated. At the end of the file, creating a new +instance will fail with a :exc:`EOFError` exception. + + +.. class:: Chunk(file[, align, bigendian, inclheader]) + + Class which represents a chunk. The *file* argument is expected to be a + file-like object. An instance of this class is specifically allowed. The + only method that is needed is :meth:`read`. If the methods :meth:`seek` and + :meth:`tell` are present and don't raise an exception, they are also used. + If these methods are present and raise an exception, they are expected to not + have altered the object. If the optional argument *align* is true, chunks + are assumed to be aligned on 2-byte boundaries. If *align* is false, no + alignment is assumed. The default value is true. If the optional argument + *bigendian* is false, the chunk size is assumed to be in little-endian order. + This is needed for WAVE audio files. The default value is true. If the + optional argument *inclheader* is true, the size given in the chunk header + includes the size of the header. The default value is false. + +A :class:`Chunk` object supports the following methods: + + +.. method:: Chunk.getname() + + Returns the name (ID) of the chunk. This is the first 4 bytes of the chunk. + + +.. method:: Chunk.getsize() + + Returns the size of the chunk. + + +.. method:: Chunk.close() + + Close and skip to the end of the chunk. This does not close the underlying + file. + +The remaining methods will raise :exc:`IOError` if called after the +:meth:`close` method has been called. + + +.. method:: Chunk.isatty() + + Returns ``False``. + + +.. method:: Chunk.seek(pos[, whence]) + + Set the chunk's current position. The *whence* argument is optional and + defaults to ``0`` (absolute file positioning); other values are ``1`` (seek + relative to the current position) and ``2`` (seek relative to the file's end). + There is no return value. If the underlying file does not allow seek, only + forward seeks are allowed. + + +.. method:: Chunk.tell() + + Return the current position into the chunk. + + +.. method:: Chunk.read([size]) + + Read at most *size* bytes from the chunk (less if the read hits the end of the + chunk before obtaining *size* bytes). If the *size* argument is negative or + omitted, read all data until the end of the chunk. The bytes are returned as a + string object. An empty string is returned when the end of the chunk is + encountered immediately. + + +.. method:: Chunk.skip() + + Skip to the end of the chunk. All further calls to :meth:`read` for the chunk + will return ``''``. If you are not interested in the contents of the chunk, + this method should be called so that the file points to the start of the next + chunk. + +.. rubric:: Footnotes + +.. [#] "EA IFF 85" Standard for Interchange Format Files, Jerry Morrison, Electronic + Arts, January 1985. + diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst new file mode 100644 index 0000000..2bc162c --- /dev/null +++ b/Doc/library/cmath.rst @@ -0,0 +1,156 @@ + +:mod:`cmath` --- Mathematical functions for complex numbers +=========================================================== + +.. module:: cmath + :synopsis: Mathematical functions for complex numbers. + + +This module is always available. It provides access to mathematical functions +for complex numbers. The functions in this module accept integers, +floating-point numbers or complex numbers as arguments. They will also accept +any Python object that has either a :meth:`__complex__` or a :meth:`__float__` +method: these methods are used to convert the object to a complex or +floating-point number, respectively, and the function is then applied to the +result of the conversion. + +The functions are: + + +.. function:: acos(x) + + Return the arc cosine of *x*. There are two branch cuts: One extends right from + 1 along the real axis to ∞, continuous from below. The other extends left from + -1 along the real axis to -∞, continuous from above. + + +.. function:: acosh(x) + + Return the hyperbolic arc cosine of *x*. There is one branch cut, extending left + from 1 along the real axis to -∞, continuous from above. + + +.. function:: asin(x) + + Return the arc sine of *x*. This has the same branch cuts as :func:`acos`. + + +.. function:: asinh(x) + + Return the hyperbolic arc sine of *x*. There are two branch cuts, extending + left from ``±1j`` to ``±∞j``, both continuous from above. These branch cuts + should be considered a bug to be corrected in a future release. The correct + branch cuts should extend along the imaginary axis, one from ``1j`` up to + ``∞j`` and continuous from the right, and one from ``-1j`` down to ``-∞j`` + and continuous from the left. + + +.. function:: atan(x) + + Return the arc tangent of *x*. There are two branch cuts: One extends from + ``1j`` along the imaginary axis to ``∞j``, continuous from the left. The + other extends from ``-1j`` along the imaginary axis to ``-∞j``, continuous + from the left. (This should probably be changed so the upper cut becomes + continuous from the other side.) + + +.. function:: atanh(x) + + Return the hyperbolic arc tangent of *x*. There are two branch cuts: One + extends from ``1`` along the real axis to ``∞``, continuous from above. The + other extends from ``-1`` along the real axis to ``-∞``, continuous from + above. (This should probably be changed so the right cut becomes continuous + from the other side.) + + +.. function:: cos(x) + + Return the cosine of *x*. + + +.. function:: cosh(x) + + Return the hyperbolic cosine of *x*. + + +.. function:: exp(x) + + Return the exponential value ``e**x``. + + +.. function:: log(x[, base]) + + Returns the logarithm of *x* to the given *base*. If the *base* is not + specified, returns the natural logarithm of *x*. There is one branch cut, from 0 + along the negative real axis to -∞, continuous from above. + + .. versionchanged:: 2.4 + *base* argument added. + + +.. function:: log10(x) + + Return the base-10 logarithm of *x*. This has the same branch cut as + :func:`log`. + + +.. function:: sin(x) + + Return the sine of *x*. + + +.. function:: sinh(x) + + Return the hyperbolic sine of *x*. + + +.. function:: sqrt(x) + + Return the square root of *x*. This has the same branch cut as :func:`log`. + + +.. function:: tan(x) + + Return the tangent of *x*. + + +.. function:: tanh(x) + + Return the hyperbolic tangent of *x*. + +The module also defines two mathematical constants: + + +.. data:: pi + + The mathematical constant *pi*, as a float. + + +.. data:: e + + The mathematical constant *e*, as a float. + +.. index:: module: math + +Note that the selection of functions is similar, but not identical, to that in +module :mod:`math`. The reason for having two modules is that some users aren't +interested in complex numbers, and perhaps don't even know what they are. They +would rather have ``math.sqrt(-1)`` raise an exception than return a complex +number. Also note that the functions defined in :mod:`cmath` always return a +complex number, even if the answer can be expressed as a real number (in which +case the complex number has an imaginary part of zero). + +A note on branch cuts: They are curves along which the given function fails to +be continuous. They are a necessary feature of many complex functions. It is +assumed that if you need to compute with complex functions, you will understand +about branch cuts. Consult almost any (not too elementary) book on complex +variables for enlightenment. For information of the proper choice of branch +cuts for numerical purposes, a good reference should be the following: + + +.. seealso:: + + Kahan, W: Branch cuts for complex elementary functions; or, Much ado about + nothing's sign bit. In Iserles, A., and Powell, M. (eds.), The state of the art + in numerical analysis. Clarendon Press (1987) pp165-211. + diff --git a/Doc/library/cmd.rst b/Doc/library/cmd.rst new file mode 100644 index 0000000..9af08e2 --- /dev/null +++ b/Doc/library/cmd.rst @@ -0,0 +1,202 @@ + +:mod:`cmd` --- Support for line-oriented command interpreters +============================================================= + +.. module:: cmd + :synopsis: Build line-oriented command interpreters. +.. sectionauthor:: Eric S. Raymond + + +The :class:`Cmd` class provides a simple framework for writing line-oriented +command interpreters. These are often useful for test harnesses, administrative +tools, and prototypes that will later be wrapped in a more sophisticated +interface. + + +.. class:: Cmd([completekey[, stdin[, stdout]]]) + + A :class:`Cmd` instance or subclass instance is a line-oriented interpreter + framework. There is no good reason to instantiate :class:`Cmd` itself; rather, + it's useful as a superclass of an interpreter class you define yourself in order + to inherit :class:`Cmd`'s methods and encapsulate action methods. + + The optional argument *completekey* is the :mod:`readline` name of a completion + key; it defaults to :kbd:`Tab`. If *completekey* is not :const:`None` and + :mod:`readline` is available, command completion is done automatically. + + The optional arguments *stdin* and *stdout* specify the input and output file + objects that the Cmd instance or subclass instance will use for input and + output. If not specified, they will default to *sys.stdin* and *sys.stdout*. + + .. versionchanged:: 2.3 + The *stdin* and *stdout* parameters were added. + + +.. _cmd-objects: + +Cmd Objects +----------- + +A :class:`Cmd` instance has the following methods: + + +.. method:: Cmd.cmdloop([intro]) + + Repeatedly issue a prompt, accept input, parse an initial prefix off the + received input, and dispatch to action methods, passing them the remainder of + the line as argument. + + The optional argument is a banner or intro string to be issued before the first + prompt (this overrides the :attr:`intro` class member). + + If the :mod:`readline` module is loaded, input will automatically inherit + :program:`bash`\ -like history-list editing (e.g. :kbd:`Control-P` scrolls back + to the last command, :kbd:`Control-N` forward to the next one, :kbd:`Control-F` + moves the cursor to the right non-destructively, :kbd:`Control-B` moves the + cursor to the left non-destructively, etc.). + + An end-of-file on input is passed back as the string ``'EOF'``. + + An interpreter instance will recognize a command name ``foo`` if and only if it + has a method :meth:`do_foo`. As a special case, a line beginning with the + character ``'?'`` is dispatched to the method :meth:`do_help`. As another + special case, a line beginning with the character ``'!'`` is dispatched to the + method :meth:`do_shell` (if such a method is defined). + + This method will return when the :meth:`postcmd` method returns a true value. + The *stop* argument to :meth:`postcmd` is the return value from the command's + corresponding :meth:`do_\*` method. + + If completion is enabled, completing commands will be done automatically, and + completing of commands args is done by calling :meth:`complete_foo` with + arguments *text*, *line*, *begidx*, and *endidx*. *text* is the string prefix + we are attempting to match: all returned matches must begin with it. *line* is + the current input line with leading whitespace removed, *begidx* and *endidx* + are the beginning and ending indexes of the prefix text, which could be used to + provide different completion depending upon which position the argument is in. + + All subclasses of :class:`Cmd` inherit a predefined :meth:`do_help`. This + method, called with an argument ``'bar'``, invokes the corresponding method + :meth:`help_bar`. With no argument, :meth:`do_help` lists all available help + topics (that is, all commands with corresponding :meth:`help_\*` methods), and + also lists any undocumented commands. + + +.. method:: Cmd.onecmd(str) + + Interpret the argument as though it had been typed in response to the prompt. + This may be overridden, but should not normally need to be; see the + :meth:`precmd` and :meth:`postcmd` methods for useful execution hooks. The + return value is a flag indicating whether interpretation of commands by the + interpreter should stop. If there is a :meth:`do_\*` method for the command + *str*, the return value of that method is returned, otherwise the return value + from the :meth:`default` method is returned. + + +.. method:: Cmd.emptyline() + + Method called when an empty line is entered in response to the prompt. If this + method is not overridden, it repeats the last nonempty command entered. + + +.. method:: Cmd.default(line) + + Method called on an input line when the command prefix is not recognized. If + this method is not overridden, it prints an error message and returns. + + +.. method:: Cmd.completedefault(text, line, begidx, endidx) + + Method called to complete an input line when no command-specific + :meth:`complete_\*` method is available. By default, it returns an empty list. + + +.. method:: Cmd.precmd(line) + + Hook method executed just before the command line *line* is interpreted, but + after the input prompt is generated and issued. This method is a stub in + :class:`Cmd`; it exists to be overridden by subclasses. The return value is + used as the command which will be executed by the :meth:`onecmd` method; the + :meth:`precmd` implementation may re-write the command or simply return *line* + unchanged. + + +.. method:: Cmd.postcmd(stop, line) + + Hook method executed just after a command dispatch is finished. This method is + a stub in :class:`Cmd`; it exists to be overridden by subclasses. *line* is the + command line which was executed, and *stop* is a flag which indicates whether + execution will be terminated after the call to :meth:`postcmd`; this will be the + return value of the :meth:`onecmd` method. The return value of this method will + be used as the new value for the internal flag which corresponds to *stop*; + returning false will cause interpretation to continue. + + +.. method:: Cmd.preloop() + + Hook method executed once when :meth:`cmdloop` is called. This method is a stub + in :class:`Cmd`; it exists to be overridden by subclasses. + + +.. method:: Cmd.postloop() + + Hook method executed once when :meth:`cmdloop` is about to return. This method + is a stub in :class:`Cmd`; it exists to be overridden by subclasses. + +Instances of :class:`Cmd` subclasses have some public instance variables: + + +.. attribute:: Cmd.prompt + + The prompt issued to solicit input. + + +.. attribute:: Cmd.identchars + + The string of characters accepted for the command prefix. + + +.. attribute:: Cmd.lastcmd + + The last nonempty command prefix seen. + + +.. attribute:: Cmd.intro + + A string to issue as an intro or banner. May be overridden by giving the + :meth:`cmdloop` method an argument. + + +.. attribute:: Cmd.doc_header + + The header to issue if the help output has a section for documented commands. + + +.. attribute:: Cmd.misc_header + + The header to issue if the help output has a section for miscellaneous help + topics (that is, there are :meth:`help_\*` methods without corresponding + :meth:`do_\*` methods). + + +.. attribute:: Cmd.undoc_header + + The header to issue if the help output has a section for undocumented commands + (that is, there are :meth:`do_\*` methods without corresponding :meth:`help_\*` + methods). + + +.. attribute:: Cmd.ruler + + The character used to draw separator lines under the help-message headers. If + empty, no ruler line is drawn. It defaults to ``'='``. + + +.. attribute:: Cmd.use_rawinput + + A flag, defaulting to true. If true, :meth:`cmdloop` uses :func:`input` to + display a prompt and read the next command; if false, :meth:`sys.stdout.write` + and :meth:`sys.stdin.readline` are used. (This means that by importing + :mod:`readline`, on systems that support it, the interpreter will automatically + support :program:`Emacs`\ -like line editing and command-history keystrokes.) + diff --git a/Doc/library/code.rst b/Doc/library/code.rst new file mode 100644 index 0000000..4e00639 --- /dev/null +++ b/Doc/library/code.rst @@ -0,0 +1,167 @@ + +:mod:`code` --- Interpreter base classes +======================================== + +.. module:: code + :synopsis: Facilities to implement read-eval-print loops. + + + +The ``code`` module provides facilities to implement read-eval-print loops in +Python. Two classes and convenience functions are included which can be used to +build applications which provide an interactive interpreter prompt. + + +.. class:: InteractiveInterpreter([locals]) + + This class deals with parsing and interpreter state (the user's namespace); it + does not deal with input buffering or prompting or input file naming (the + filename is always passed in explicitly). The optional *locals* argument + specifies the dictionary in which code will be executed; it defaults to a newly + created dictionary with key ``'__name__'`` set to ``'__console__'`` and key + ``'__doc__'`` set to ``None``. + + +.. class:: InteractiveConsole([locals[, filename]]) + + Closely emulate the behavior of the interactive Python interpreter. This class + builds on :class:`InteractiveInterpreter` and adds prompting using the familiar + ``sys.ps1`` and ``sys.ps2``, and input buffering. + + +.. function:: interact([banner[, readfunc[, local]]]) + + Convenience function to run a read-eval-print loop. This creates a new instance + of :class:`InteractiveConsole` and sets *readfunc* to be used as the + :meth:`raw_input` method, if provided. If *local* is provided, it is passed to + the :class:`InteractiveConsole` constructor for use as the default namespace for + the interpreter loop. The :meth:`interact` method of the instance is then run + with *banner* passed as the banner to use, if provided. The console object is + discarded after use. + + +.. function:: compile_command(source[, filename[, symbol]]) + + This function is useful for programs that want to emulate Python's interpreter + main loop (a.k.a. the read-eval-print loop). The tricky part is to determine + when the user has entered an incomplete command that can be completed by + entering more text (as opposed to a complete command or a syntax error). This + function *almost* always makes the same decision as the real interpreter main + loop. + + *source* is the source string; *filename* is the optional filename from which + source was read, defaulting to ``''``; and *symbol* is the optional + grammar start symbol, which should be either ``'single'`` (the default) or + ``'eval'``. + + Returns a code object (the same as ``compile(source, filename, symbol)``) if the + command is complete and valid; ``None`` if the command is incomplete; raises + :exc:`SyntaxError` if the command is complete and contains a syntax error, or + raises :exc:`OverflowError` or :exc:`ValueError` if the command contains an + invalid literal. + + +.. _interpreter-objects: + +Interactive Interpreter Objects +------------------------------- + + +.. method:: InteractiveInterpreter.runsource(source[, filename[, symbol]]) + + Compile and run some source in the interpreter. Arguments are the same as for + :func:`compile_command`; the default for *filename* is ``''``, and for + *symbol* is ``'single'``. One several things can happen: + + * The input is incorrect; :func:`compile_command` raised an exception + (:exc:`SyntaxError` or :exc:`OverflowError`). A syntax traceback will be + printed by calling the :meth:`showsyntaxerror` method. :meth:`runsource` + returns ``False``. + + * The input is incomplete, and more input is required; :func:`compile_command` + returned ``None``. :meth:`runsource` returns ``True``. + + * The input is complete; :func:`compile_command` returned a code object. The + code is executed by calling the :meth:`runcode` (which also handles run-time + exceptions, except for :exc:`SystemExit`). :meth:`runsource` returns ``False``. + + The return value can be used to decide whether to use ``sys.ps1`` or ``sys.ps2`` + to prompt the next line. + + +.. method:: InteractiveInterpreter.runcode(code) + + Execute a code object. When an exception occurs, :meth:`showtraceback` is called + to display a traceback. All exceptions are caught except :exc:`SystemExit`, + which is allowed to propagate. + + A note about :exc:`KeyboardInterrupt`: this exception may occur elsewhere in + this code, and may not always be caught. The caller should be prepared to deal + with it. + + +.. method:: InteractiveInterpreter.showsyntaxerror([filename]) + + Display the syntax error that just occurred. This does not display a stack + trace because there isn't one for syntax errors. If *filename* is given, it is + stuffed into the exception instead of the default filename provided by Python's + parser, because it always uses ``''`` when reading from a string. The + output is written by the :meth:`write` method. + + +.. method:: InteractiveInterpreter.showtraceback() + + Display the exception that just occurred. We remove the first stack item + because it is within the interpreter object implementation. The output is + written by the :meth:`write` method. + + +.. method:: InteractiveInterpreter.write(data) + + Write a string to the standard error stream (``sys.stderr``). Derived classes + should override this to provide the appropriate output handling as needed. + + +.. _console-objects: + +Interactive Console Objects +--------------------------- + +The :class:`InteractiveConsole` class is a subclass of +:class:`InteractiveInterpreter`, and so offers all the methods of the +interpreter objects as well as the following additions. + + +.. method:: InteractiveConsole.interact([banner]) + + Closely emulate the interactive Python console. The optional banner argument + specify the banner to print before the first interaction; by default it prints a + banner similar to the one printed by the standard Python interpreter, followed + by the class name of the console object in parentheses (so as not to confuse + this with the real interpreter -- since it's so close!). + + +.. method:: InteractiveConsole.push(line) + + Push a line of source text to the interpreter. The line should not have a + trailing newline; it may have internal newlines. The line is appended to a + buffer and the interpreter's :meth:`runsource` method is called with the + concatenated contents of the buffer as source. If this indicates that the + command was executed or invalid, the buffer is reset; otherwise, the command is + incomplete, and the buffer is left as it was after the line was appended. The + return value is ``True`` if more input is required, ``False`` if the line was + dealt with in some way (this is the same as :meth:`runsource`). + + +.. method:: InteractiveConsole.resetbuffer() + + Remove any unhandled source text from the input buffer. + + +.. method:: InteractiveConsole.raw_input([prompt]) + + Write a prompt and read a line. The returned line does not include the trailing + newline. When the user enters the EOF key sequence, :exc:`EOFError` is raised. + The base implementation reads from ``sys.stdin``; a subclass may replace this + with a different implementation. + diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst new file mode 100644 index 0000000..38264df --- /dev/null +++ b/Doc/library/codecs.rst @@ -0,0 +1,1230 @@ + +:mod:`codecs` --- Codec registry and base classes +================================================= + +.. module:: codecs + :synopsis: Encode and decode data and streams. +.. moduleauthor:: Marc-Andre Lemburg +.. sectionauthor:: Marc-Andre Lemburg +.. sectionauthor:: Martin v. Löwis + + +.. index:: + single: Unicode + single: Codecs + pair: Codecs; encode + pair: Codecs; decode + single: streams + pair: stackable; streams + +This module defines base classes for standard Python codecs (encoders and +decoders) and provides access to the internal Python codec registry which +manages the codec and error handling lookup process. + +It defines the following functions: + + +.. function:: register(search_function) + + Register a codec search function. Search functions are expected to take one + argument, the encoding name in all lower case letters, and return a + :class:`CodecInfo` object having the following attributes: + + * ``name`` The name of the encoding; + + * ``encoder`` The stateless encoding function; + + * ``decoder`` The stateless decoding function; + + * ``incrementalencoder`` An incremental encoder class or factory function; + + * ``incrementaldecoder`` An incremental decoder class or factory function; + + * ``streamwriter`` A stream writer class or factory function; + + * ``streamreader`` A stream reader class or factory function. + + The various functions or classes take the following arguments: + + *encoder* and *decoder*: These must be functions or methods which have the same + interface as the :meth:`encode`/:meth:`decode` methods of Codec instances (see + Codec Interface). The functions/methods are expected to work in a stateless + mode. + + *incrementalencoder* and *incrementalencoder*: These have to be factory + functions providing the following interface: + + ``factory(errors='strict')`` + + The factory functions must return objects providing the interfaces defined by + the base classes :class:`IncrementalEncoder` and :class:`IncrementalEncoder`, + respectively. Incremental codecs can maintain state. + + *streamreader* and *streamwriter*: These have to be factory functions providing + the following interface: + + ``factory(stream, errors='strict')`` + + The factory functions must return objects providing the interfaces defined by + the base classes :class:`StreamWriter` and :class:`StreamReader`, respectively. + Stream codecs can maintain state. + + Possible values for errors are ``'strict'`` (raise an exception in case of an + encoding error), ``'replace'`` (replace malformed data with a suitable + replacement marker, such as ``'?'``), ``'ignore'`` (ignore malformed data and + continue without further notice), ``'xmlcharrefreplace'`` (replace with the + appropriate XML character reference (for encoding only)) and + ``'backslashreplace'`` (replace with backslashed escape sequences (for encoding + only)) as well as any other error handling name defined via + :func:`register_error`. + + In case a search function cannot find a given encoding, it should return + ``None``. + + +.. function:: lookup(encoding) + + Looks up the codec info in the Python codec registry and returns a + :class:`CodecInfo` object as defined above. + + Encodings are first looked up in the registry's cache. If not found, the list of + registered search functions is scanned. If no :class:`CodecInfo` object is + found, a :exc:`LookupError` is raised. Otherwise, the :class:`CodecInfo` object + is stored in the cache and returned to the caller. + +To simplify access to the various codecs, the module provides these additional +functions which use :func:`lookup` for the codec lookup: + + +.. function:: getencoder(encoding) + + Look up the codec for the given encoding and return its encoder function. + + Raises a :exc:`LookupError` in case the encoding cannot be found. + + +.. function:: getdecoder(encoding) + + Look up the codec for the given encoding and return its decoder function. + + Raises a :exc:`LookupError` in case the encoding cannot be found. + + +.. function:: getincrementalencoder(encoding) + + Look up the codec for the given encoding and return its incremental encoder + class or factory function. + + Raises a :exc:`LookupError` in case the encoding cannot be found or the codec + doesn't support an incremental encoder. + + .. versionadded:: 2.5 + + +.. function:: getincrementaldecoder(encoding) + + Look up the codec for the given encoding and return its incremental decoder + class or factory function. + + Raises a :exc:`LookupError` in case the encoding cannot be found or the codec + doesn't support an incremental decoder. + + .. versionadded:: 2.5 + + +.. function:: getreader(encoding) + + Look up the codec for the given encoding and return its StreamReader class or + factory function. + + Raises a :exc:`LookupError` in case the encoding cannot be found. + + +.. function:: getwriter(encoding) + + Look up the codec for the given encoding and return its StreamWriter class or + factory function. + + Raises a :exc:`LookupError` in case the encoding cannot be found. + + +.. function:: register_error(name, error_handler) + + Register the error handling function *error_handler* under the name *name*. + *error_handler* will be called during encoding and decoding in case of an error, + when *name* is specified as the errors parameter. + + For encoding *error_handler* will be called with a :exc:`UnicodeEncodeError` + instance, which contains information about the location of the error. The error + handler must either raise this or a different exception or return a tuple with a + replacement for the unencodable part of the input and a position where encoding + should continue. The encoder will encode the replacement and continue encoding + the original input at the specified position. Negative position values will be + treated as being relative to the end of the input string. If the resulting + position is out of bound an :exc:`IndexError` will be raised. + + Decoding and translating works similar, except :exc:`UnicodeDecodeError` or + :exc:`UnicodeTranslateError` will be passed to the handler and that the + replacement from the error handler will be put into the output directly. + + +.. function:: lookup_error(name) + + Return the error handler previously registered under the name *name*. + + Raises a :exc:`LookupError` in case the handler cannot be found. + + +.. function:: strict_errors(exception) + + Implements the ``strict`` error handling. + + +.. function:: replace_errors(exception) + + Implements the ``replace`` error handling. + + +.. function:: ignore_errors(exception) + + Implements the ``ignore`` error handling. + + +.. function:: xmlcharrefreplace_errors_errors(exception) + + Implements the ``xmlcharrefreplace`` error handling. + + +.. function:: backslashreplace_errors_errors(exception) + + Implements the ``backslashreplace`` error handling. + +To simplify working with encoded files or stream, the module also defines these +utility functions: + + +.. function:: open(filename, mode[, encoding[, errors[, buffering]]]) + + Open an encoded file using the given *mode* and return a wrapped version + providing transparent encoding/decoding. + + .. note:: + + The wrapped version will only accept the object format defined by the codecs, + i.e. Unicode objects for most built-in codecs. Output is also codec-dependent + and will usually be Unicode as well. + + *encoding* specifies the encoding which is to be used for the file. + + *errors* may be given to define the error handling. It defaults to ``'strict'`` + which causes a :exc:`ValueError` to be raised in case an encoding error occurs. + + *buffering* has the same meaning as for the built-in :func:`open` function. It + defaults to line buffered. + + +.. function:: EncodedFile(file, input[, output[, errors]]) + + Return a wrapped version of file which provides transparent encoding + translation. + + Strings written to the wrapped file are interpreted according to the given + *input* encoding and then written to the original file as strings using the + *output* encoding. The intermediate encoding will usually be Unicode but depends + on the specified codecs. + + If *output* is not given, it defaults to *input*. + + *errors* may be given to define the error handling. It defaults to ``'strict'``, + which causes :exc:`ValueError` to be raised in case an encoding error occurs. + + +.. function:: iterencode(iterable, encoding[, errors]) + + Uses an incremental encoder to iteratively encode the input provided by + *iterable*. This function is a generator. *errors* (as well as any other keyword + argument) is passed through to the incremental encoder. + + .. versionadded:: 2.5 + + +.. function:: iterdecode(iterable, encoding[, errors]) + + Uses an incremental decoder to iteratively decode the input provided by + *iterable*. This function is a generator. *errors* (as well as any other keyword + argument) is passed through to the incremental decoder. + + .. versionadded:: 2.5 + +The module also provides the following constants which are useful for reading +and writing to platform dependent files: + + +.. data:: BOM + BOM_BE + BOM_LE + BOM_UTF8 + BOM_UTF16 + BOM_UTF16_BE + BOM_UTF16_LE + BOM_UTF32 + BOM_UTF32_BE + BOM_UTF32_LE + + These constants define various encodings of the Unicode byte order mark (BOM) + used in UTF-16 and UTF-32 data streams to indicate the byte order used in the + stream or file and in UTF-8 as a Unicode signature. :const:`BOM_UTF16` is either + :const:`BOM_UTF16_BE` or :const:`BOM_UTF16_LE` depending on the platform's + native byte order, :const:`BOM` is an alias for :const:`BOM_UTF16`, + :const:`BOM_LE` for :const:`BOM_UTF16_LE` and :const:`BOM_BE` for + :const:`BOM_UTF16_BE`. The others represent the BOM in UTF-8 and UTF-32 + encodings. + + +.. _codec-base-classes: + +Codec Base Classes +------------------ + +The :mod:`codecs` module defines a set of base classes which define the +interface and can also be used to easily write you own codecs for use in Python. + +Each codec has to define four interfaces to make it usable as codec in Python: +stateless encoder, stateless decoder, stream reader and stream writer. The +stream reader and writers typically reuse the stateless encoder/decoder to +implement the file protocols. + +The :class:`Codec` class defines the interface for stateless encoders/decoders. + +To simplify and standardize error handling, the :meth:`encode` and +:meth:`decode` methods may implement different error handling schemes by +providing the *errors* string argument. The following string values are defined +and implemented by all standard Python codecs: + ++-------------------------+-----------------------------------------------+ +| Value | Meaning | ++=========================+===============================================+ +| ``'strict'`` | Raise :exc:`UnicodeError` (or a subclass); | +| | this is the default. | ++-------------------------+-----------------------------------------------+ +| ``'ignore'`` | Ignore the character and continue with the | +| | next. | ++-------------------------+-----------------------------------------------+ +| ``'replace'`` | Replace with a suitable replacement | +| | character; Python will use the official | +| | U+FFFD REPLACEMENT CHARACTER for the built-in | +| | Unicode codecs on decoding and '?' on | +| | encoding. | ++-------------------------+-----------------------------------------------+ +| ``'xmlcharrefreplace'`` | Replace with the appropriate XML character | +| | reference (only for encoding). | ++-------------------------+-----------------------------------------------+ +| ``'backslashreplace'`` | Replace with backslashed escape sequences | +| | (only for encoding). | ++-------------------------+-----------------------------------------------+ + +The set of allowed values can be extended via :meth:`register_error`. + + +.. _codec-objects: + +Codec Objects +^^^^^^^^^^^^^ + +The :class:`Codec` class defines these methods which also define the function +interfaces of the stateless encoder and decoder: + + +.. method:: Codec.encode(input[, errors]) + + Encodes the object *input* and returns a tuple (output object, length consumed). + While codecs are not restricted to use with Unicode, in a Unicode context, + encoding converts a Unicode object to a plain string using a particular + character set encoding (e.g., ``cp1252`` or ``iso-8859-1``). + + *errors* defines the error handling to apply. It defaults to ``'strict'`` + handling. + + The method may not store state in the :class:`Codec` instance. Use + :class:`StreamCodec` for codecs which have to keep state in order to make + encoding/decoding efficient. + + The encoder must be able to handle zero length input and return an empty object + of the output object type in this situation. + + +.. method:: Codec.decode(input[, errors]) + + Decodes the object *input* and returns a tuple (output object, length consumed). + In a Unicode context, decoding converts a plain string encoded using a + particular character set encoding to a Unicode object. + + *input* must be an object which provides the ``bf_getreadbuf`` buffer slot. + Python strings, buffer objects and memory mapped files are examples of objects + providing this slot. + + *errors* defines the error handling to apply. It defaults to ``'strict'`` + handling. + + The method may not store state in the :class:`Codec` instance. Use + :class:`StreamCodec` for codecs which have to keep state in order to make + encoding/decoding efficient. + + The decoder must be able to handle zero length input and return an empty object + of the output object type in this situation. + +The :class:`IncrementalEncoder` and :class:`IncrementalDecoder` classes provide +the basic interface for incremental encoding and decoding. Encoding/decoding the +input isn't done with one call to the stateless encoder/decoder function, but +with multiple calls to the :meth:`encode`/:meth:`decode` method of the +incremental encoder/decoder. The incremental encoder/decoder keeps track of the +encoding/decoding process during method calls. + +The joined output of calls to the :meth:`encode`/:meth:`decode` method is the +same as if all the single inputs were joined into one, and this input was +encoded/decoded with the stateless encoder/decoder. + + +.. _incremental-encoder-objects: + +IncrementalEncoder Objects +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. versionadded:: 2.5 + +The :class:`IncrementalEncoder` class is used for encoding an input in multiple +steps. It defines the following methods which every incremental encoder must +define in order to be compatible with the Python codec registry. + + +.. class:: IncrementalEncoder([errors]) + + Constructor for an :class:`IncrementalEncoder` instance. + + All incremental encoders must provide this constructor interface. They are free + to add additional keyword arguments, but only the ones defined here are used by + the Python codec registry. + + The :class:`IncrementalEncoder` may implement different error handling schemes + by providing the *errors* keyword argument. These parameters are predefined: + + * ``'strict'`` Raise :exc:`ValueError` (or a subclass); this is the default. + + * ``'ignore'`` Ignore the character and continue with the next. + + * ``'replace'`` Replace with a suitable replacement character + + * ``'xmlcharrefreplace'`` Replace with the appropriate XML character reference + + * ``'backslashreplace'`` Replace with backslashed escape sequences. + + The *errors* argument will be assigned to an attribute of the same name. + Assigning to this attribute makes it possible to switch between different error + handling strategies during the lifetime of the :class:`IncrementalEncoder` + object. + + The set of allowed values for the *errors* argument can be extended with + :func:`register_error`. + + +.. method:: IncrementalEncoder.encode(object[, final]) + + Encodes *object* (taking the current state of the encoder into account) and + returns the resulting encoded object. If this is the last call to :meth:`encode` + *final* must be true (the default is false). + + +.. method:: IncrementalEncoder.reset() + + Reset the encoder to the initial state. + + +.. method:: IncrementalEncoder.getstate() + + Return the current state of the encoder which must be an integer. The + implementation should make sure that ``0`` is the most common state. (States + that are more complicated than integers can be converted into an integer by + marshaling/pickling the state and encoding the bytes of the resulting string + into an integer). + + .. versionadded:: 3.0 + + +.. method:: IncrementalEncoder.setstate(state) + + Set the state of the encoder to *state*. *state* must be an encoder state + returned by :meth:`getstate`. + + .. versionadded:: 3.0 + + +.. _incremental-decoder-objects: + +IncrementalDecoder Objects +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The :class:`IncrementalDecoder` class is used for decoding an input in multiple +steps. It defines the following methods which every incremental decoder must +define in order to be compatible with the Python codec registry. + + +.. class:: IncrementalDecoder([errors]) + + Constructor for an :class:`IncrementalDecoder` instance. + + All incremental decoders must provide this constructor interface. They are free + to add additional keyword arguments, but only the ones defined here are used by + the Python codec registry. + + The :class:`IncrementalDecoder` may implement different error handling schemes + by providing the *errors* keyword argument. These parameters are predefined: + + * ``'strict'`` Raise :exc:`ValueError` (or a subclass); this is the default. + + * ``'ignore'`` Ignore the character and continue with the next. + + * ``'replace'`` Replace with a suitable replacement character. + + The *errors* argument will be assigned to an attribute of the same name. + Assigning to this attribute makes it possible to switch between different error + handling strategies during the lifetime of the :class:`IncrementalEncoder` + object. + + The set of allowed values for the *errors* argument can be extended with + :func:`register_error`. + + +.. method:: IncrementalDecoder.decode(object[, final]) + + Decodes *object* (taking the current state of the decoder into account) and + returns the resulting decoded object. If this is the last call to :meth:`decode` + *final* must be true (the default is false). If *final* is true the decoder must + decode the input completely and must flush all buffers. If this isn't possible + (e.g. because of incomplete byte sequences at the end of the input) it must + initiate error handling just like in the stateless case (which might raise an + exception). + + +.. method:: IncrementalDecoder.reset() + + Reset the decoder to the initial state. + + +.. method:: IncrementalDecoder.getstate() + + Return the current state of the decoder. This must be a tuple with two items, + the first must be the buffer containing the still undecoded input. The second + must be an integer and can be additional state info. (The implementation should + make sure that ``0`` is the most common additional state info.) If this + additional state info is ``0`` it must be possible to set the decoder to the + state which has no input buffered and ``0`` as the additional state info, so + that feeding the previously buffered input to the decoder returns it to the + previous state without producing any output. (Additional state info that is more + complicated than integers can be converted into an integer by + marshaling/pickling the info and encoding the bytes of the resulting string into + an integer.) + + .. versionadded:: 3.0 + + +.. method:: IncrementalDecoder.setstate(state) + + Set the state of the encoder to *state*. *state* must be a decoder state + returned by :meth:`getstate`. + + .. versionadded:: 3.0 + +The :class:`StreamWriter` and :class:`StreamReader` classes provide generic +working interfaces which can be used to implement new encoding submodules very +easily. See :mod:`encodings.utf_8` for an example of how this is done. + + +.. _stream-writer-objects: + +StreamWriter Objects +^^^^^^^^^^^^^^^^^^^^ + +The :class:`StreamWriter` class is a subclass of :class:`Codec` and defines the +following methods which every stream writer must define in order to be +compatible with the Python codec registry. + + +.. class:: StreamWriter(stream[, errors]) + + Constructor for a :class:`StreamWriter` instance. + + All stream writers must provide this constructor interface. They are free to add + additional keyword arguments, but only the ones defined here are used by the + Python codec registry. + + *stream* must be a file-like object open for writing binary data. + + The :class:`StreamWriter` may implement different error handling schemes by + providing the *errors* keyword argument. These parameters are predefined: + + * ``'strict'`` Raise :exc:`ValueError` (or a subclass); this is the default. + + * ``'ignore'`` Ignore the character and continue with the next. + + * ``'replace'`` Replace with a suitable replacement character + + * ``'xmlcharrefreplace'`` Replace with the appropriate XML character reference + + * ``'backslashreplace'`` Replace with backslashed escape sequences. + + The *errors* argument will be assigned to an attribute of the same name. + Assigning to this attribute makes it possible to switch between different error + handling strategies during the lifetime of the :class:`StreamWriter` object. + + The set of allowed values for the *errors* argument can be extended with + :func:`register_error`. + + +.. method:: StreamWriter.write(object) + + Writes the object's contents encoded to the stream. + + +.. method:: StreamWriter.writelines(list) + + Writes the concatenated list of strings to the stream (possibly by reusing the + :meth:`write` method). + + +.. method:: StreamWriter.reset() + + Flushes and resets the codec buffers used for keeping state. + + Calling this method should ensure that the data on the output is put into a + clean state that allows appending of new fresh data without having to rescan the + whole stream to recover state. + +In addition to the above methods, the :class:`StreamWriter` must also inherit +all other methods and attributes from the underlying stream. + + +.. _stream-reader-objects: + +StreamReader Objects +^^^^^^^^^^^^^^^^^^^^ + +The :class:`StreamReader` class is a subclass of :class:`Codec` and defines the +following methods which every stream reader must define in order to be +compatible with the Python codec registry. + + +.. class:: StreamReader(stream[, errors]) + + Constructor for a :class:`StreamReader` instance. + + All stream readers must provide this constructor interface. They are free to add + additional keyword arguments, but only the ones defined here are used by the + Python codec registry. + + *stream* must be a file-like object open for reading (binary) data. + + The :class:`StreamReader` may implement different error handling schemes by + providing the *errors* keyword argument. These parameters are defined: + + * ``'strict'`` Raise :exc:`ValueError` (or a subclass); this is the default. + + * ``'ignore'`` Ignore the character and continue with the next. + + * ``'replace'`` Replace with a suitable replacement character. + + The *errors* argument will be assigned to an attribute of the same name. + Assigning to this attribute makes it possible to switch between different error + handling strategies during the lifetime of the :class:`StreamReader` object. + + The set of allowed values for the *errors* argument can be extended with + :func:`register_error`. + + +.. method:: StreamReader.read([size[, chars, [firstline]]]) + + Decodes data from the stream and returns the resulting object. + + *chars* indicates the number of characters to read from the stream. :func:`read` + will never return more than *chars* characters, but it might return less, if + there are not enough characters available. + + *size* indicates the approximate maximum number of bytes to read from the stream + for decoding purposes. The decoder can modify this setting as appropriate. The + default value -1 indicates to read and decode as much as possible. *size* is + intended to prevent having to decode huge files in one step. + + *firstline* indicates that it would be sufficient to only return the first line, + if there are decoding errors on later lines. + + The method should use a greedy read strategy meaning that it should read as much + data as is allowed within the definition of the encoding and the given size, + e.g. if optional encoding endings or state markers are available on the stream, + these should be read too. + + .. versionchanged:: 2.4 + *chars* argument added. + + .. versionchanged:: 2.4.2 + *firstline* argument added. + + +.. method:: StreamReader.readline([size[, keepends]]) + + Read one line from the input stream and return the decoded data. + + *size*, if given, is passed as size argument to the stream's :meth:`readline` + method. + + If *keepends* is false line-endings will be stripped from the lines returned. + + .. versionchanged:: 2.4 + *keepends* argument added. + + +.. method:: StreamReader.readlines([sizehint[, keepends]]) + + Read all lines available on the input stream and return them as a list of lines. + + Line-endings are implemented using the codec's decoder method and are included + in the list entries if *keepends* is true. + + *sizehint*, if given, is passed as the *size* argument to the stream's + :meth:`read` method. + + +.. method:: StreamReader.reset() + + Resets the codec buffers used for keeping state. + + Note that no stream repositioning should take place. This method is primarily + intended to be able to recover from decoding errors. + +In addition to the above methods, the :class:`StreamReader` must also inherit +all other methods and attributes from the underlying stream. + +The next two base classes are included for convenience. They are not needed by +the codec registry, but may provide useful in practice. + + +.. _stream-reader-writer: + +StreamReaderWriter Objects +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The :class:`StreamReaderWriter` allows wrapping streams which work in both read +and write modes. + +The design is such that one can use the factory functions returned by the +:func:`lookup` function to construct the instance. + + +.. class:: StreamReaderWriter(stream, Reader, Writer, errors) + + Creates a :class:`StreamReaderWriter` instance. *stream* must be a file-like + object. *Reader* and *Writer* must be factory functions or classes providing the + :class:`StreamReader` and :class:`StreamWriter` interface resp. Error handling + is done in the same way as defined for the stream readers and writers. + +:class:`StreamReaderWriter` instances define the combined interfaces of +:class:`StreamReader` and :class:`StreamWriter` classes. They inherit all other +methods and attributes from the underlying stream. + + +.. _stream-recoder-objects: + +StreamRecoder Objects +^^^^^^^^^^^^^^^^^^^^^ + +The :class:`StreamRecoder` provide a frontend - backend view of encoding data +which is sometimes useful when dealing with different encoding environments. + +The design is such that one can use the factory functions returned by the +:func:`lookup` function to construct the instance. + + +.. class:: StreamRecoder(stream, encode, decode, Reader, Writer, errors) + + Creates a :class:`StreamRecoder` instance which implements a two-way conversion: + *encode* and *decode* work on the frontend (the input to :meth:`read` and output + of :meth:`write`) while *Reader* and *Writer* work on the backend (reading and + writing to the stream). + + You can use these objects to do transparent direct recodings from e.g. Latin-1 + to UTF-8 and back. + + *stream* must be a file-like object. + + *encode*, *decode* must adhere to the :class:`Codec` interface. *Reader*, + *Writer* must be factory functions or classes providing objects of the + :class:`StreamReader` and :class:`StreamWriter` interface respectively. + + *encode* and *decode* are needed for the frontend translation, *Reader* and + *Writer* for the backend translation. The intermediate format used is + determined by the two sets of codecs, e.g. the Unicode codecs will use Unicode + as the intermediate encoding. + + Error handling is done in the same way as defined for the stream readers and + writers. + +:class:`StreamRecoder` instances define the combined interfaces of +:class:`StreamReader` and :class:`StreamWriter` classes. They inherit all other +methods and attributes from the underlying stream. + + +.. _encodings-overview: + +Encodings and Unicode +--------------------- + +Unicode strings are stored internally as sequences of codepoints (to be precise +as :ctype:`Py_UNICODE` arrays). Depending on the way Python is compiled (either +via :option:`--enable-unicode=ucs2` or :option:`--enable-unicode=ucs4`, with the +former being the default) :ctype:`Py_UNICODE` is either a 16-bit or 32-bit data +type. Once a Unicode object is used outside of CPU and memory, CPU endianness +and how these arrays are stored as bytes become an issue. Transforming a +unicode object into a sequence of bytes is called encoding and recreating the +unicode object from the sequence of bytes is known as decoding. There are many +different methods for how this transformation can be done (these methods are +also called encodings). The simplest method is to map the codepoints 0-255 to +the bytes ``0x0``-``0xff``. This means that a unicode object that contains +codepoints above ``U+00FF`` can't be encoded with this method (which is called +``'latin-1'`` or ``'iso-8859-1'``). :func:`unicode.encode` will raise a +:exc:`UnicodeEncodeError` that looks like this: ``UnicodeEncodeError: 'latin-1' +codec can't encode character u'\u1234' in position 3: ordinal not in +range(256)``. + +There's another group of encodings (the so called charmap encodings) that choose +a different subset of all unicode code points and how these codepoints are +mapped to the bytes ``0x0``-``0xff``. To see how this is done simply open +e.g. :file:`encodings/cp1252.py` (which is an encoding that is used primarily on +Windows). There's a string constant with 256 characters that shows you which +character is mapped to which byte value. + +All of these encodings can only encode 256 of the 65536 (or 1114111) codepoints +defined in unicode. A simple and straightforward way that can store each Unicode +code point, is to store each codepoint as two consecutive bytes. There are two +possibilities: Store the bytes in big endian or in little endian order. These +two encodings are called UTF-16-BE and UTF-16-LE respectively. Their +disadvantage is that if e.g. you use UTF-16-BE on a little endian machine you +will always have to swap bytes on encoding and decoding. UTF-16 avoids this +problem: Bytes will always be in natural endianness. When these bytes are read +by a CPU with a different endianness, then bytes have to be swapped though. To +be able to detect the endianness of a UTF-16 byte sequence, there's the so +called BOM (the "Byte Order Mark"). This is the Unicode character ``U+FEFF``. +This character will be prepended to every UTF-16 byte sequence. The byte swapped +version of this character (``0xFFFE``) is an illegal character that may not +appear in a Unicode text. So when the first character in an UTF-16 byte sequence +appears to be a ``U+FFFE`` the bytes have to be swapped on decoding. +Unfortunately upto Unicode 4.0 the character ``U+FEFF`` had a second purpose as +a ``ZERO WIDTH NO-BREAK SPACE``: A character that has no width and doesn't allow +a word to be split. It can e.g. be used to give hints to a ligature algorithm. +With Unicode 4.0 using ``U+FEFF`` as a ``ZERO WIDTH NO-BREAK SPACE`` has been +deprecated (with ``U+2060`` (``WORD JOINER``) assuming this role). Nevertheless +Unicode software still must be able to handle ``U+FEFF`` in both roles: As a BOM +it's a device to determine the storage layout of the encoded bytes, and vanishes +once the byte sequence has been decoded into a Unicode string; as a ``ZERO WIDTH +NO-BREAK SPACE`` it's a normal character that will be decoded like any other. + +There's another encoding that is able to encoding the full range of Unicode +characters: UTF-8. UTF-8 is an 8-bit encoding, which means there are no issues +with byte order in UTF-8. Each byte in a UTF-8 byte sequence consists of two +parts: Marker bits (the most significant bits) and payload bits. The marker bits +are a sequence of zero to six 1 bits followed by a 0 bit. Unicode characters are +encoded like this (with x being payload bits, which when concatenated give the +Unicode character): + ++-----------------------------------+----------------------------------------------+ +| Range | Encoding | ++===================================+==============================================+ +| ``U-00000000`` ... ``U-0000007F`` | 0xxxxxxx | ++-----------------------------------+----------------------------------------------+ +| ``U-00000080`` ... ``U-000007FF`` | 110xxxxx 10xxxxxx | ++-----------------------------------+----------------------------------------------+ +| ``U-00000800`` ... ``U-0000FFFF`` | 1110xxxx 10xxxxxx 10xxxxxx | ++-----------------------------------+----------------------------------------------+ +| ``U-00010000`` ... ``U-001FFFFF`` | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | ++-----------------------------------+----------------------------------------------+ +| ``U-00200000`` ... ``U-03FFFFFF`` | 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx | ++-----------------------------------+----------------------------------------------+ +| ``U-04000000`` ... ``U-7FFFFFFF`` | 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx | +| | 10xxxxxx | ++-----------------------------------+----------------------------------------------+ + +The least significant bit of the Unicode character is the rightmost x bit. + +As UTF-8 is an 8-bit encoding no BOM is required and any ``U+FEFF`` character in +the decoded Unicode string (even if it's the first character) is treated as a +``ZERO WIDTH NO-BREAK SPACE``. + +Without external information it's impossible to reliably determine which +encoding was used for encoding a Unicode string. Each charmap encoding can +decode any random byte sequence. However that's not possible with UTF-8, as +UTF-8 byte sequences have a structure that doesn't allow arbitrary byte +sequence. To increase the reliability with which a UTF-8 encoding can be +detected, Microsoft invented a variant of UTF-8 (that Python 2.5 calls +``"utf-8-sig"``) for its Notepad program: Before any of the Unicode characters +is written to the file, a UTF-8 encoded BOM (which looks like this as a byte +sequence: ``0xef``, ``0xbb``, ``0xbf``) is written. As it's rather improbable +that any charmap encoded file starts with these byte values (which would e.g. +map to + + | LATIN SMALL LETTER I WITH DIAERESIS + | RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK + | INVERTED QUESTION MARK + +in iso-8859-1), this increases the probability that a utf-8-sig encoding can be +correctly guessed from the byte sequence. So here the BOM is not used to be able +to determine the byte order used for generating the byte sequence, but as a +signature that helps in guessing the encoding. On encoding the utf-8-sig codec +will write ``0xef``, ``0xbb``, ``0xbf`` as the first three bytes to the file. On +decoding utf-8-sig will skip those three bytes if they appear as the first three +bytes in the file. + + +.. _standard-encodings: + +Standard Encodings +------------------ + +Python comes with a number of codecs built-in, either implemented as C functions +or with dictionaries as mapping tables. The following table lists the codecs by +name, together with a few common aliases, and the languages for which the +encoding is likely used. Neither the list of aliases nor the list of languages +is meant to be exhaustive. Notice that spelling alternatives that only differ in +case or use a hyphen instead of an underscore are also valid aliases. + +Many of the character sets support the same languages. They vary in individual +characters (e.g. whether the EURO SIGN is supported or not), and in the +assignment of characters to code positions. For the European languages in +particular, the following variants typically exist: + +* an ISO 8859 codeset + +* a Microsoft Windows code page, which is typically derived from a 8859 codeset, + but replaces control characters with additional graphic characters + +* an IBM EBCDIC code page + +* an IBM PC code page, which is ASCII compatible + ++-----------------+--------------------------------+--------------------------------+ +| Codec | Aliases | Languages | ++=================+================================+================================+ +| ascii | 646, us-ascii | English | ++-----------------+--------------------------------+--------------------------------+ +| big5 | big5-tw, csbig5 | Traditional Chinese | ++-----------------+--------------------------------+--------------------------------+ +| big5hkscs | big5-hkscs, hkscs | Traditional Chinese | ++-----------------+--------------------------------+--------------------------------+ +| cp037 | IBM037, IBM039 | English | ++-----------------+--------------------------------+--------------------------------+ +| cp424 | EBCDIC-CP-HE, IBM424 | Hebrew | ++-----------------+--------------------------------+--------------------------------+ +| cp437 | 437, IBM437 | English | ++-----------------+--------------------------------+--------------------------------+ +| cp500 | EBCDIC-CP-BE, EBCDIC-CP-CH, | Western Europe | +| | IBM500 | | ++-----------------+--------------------------------+--------------------------------+ +| cp737 | | Greek | ++-----------------+--------------------------------+--------------------------------+ +| cp775 | IBM775 | Baltic languages | ++-----------------+--------------------------------+--------------------------------+ +| cp850 | 850, IBM850 | Western Europe | ++-----------------+--------------------------------+--------------------------------+ +| cp852 | 852, IBM852 | Central and Eastern Europe | ++-----------------+--------------------------------+--------------------------------+ +| cp855 | 855, IBM855 | Bulgarian, Byelorussian, | +| | | Macedonian, Russian, Serbian | ++-----------------+--------------------------------+--------------------------------+ +| cp856 | | Hebrew | ++-----------------+--------------------------------+--------------------------------+ +| cp857 | 857, IBM857 | Turkish | ++-----------------+--------------------------------+--------------------------------+ +| cp860 | 860, IBM860 | Portuguese | ++-----------------+--------------------------------+--------------------------------+ +| cp861 | 861, CP-IS, IBM861 | Icelandic | ++-----------------+--------------------------------+--------------------------------+ +| cp862 | 862, IBM862 | Hebrew | ++-----------------+--------------------------------+--------------------------------+ +| cp863 | 863, IBM863 | Canadian | ++-----------------+--------------------------------+--------------------------------+ +| cp864 | IBM864 | Arabic | ++-----------------+--------------------------------+--------------------------------+ +| cp865 | 865, IBM865 | Danish, Norwegian | ++-----------------+--------------------------------+--------------------------------+ +| cp866 | 866, IBM866 | Russian | ++-----------------+--------------------------------+--------------------------------+ +| cp869 | 869, CP-GR, IBM869 | Greek | ++-----------------+--------------------------------+--------------------------------+ +| cp874 | | Thai | ++-----------------+--------------------------------+--------------------------------+ +| cp875 | | Greek | ++-----------------+--------------------------------+--------------------------------+ +| cp932 | 932, ms932, mskanji, ms-kanji | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| cp949 | 949, ms949, uhc | Korean | ++-----------------+--------------------------------+--------------------------------+ +| cp950 | 950, ms950 | Traditional Chinese | ++-----------------+--------------------------------+--------------------------------+ +| cp1006 | | Urdu | ++-----------------+--------------------------------+--------------------------------+ +| cp1026 | ibm1026 | Turkish | ++-----------------+--------------------------------+--------------------------------+ +| cp1140 | ibm1140 | Western Europe | ++-----------------+--------------------------------+--------------------------------+ +| cp1250 | windows-1250 | Central and Eastern Europe | ++-----------------+--------------------------------+--------------------------------+ +| cp1251 | windows-1251 | Bulgarian, Byelorussian, | +| | | Macedonian, Russian, Serbian | ++-----------------+--------------------------------+--------------------------------+ +| cp1252 | windows-1252 | Western Europe | ++-----------------+--------------------------------+--------------------------------+ +| cp1253 | windows-1253 | Greek | ++-----------------+--------------------------------+--------------------------------+ +| cp1254 | windows-1254 | Turkish | ++-----------------+--------------------------------+--------------------------------+ +| cp1255 | windows-1255 | Hebrew | ++-----------------+--------------------------------+--------------------------------+ +| cp1256 | windows1256 | Arabic | ++-----------------+--------------------------------+--------------------------------+ +| cp1257 | windows-1257 | Baltic languages | ++-----------------+--------------------------------+--------------------------------+ +| cp1258 | windows-1258 | Vietnamese | ++-----------------+--------------------------------+--------------------------------+ +| euc_jp | eucjp, ujis, u-jis | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| euc_jis_2004 | jisx0213, eucjis2004 | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| euc_jisx0213 | eucjisx0213 | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| euc_kr | euckr, korean, ksc5601, | Korean | +| | ks_c-5601, ks_c-5601-1987, | | +| | ksx1001, ks_x-1001 | | ++-----------------+--------------------------------+--------------------------------+ +| gb2312 | chinese, csiso58gb231280, euc- | Simplified Chinese | +| | cn, euccn, eucgb2312-cn, | | +| | gb2312-1980, gb2312-80, iso- | | +| | ir-58 | | ++-----------------+--------------------------------+--------------------------------+ +| gbk | 936, cp936, ms936 | Unified Chinese | ++-----------------+--------------------------------+--------------------------------+ +| gb18030 | gb18030-2000 | Unified Chinese | ++-----------------+--------------------------------+--------------------------------+ +| hz | hzgb, hz-gb, hz-gb-2312 | Simplified Chinese | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_jp | csiso2022jp, iso2022jp, | Japanese | +| | iso-2022-jp | | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_jp_1 | iso2022jp-1, iso-2022-jp-1 | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_jp_2 | iso2022jp-2, iso-2022-jp-2 | Japanese, Korean, Simplified | +| | | Chinese, Western Europe, Greek | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_jp_2004 | iso2022jp-2004, | Japanese | +| | iso-2022-jp-2004 | | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_jp_3 | iso2022jp-3, iso-2022-jp-3 | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_jp_ext | iso2022jp-ext, iso-2022-jp-ext | Japanese | ++-----------------+--------------------------------+--------------------------------+ +| iso2022_kr | csiso2022kr, iso2022kr, | Korean | +| | iso-2022-kr | | ++-----------------+--------------------------------+--------------------------------+ +| latin_1 | iso-8859-1, iso8859-1, 8859, | West Europe | +| | cp819, latin, latin1, L1 | | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_2 | iso-8859-2, latin2, L2 | Central and Eastern Europe | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_3 | iso-8859-3, latin3, L3 | Esperanto, Maltese | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_4 | iso-8859-4, latin4, L4 | Baltic languagues | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_5 | iso-8859-5, cyrillic | Bulgarian, Byelorussian, | +| | | Macedonian, Russian, Serbian | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_6 | iso-8859-6, arabic | Arabic | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_7 | iso-8859-7, greek, greek8 | Greek | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_8 | iso-8859-8, hebrew | Hebrew | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_9 | iso-8859-9, latin5, L5 | Turkish | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_10 | iso-8859-10, latin6, L6 | Nordic languages | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_13 | iso-8859-13 | Baltic languages | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_14 | iso-8859-14, latin8, L8 | Celtic languages | ++-----------------+--------------------------------+--------------------------------+ +| iso8859_15 | iso-8859-15 | Western Europe | ++-----------------+--------------------------------+--------------------------------+ +| johab | cp1361, ms1361 | Korean | ++-----------------+--------------------------------+--------------------------------+ +| koi8_r | | Russian | ++-----------------+--------------------------------+--------------------------------+ +| koi8_u | | Ukrainian | ++-----------------+--------------------------------+--------------------------------+ +| mac_cyrillic | maccyrillic | Bulgarian, Byelorussian, | +| | | Macedonian, Russian, Serbian | ++-----------------+--------------------------------+--------------------------------+ +| mac_greek | macgreek | Greek | ++-----------------+--------------------------------+--------------------------------+ +| mac_iceland | maciceland | Icelandic | ++-----------------+--------------------------------+--------------------------------+ +| mac_latin2 | maclatin2, maccentraleurope | Central and Eastern Europe | ++-----------------+--------------------------------+--------------------------------+ +| mac_roman | macroman | Western Europe | ++-----------------+--------------------------------+--------------------------------+ +| mac_turkish | macturkish | Turkish | ++-----------------+--------------------------------+--------------------------------+ +| ptcp154 | csptcp154, pt154, cp154, | Kazakh | +| | cyrillic-asian | | ++-----------------+--------------------------------+--------------------------------+ +| shift_jis | csshiftjis, shiftjis, sjis, | Japanese | +| | s_jis | | ++-----------------+--------------------------------+--------------------------------+ +| shift_jis_2004 | shiftjis2004, sjis_2004, | Japanese | +| | sjis2004 | | ++-----------------+--------------------------------+--------------------------------+ +| shift_jisx0213 | shiftjisx0213, sjisx0213, | Japanese | +| | s_jisx0213 | | ++-----------------+--------------------------------+--------------------------------+ +| utf_16 | U16, utf16 | all languages | ++-----------------+--------------------------------+--------------------------------+ +| utf_16_be | UTF-16BE | all languages (BMP only) | ++-----------------+--------------------------------+--------------------------------+ +| utf_16_le | UTF-16LE | all languages (BMP only) | ++-----------------+--------------------------------+--------------------------------+ +| utf_7 | U7, unicode-1-1-utf-7 | all languages | ++-----------------+--------------------------------+--------------------------------+ +| utf_8 | U8, UTF, utf8 | all languages | ++-----------------+--------------------------------+--------------------------------+ +| utf_8_sig | | all languages | ++-----------------+--------------------------------+--------------------------------+ + +A number of codecs are specific to Python, so their codec names have no meaning +outside Python. Some of them don't convert from Unicode strings to byte strings, +but instead use the property of the Python codecs machinery that any bijective +function with one argument can be considered as an encoding. + +For the codecs listed below, the result in the "encoding" direction is always a +byte string. The result of the "decoding" direction is listed as operand type in +the table. + ++--------------------+---------+----------------+---------------------------+ +| Codec | Aliases | Operand type | Purpose | ++====================+=========+================+===========================+ +| idna | | Unicode string | Implements :rfc:`3490`, | +| | | | see also | +| | | | :mod:`encodings.idna` | ++--------------------+---------+----------------+---------------------------+ +| mbcs | dbcs | Unicode string | Windows only: Encode | +| | | | operand according to the | +| | | | ANSI codepage (CP_ACP) | ++--------------------+---------+----------------+---------------------------+ +| palmos | | Unicode string | Encoding of PalmOS 3.5 | ++--------------------+---------+----------------+---------------------------+ +| punycode | | Unicode string | Implements :rfc:`3492` | ++--------------------+---------+----------------+---------------------------+ +| raw_unicode_escape | | Unicode string | Produce a string that is | +| | | | suitable as raw Unicode | +| | | | literal in Python source | +| | | | code | ++--------------------+---------+----------------+---------------------------+ +| undefined | | any | Raise an exception for | +| | | | all conversions. Can be | +| | | | used as the system | +| | | | encoding if no automatic | +| | | | coercion between byte and | +| | | | Unicode strings is | +| | | | desired. | ++--------------------+---------+----------------+---------------------------+ +| unicode_escape | | Unicode string | Produce a string that is | +| | | | suitable as Unicode | +| | | | literal in Python source | +| | | | code | ++--------------------+---------+----------------+---------------------------+ +| unicode_internal | | Unicode string | Return the internal | +| | | | representation of the | +| | | | operand | ++--------------------+---------+----------------+---------------------------+ + +.. versionadded:: 2.3 + The ``idna`` and ``punycode`` encodings. + + +:mod:`encodings.idna` --- Internationalized Domain Names in Applications +------------------------------------------------------------------------ + +.. module:: encodings.idna + :synopsis: Internationalized Domain Names implementation +.. moduleauthor:: Martin v. Löwis + +.. versionadded:: 2.3 + +This module implements :rfc:`3490` (Internationalized Domain Names in +Applications) and :rfc:`3492` (Nameprep: A Stringprep Profile for +Internationalized Domain Names (IDN)). It builds upon the ``punycode`` encoding +and :mod:`stringprep`. + +These RFCs together define a protocol to support non-ASCII characters in domain +names. A domain name containing non-ASCII characters (such as +``www.Alliancefrançaise.nu``) is converted into an ASCII-compatible encoding +(ACE, such as ``www.xn--alliancefranaise-npb.nu``). The ACE form of the domain +name is then used in all places where arbitrary characters are not allowed by +the protocol, such as DNS queries, HTTP :mailheader:`Host` fields, and so +on. This conversion is carried out in the application; if possible invisible to +the user: The application should transparently convert Unicode domain labels to +IDNA on the wire, and convert back ACE labels to Unicode before presenting them +to the user. + +Python supports this conversion in several ways: The ``idna`` codec allows to +convert between Unicode and the ACE. Furthermore, the :mod:`socket` module +transparently converts Unicode host names to ACE, so that applications need not +be concerned about converting host names themselves when they pass them to the +socket module. On top of that, modules that have host names as function +parameters, such as :mod:`httplib` and :mod:`ftplib`, accept Unicode host names +(:mod:`httplib` then also transparently sends an IDNA hostname in the +:mailheader:`Host` field if it sends that field at all). + +When receiving host names from the wire (such as in reverse name lookup), no +automatic conversion to Unicode is performed: Applications wishing to present +such host names to the user should decode them to Unicode. + +The module :mod:`encodings.idna` also implements the nameprep procedure, which +performs certain normalizations on host names, to achieve case-insensitivity of +international domain names, and to unify similar characters. The nameprep +functions can be used directly if desired. + + +.. function:: nameprep(label) + + Return the nameprepped version of *label*. The implementation currently assumes + query strings, so ``AllowUnassigned`` is true. + + +.. function:: ToASCII(label) + + Convert a label to ASCII, as specified in :rfc:`3490`. ``UseSTD3ASCIIRules`` is + assumed to be false. + + +.. function:: ToUnicode(label) + + Convert a label to Unicode, as specified in :rfc:`3490`. + + +:mod:`encodings.utf_8_sig` --- UTF-8 codec with BOM signature +------------------------------------------------------------- + +.. module:: encodings.utf_8_sig + :synopsis: UTF-8 codec with BOM signature +.. moduleauthor:: Walter Dörwald + +.. versionadded:: 2.5 + +This module implements a variant of the UTF-8 codec: On encoding a UTF-8 encoded +BOM will be prepended to the UTF-8 encoded bytes. For the stateful encoder this +is only done once (on the first write to the byte stream). For decoding an +optional UTF-8 encoded BOM at the start of the data will be skipped. + diff --git a/Doc/library/codeop.rst b/Doc/library/codeop.rst new file mode 100644 index 0000000..8a730ec --- /dev/null +++ b/Doc/library/codeop.rst @@ -0,0 +1,95 @@ + +:mod:`codeop` --- Compile Python code +===================================== + +.. module:: codeop + :synopsis: Compile (possibly incomplete) Python code. +.. sectionauthor:: Moshe Zadka +.. sectionauthor:: Michael Hudson + + +.. % LaTeXed from excellent doc-string. + +The :mod:`codeop` module provides utilities upon which the Python +read-eval-print loop can be emulated, as is done in the :mod:`code` module. As +a result, you probably don't want to use the module directly; if you want to +include such a loop in your program you probably want to use the :mod:`code` +module instead. + +There are two parts to this job: + +#. Being able to tell if a line of input completes a Python statement: in + short, telling whether to print '``>>>``' or '``...``' next. + +#. Remembering which future statements the user has entered, so subsequent + input can be compiled with these in effect. + +The :mod:`codeop` module provides a way of doing each of these things, and a way +of doing them both. + +To do just the former: + + +.. function:: compile_command(source[, filename[, symbol]]) + + Tries to compile *source*, which should be a string of Python code and return a + code object if *source* is valid Python code. In that case, the filename + attribute of the code object will be *filename*, which defaults to + ``''``. Returns ``None`` if *source* is *not* valid Python code, but is a + prefix of valid Python code. + + If there is a problem with *source*, an exception will be raised. + :exc:`SyntaxError` is raised if there is invalid Python syntax, and + :exc:`OverflowError` or :exc:`ValueError` if there is an invalid literal. + + The *symbol* argument determines whether *source* is compiled as a statement + (``'single'``, the default) or as an expression (``'eval'``). Any other value + will cause :exc:`ValueError` to be raised. + + **Caveat:** It is possible (but not likely) that the parser stops parsing with a + successful outcome before reaching the end of the source; in this case, trailing + symbols may be ignored instead of causing an error. For example, a backslash + followed by two newlines may be followed by arbitrary garbage. This will be + fixed once the API for the parser is better. + + +.. class:: Compile() + + Instances of this class have :meth:`__call__` methods identical in signature to + the built-in function :func:`compile`, but with the difference that if the + instance compiles program text containing a :mod:`__future__` statement, the + instance 'remembers' and compiles all subsequent program texts with the + statement in force. + + +.. class:: CommandCompiler() + + Instances of this class have :meth:`__call__` methods identical in signature to + :func:`compile_command`; the difference is that if the instance compiles program + text containing a ``__future__`` statement, the instance 'remembers' and + compiles all subsequent program texts with the statement in force. + +A note on version compatibility: the :class:`Compile` and +:class:`CommandCompiler` are new in Python 2.2. If you want to enable the +future-tracking features of 2.2 but also retain compatibility with 2.1 and +earlier versions of Python you can either write :: + + try: + from codeop import CommandCompiler + compile_command = CommandCompiler() + del CommandCompiler + except ImportError: + from codeop import compile_command + +which is a low-impact change, but introduces possibly unwanted global state into +your program, or you can write:: + + try: + from codeop import CommandCompiler + except ImportError: + def CommandCompiler(): + from codeop import compile_command + return compile_command + +and then call ``CommandCompiler`` every time you need a fresh compiler object. + diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst new file mode 100644 index 0000000..c2c9262 --- /dev/null +++ b/Doc/library/collections.rst @@ -0,0 +1,414 @@ + +:mod:`collections` --- High-performance container datatypes +=========================================================== + +.. module:: collections + :synopsis: High-performance datatypes +.. moduleauthor:: Raymond Hettinger +.. sectionauthor:: Raymond Hettinger + + +.. versionadded:: 2.4 + +This module implements high-performance container datatypes. Currently, +there are two datatypes, :class:`deque` and :class:`defaultdict`, and +one datatype factory function, :func:`NamedTuple`. Python already +includes built-in containers, :class:`dict`, :class:`list`, +:class:`set`, and :class:`tuple`. In addition, the optional :mod:`bsddb` +module has a :meth:`bsddb.btopen` method that can be used to create in-memory +or file based ordered dictionaries with string keys. + +Future editions of the standard library may include balanced trees and +ordered dictionaries. + +.. versionchanged:: 2.5 + Added :class:`defaultdict`. + +.. versionchanged:: 2.6 + Added :class:`NamedTuple`. + + +.. _deque-objects: + +:class:`deque` objects +---------------------- + + +.. class:: deque([iterable]) + + Returns a new deque object initialized left-to-right (using :meth:`append`) with + data from *iterable*. If *iterable* is not specified, the new deque is empty. + + Deques are a generalization of stacks and queues (the name is pronounced "deck" + and is short for "double-ended queue"). Deques support thread-safe, memory + efficient appends and pops from either side of the deque with approximately the + same O(1) performance in either direction. + + Though :class:`list` objects support similar operations, they are optimized for + fast fixed-length operations and incur O(n) memory movement costs for + ``pop(0)`` and ``insert(0, v)`` operations which change both the size and + position of the underlying data representation. + + .. versionadded:: 2.4 + +Deque objects support the following methods: + + +.. method:: deque.append(x) + + Add *x* to the right side of the deque. + + +.. method:: deque.appendleft(x) + + Add *x* to the left side of the deque. + + +.. method:: deque.clear() + + Remove all elements from the deque leaving it with length 0. + + +.. method:: deque.extend(iterable) + + Extend the right side of the deque by appending elements from the iterable + argument. + + +.. method:: deque.extendleft(iterable) + + Extend the left side of the deque by appending elements from *iterable*. Note, + the series of left appends results in reversing the order of elements in the + iterable argument. + + +.. method:: deque.pop() + + Remove and return an element from the right side of the deque. If no elements + are present, raises an :exc:`IndexError`. + + +.. method:: deque.popleft() + + Remove and return an element from the left side of the deque. If no elements are + present, raises an :exc:`IndexError`. + + +.. method:: deque.remove(value) + + Removed the first occurrence of *value*. If not found, raises a + :exc:`ValueError`. + + .. versionadded:: 2.5 + + +.. method:: deque.rotate(n) + + Rotate the deque *n* steps to the right. If *n* is negative, rotate to the + left. Rotating one step to the right is equivalent to: + ``d.appendleft(d.pop())``. + +In addition to the above, deques support iteration, pickling, ``len(d)``, +``reversed(d)``, ``copy.copy(d)``, ``copy.deepcopy(d)``, membership testing with +the :keyword:`in` operator, and subscript references such as ``d[-1]``. + +Example:: + + >>> from collections import deque + >>> d = deque('ghi') # make a new deque with three items + >>> for elem in d: # iterate over the deque's elements + ... print elem.upper() + G + H + I + + >>> d.append('j') # add a new entry to the right side + >>> d.appendleft('f') # add a new entry to the left side + >>> d # show the representation of the deque + deque(['f', 'g', 'h', 'i', 'j']) + + >>> d.pop() # return and remove the rightmost item + 'j' + >>> d.popleft() # return and remove the leftmost item + 'f' + >>> list(d) # list the contents of the deque + ['g', 'h', 'i'] + >>> d[0] # peek at leftmost item + 'g' + >>> d[-1] # peek at rightmost item + 'i' + + >>> list(reversed(d)) # list the contents of a deque in reverse + ['i', 'h', 'g'] + >>> 'h' in d # search the deque + True + >>> d.extend('jkl') # add multiple elements at once + >>> d + deque(['g', 'h', 'i', 'j', 'k', 'l']) + >>> d.rotate(1) # right rotation + >>> d + deque(['l', 'g', 'h', 'i', 'j', 'k']) + >>> d.rotate(-1) # left rotation + >>> d + deque(['g', 'h', 'i', 'j', 'k', 'l']) + + >>> deque(reversed(d)) # make a new deque in reverse order + deque(['l', 'k', 'j', 'i', 'h', 'g']) + >>> d.clear() # empty the deque + >>> d.pop() # cannot pop from an empty deque + Traceback (most recent call last): + File "", line 1, in -toplevel- + d.pop() + IndexError: pop from an empty deque + + >>> d.extendleft('abc') # extendleft() reverses the input order + >>> d + deque(['c', 'b', 'a']) + + +.. _deque-recipes: + +Recipes +^^^^^^^ + +This section shows various approaches to working with deques. + +The :meth:`rotate` method provides a way to implement :class:`deque` slicing and +deletion. For example, a pure python implementation of ``del d[n]`` relies on +the :meth:`rotate` method to position elements to be popped:: + + def delete_nth(d, n): + d.rotate(-n) + d.popleft() + d.rotate(n) + +To implement :class:`deque` slicing, use a similar approach applying +:meth:`rotate` to bring a target element to the left side of the deque. Remove +old entries with :meth:`popleft`, add new entries with :meth:`extend`, and then +reverse the rotation. + +With minor variations on that approach, it is easy to implement Forth style +stack manipulations such as ``dup``, ``drop``, ``swap``, ``over``, ``pick``, +``rot``, and ``roll``. + +A roundrobin task server can be built from a :class:`deque` using +:meth:`popleft` to select the current task and :meth:`append` to add it back to +the tasklist if the input stream is not exhausted:: + + >>> def roundrobin(*iterables): + ... pending = deque(iter(i) for i in iterables) + ... while pending: + ... task = pending.popleft() + ... try: + ... yield next(task) + ... except StopIteration: + ... continue + ... pending.append(task) + ... + >>> for value in roundrobin('abc', 'd', 'efgh'): + ... print value + + a + d + e + b + f + c + g + h + + +Multi-pass data reduction algorithms can be succinctly expressed and efficiently +coded by extracting elements with multiple calls to :meth:`popleft`, applying +the reduction function, and calling :meth:`append` to add the result back to the +queue. + +For example, building a balanced binary tree of nested lists entails reducing +two adjacent nodes into one by grouping them in a list:: + + >>> def maketree(iterable): + ... d = deque(iterable) + ... while len(d) > 1: + ... pair = [d.popleft(), d.popleft()] + ... d.append(pair) + ... return list(d) + ... + >>> print maketree('abcdefgh') + [[[['a', 'b'], ['c', 'd']], [['e', 'f'], ['g', 'h']]]] + + + +.. _defaultdict-objects: + +:class:`defaultdict` objects +---------------------------- + + +.. class:: defaultdict([default_factory[, ...]]) + + Returns a new dictionary-like object. :class:`defaultdict` is a subclass of the + builtin :class:`dict` class. It overrides one method and adds one writable + instance variable. The remaining functionality is the same as for the + :class:`dict` class and is not documented here. + + The first argument provides the initial value for the :attr:`default_factory` + attribute; it defaults to ``None``. All remaining arguments are treated the same + as if they were passed to the :class:`dict` constructor, including keyword + arguments. + + .. versionadded:: 2.5 + +:class:`defaultdict` objects support the following method in addition to the +standard :class:`dict` operations: + + +.. method:: defaultdict.__missing__(key) + + If the :attr:`default_factory` attribute is ``None``, this raises an + :exc:`KeyError` exception with the *key* as argument. + + If :attr:`default_factory` is not ``None``, it is called without arguments to + provide a default value for the given *key*, this value is inserted in the + dictionary for the *key*, and returned. + + If calling :attr:`default_factory` raises an exception this exception is + propagated unchanged. + + This method is called by the :meth:`__getitem__` method of the :class:`dict` + class when the requested key is not found; whatever it returns or raises is then + returned or raised by :meth:`__getitem__`. + +:class:`defaultdict` objects support the following instance variable: + + +.. attribute:: defaultdict.default_factory + + This attribute is used by the :meth:`__missing__` method; it is initialized from + the first argument to the constructor, if present, or to ``None``, if absent. + + +.. _defaultdict-examples: + +:class:`defaultdict` Examples +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Using :class:`list` as the :attr:`default_factory`, it is easy to group a +sequence of key-value pairs into a dictionary of lists:: + + >>> s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)] + >>> d = defaultdict(list) + >>> for k, v in s: + ... d[k].append(v) + ... + >>> d.items() + [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] + +When each key is encountered for the first time, it is not already in the +mapping; so an entry is automatically created using the :attr:`default_factory` +function which returns an empty :class:`list`. The :meth:`list.append` +operation then attaches the value to the new list. When keys are encountered +again, the look-up proceeds normally (returning the list for that key) and the +:meth:`list.append` operation adds another value to the list. This technique is +simpler and faster than an equivalent technique using :meth:`dict.setdefault`:: + + >>> d = {} + >>> for k, v in s: + ... d.setdefault(k, []).append(v) + ... + >>> d.items() + [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] + +Setting the :attr:`default_factory` to :class:`int` makes the +:class:`defaultdict` useful for counting (like a bag or multiset in other +languages):: + + >>> s = 'mississippi' + >>> d = defaultdict(int) + >>> for k in s: + ... d[k] += 1 + ... + >>> d.items() + [('i', 4), ('p', 2), ('s', 4), ('m', 1)] + +When a letter is first encountered, it is missing from the mapping, so the +:attr:`default_factory` function calls :func:`int` to supply a default count of +zero. The increment operation then builds up the count for each letter. + +The function :func:`int` which always returns zero is just a special case of +constant functions. A faster and more flexible way to create constant functions +is to use a lambda function which can supply any constant value (not just +zero):: + + >>> def constant_factory(value): + ... return lambda: value + >>> d = defaultdict(constant_factory('')) + >>> d.update(name='John', action='ran') + >>> '%(name)s %(action)s to %(object)s' % d + 'John ran to ' + +Setting the :attr:`default_factory` to :class:`set` makes the +:class:`defaultdict` useful for building a dictionary of sets:: + + >>> s = [('red', 1), ('blue', 2), ('red', 3), ('blue', 4), ('red', 1), ('blue', 4)] + >>> d = defaultdict(set) + >>> for k, v in s: + ... d[k].add(v) + ... + >>> d.items() + [('blue', set([2, 4])), ('red', set([1, 3]))] + + +.. _named-tuple-factory: + +:func:`NamedTuple` datatype factory function +-------------------------------------------- + + +.. function:: NamedTuple(typename, fieldnames) + + Returns a new tuple subclass named *typename*. The new subclass is used to + create tuple-like objects that have fields accessable by attribute lookup as + well as being indexable and iterable. Instances of the subclass also have a + helpful docstring (with typename and fieldnames) and a helpful :meth:`__repr__` + method which lists the tuple contents in a ``name=value`` format. + + .. versionadded:: 2.6 + + The *fieldnames* are specified in a single string and are separated by spaces. + Any valid Python identifier may be used for a field name. + + Example:: + + >>> Point = NamedTuple('Point', 'x y') + >>> Point.__doc__ # docstring for the new datatype + 'Point(x, y)' + >>> p = Point(11, y=22) # instantiate with positional or keyword arguments + >>> p[0] + p[1] # works just like the tuple (11, 22) + 33 + >>> x, y = p # unpacks just like a tuple + >>> x, y + (11, 22) + >>> p.x + p.y # fields also accessable by name + 33 + >>> p # readable __repr__ with name=value style + Point(x=11, y=22) + + The use cases are the same as those for tuples. The named factories assign + meaning to each tuple position and allow for more readable, self-documenting + code. Named tuples can also be used to assign field names to tuples returned + by the :mod:`csv` or :mod:`sqlite3` modules. For example:: + + from itertools import starmap + import csv + EmployeeRecord = NamedTuple('EmployeeRecord', 'name age title department paygrade') + for record in starmap(EmployeeRecord, csv.reader(open("employees.csv", "rb"))): + print record + + To cast an individual record stored as :class:`list`, :class:`tuple`, or some + other iterable type, use the star-operator to unpack the values:: + + >>> Color = NamedTuple('Color', 'name code') + >>> m = dict(red=1, green=2, blue=3) + >>> print Color(*m.popitem()) + Color(name='blue', code=3) + diff --git a/Doc/library/colorpicker.rst b/Doc/library/colorpicker.rst new file mode 100644 index 0000000..4244104 --- /dev/null +++ b/Doc/library/colorpicker.rst @@ -0,0 +1,23 @@ + +:mod:`ColorPicker` --- Color selection dialog +============================================= + +.. module:: ColorPicker + :platform: Mac + :synopsis: Interface to the standard color selection dialog. +.. moduleauthor:: Just van Rossum +.. sectionauthor:: Fred L. Drake, Jr. + + +The :mod:`ColorPicker` module provides access to the standard color picker +dialog. + + +.. function:: GetColor(prompt, rgb) + + Show a standard color selection dialog and allow the user to select a color. + The user is given instruction by the *prompt* string, and the default color is + set to *rgb*. *rgb* must be a tuple giving the red, green, and blue components + of the color. :func:`GetColor` returns a tuple giving the user's selected color + and a flag indicating whether they accepted the selection of cancelled. + diff --git a/Doc/library/colorsys.rst b/Doc/library/colorsys.rst new file mode 100644 index 0000000..2e7f3b7 --- /dev/null +++ b/Doc/library/colorsys.rst @@ -0,0 +1,60 @@ + +:mod:`colorsys` --- Conversions between color systems +===================================================== + +.. module:: colorsys + :synopsis: Conversion functions between RGB and other color systems. +.. sectionauthor:: David Ascher + + +The :mod:`colorsys` module defines bidirectional conversions of color values +between colors expressed in the RGB (Red Green Blue) color space used in +computer monitors and three other coordinate systems: YIQ, HLS (Hue Lightness +Saturation) and HSV (Hue Saturation Value). Coordinates in all of these color +spaces are floating point values. In the YIQ space, the Y coordinate is between +0 and 1, but the I and Q coordinates can be positive or negative. In all other +spaces, the coordinates are all between 0 and 1. + +More information about color spaces can be found at +http://www.poynton.com/ColorFAQ.html. + +The :mod:`colorsys` module defines the following functions: + + +.. function:: rgb_to_yiq(r, g, b) + + Convert the color from RGB coordinates to YIQ coordinates. + + +.. function:: yiq_to_rgb(y, i, q) + + Convert the color from YIQ coordinates to RGB coordinates. + + +.. function:: rgb_to_hls(r, g, b) + + Convert the color from RGB coordinates to HLS coordinates. + + +.. function:: hls_to_rgb(h, l, s) + + Convert the color from HLS coordinates to RGB coordinates. + + +.. function:: rgb_to_hsv(r, g, b) + + Convert the color from RGB coordinates to HSV coordinates. + + +.. function:: hsv_to_rgb(h, s, v) + + Convert the color from HSV coordinates to RGB coordinates. + +Example:: + + >>> import colorsys + >>> colorsys.rgb_to_hsv(.3, .4, .2) + (0.25, 0.5, 0.4) + >>> colorsys.hsv_to_rgb(0.25, 0.5, 0.4) + (0.3, 0.4, 0.2) + diff --git a/Doc/library/commands.rst b/Doc/library/commands.rst new file mode 100644 index 0000000..79e3d73 --- /dev/null +++ b/Doc/library/commands.rst @@ -0,0 +1,53 @@ + +:mod:`commands` --- Utilities for running commands +================================================== + +.. module:: commands + :platform: Unix + :synopsis: Utility functions for running external commands. +.. sectionauthor:: Sue Williams + + +The :mod:`commands` module contains wrapper functions for :func:`os.popen` which +take a system command as a string and return any output generated by the command +and, optionally, the exit status. + +The :mod:`subprocess` module provides more powerful facilities for spawning new +processes and retrieving their results. Using the :mod:`subprocess` module is +preferable to using the :mod:`commands` module. + +The :mod:`commands` module defines the following functions: + + +.. function:: getstatusoutput(cmd) + + Execute the string *cmd* in a shell with :func:`os.popen` and return a 2-tuple + ``(status, output)``. *cmd* is actually run as ``{ cmd ; } 2>&1``, so that the + returned output will contain output or error messages. A trailing newline is + stripped from the output. The exit status for the command can be interpreted + according to the rules for the C function :cfunc:`wait`. + + +.. function:: getoutput(cmd) + + Like :func:`getstatusoutput`, except the exit status is ignored and the return + value is a string containing the command's output. + +Example:: + + >>> import commands + >>> commands.getstatusoutput('ls /bin/ls') + (0, '/bin/ls') + >>> commands.getstatusoutput('cat /bin/junk') + (256, 'cat: /bin/junk: No such file or directory') + >>> commands.getstatusoutput('/bin/junk') + (256, 'sh: /bin/junk: not found') + >>> commands.getoutput('ls /bin/ls') + '/bin/ls' + + +.. seealso:: + + Module :mod:`subprocess` + Module for spawning and managing subprocesses. + diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst new file mode 100644 index 0000000..d62b785 --- /dev/null +++ b/Doc/library/compileall.rst @@ -0,0 +1,57 @@ + +:mod:`compileall` --- Byte-compile Python libraries +=================================================== + +.. module:: compileall + :synopsis: Tools for byte-compiling all Python source files in a directory tree. + + +This module provides some utility functions to support installing Python +libraries. These functions compile Python source files in a directory tree, +allowing users without permission to write to the libraries to take advantage of +cached byte-code files. + +The source file for this module may also be used as a script to compile Python +sources in directories named on the command line or in ``sys.path``. + + +.. function:: compile_dir(dir[, maxlevels[, ddir[, force[, rx[, quiet]]]]]) + + Recursively descend the directory tree named by *dir*, compiling all :file:`.py` + files along the way. The *maxlevels* parameter is used to limit the depth of + the recursion; it defaults to ``10``. If *ddir* is given, it is used as the + base path from which the filenames used in error messages will be generated. + If *force* is true, modules are re-compiled even if the timestamps are up to + date. + + If *rx* is given, it specifies a regular expression of file names to exclude + from the search; that expression is searched for in the full path. + + If *quiet* is true, nothing is printed to the standard output in normal + operation. + + +.. function:: compile_path([skip_curdir[, maxlevels[, force]]]) + + Byte-compile all the :file:`.py` files found along ``sys.path``. If + *skip_curdir* is true (the default), the current directory is not included in + the search. The *maxlevels* and *force* parameters default to ``0`` and are + passed to the :func:`compile_dir` function. + +To force a recompile of all the :file:`.py` files in the :file:`Lib/` +subdirectory and all its subdirectories:: + + import compileall + + compileall.compile_dir('Lib/', force=True) + + # Perform same compilation, excluding files in .svn directories. + import re + compileall.compile_dir('Lib/', rx=re.compile('/[.]svn'), force=True) + + +.. seealso:: + + Module :mod:`py_compile` + Byte-compile a single source file. + diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst new file mode 100644 index 0000000..dd91d59 --- /dev/null +++ b/Doc/library/configparser.rst @@ -0,0 +1,361 @@ + +:mod:`ConfigParser` --- Configuration file parser +================================================= + +.. module:: ConfigParser + :synopsis: Configuration file parser. +.. moduleauthor:: Ken Manheimer +.. moduleauthor:: Barry Warsaw +.. moduleauthor:: Eric S. Raymond +.. sectionauthor:: Christopher G. Petrilli + + +.. index:: + pair: .ini; file + pair: configuration; file + single: ini file + single: Windows ini file + +This module defines the class :class:`ConfigParser`. The :class:`ConfigParser` +class implements a basic configuration file parser language which provides a +structure similar to what you would find on Microsoft Windows INI files. You +can use this to write Python programs which can be customized by end users +easily. + +.. warning:: + + This library does *not* interpret or write the value-type prefixes used in the + Windows Registry extended version of INI syntax. + +The configuration file consists of sections, led by a ``[section]`` header and +followed by ``name: value`` entries, with continuations in the style of +:rfc:`822`; ``name=value`` is also accepted. Note that leading whitespace is +removed from values. The optional values can contain format strings which refer +to other values in the same section, or values in a special ``DEFAULT`` section. +Additional defaults can be provided on initialization and retrieval. Lines +beginning with ``'#'`` or ``';'`` are ignored and may be used to provide +comments. + +For example:: + + [My Section] + foodir: %(dir)s/whatever + dir=frob + +would resolve the ``%(dir)s`` to the value of ``dir`` (``frob`` in this case). +All reference expansions are done on demand. + +Default values can be specified by passing them into the :class:`ConfigParser` +constructor as a dictionary. Additional defaults may be passed into the +:meth:`get` method which will override all others. + +Sections are normally stored in a builtin dictionary. An alternative dictionary +type can be passed to the :class:`ConfigParser` constructor. For example, if a +dictionary type is passed that sorts its keys, the sections will be sorted on +write-back, as will be the keys within each section. + + +.. class:: RawConfigParser([defaults[, dict_type]]) + + The basic configuration object. When *defaults* is given, it is initialized + into the dictionary of intrinsic defaults. When *dict_type* is given, it will + be used to create the dictionary objects for the list of sections, for the + options within a section, and for the default values. This class does not + support the magical interpolation behavior. + + .. versionadded:: 2.3 + + .. versionchanged:: 2.6 + *dict_type* was added. + + +.. class:: ConfigParser([defaults]) + + Derived class of :class:`RawConfigParser` that implements the magical + interpolation feature and adds optional arguments to the :meth:`get` and + :meth:`items` methods. The values in *defaults* must be appropriate for the + ``%()s`` string interpolation. Note that *__name__* is an intrinsic default; + its value is the section name, and will override any value provided in + *defaults*. + + All option names used in interpolation will be passed through the + :meth:`optionxform` method just like any other option name reference. For + example, using the default implementation of :meth:`optionxform` (which converts + option names to lower case), the values ``foo %(bar)s`` and ``foo %(BAR)s`` are + equivalent. + + +.. class:: SafeConfigParser([defaults]) + + Derived class of :class:`ConfigParser` that implements a more-sane variant of + the magical interpolation feature. This implementation is more predictable as + well. New applications should prefer this version if they don't need to be + compatible with older versions of Python. + + .. % XXX Need to explain what's safer/more predictable about it. + + .. versionadded:: 2.3 + + +.. exception:: NoSectionError + + Exception raised when a specified section is not found. + + +.. exception:: DuplicateSectionError + + Exception raised if :meth:`add_section` is called with the name of a section + that is already present. + + +.. exception:: NoOptionError + + Exception raised when a specified option is not found in the specified section. + + +.. exception:: InterpolationError + + Base class for exceptions raised when problems occur performing string + interpolation. + + +.. exception:: InterpolationDepthError + + Exception raised when string interpolation cannot be completed because the + number of iterations exceeds :const:`MAX_INTERPOLATION_DEPTH`. Subclass of + :exc:`InterpolationError`. + + +.. exception:: InterpolationMissingOptionError + + Exception raised when an option referenced from a value does not exist. Subclass + of :exc:`InterpolationError`. + + .. versionadded:: 2.3 + + +.. exception:: InterpolationSyntaxError + + Exception raised when the source text into which substitutions are made does not + conform to the required syntax. Subclass of :exc:`InterpolationError`. + + .. versionadded:: 2.3 + + +.. exception:: MissingSectionHeaderError + + Exception raised when attempting to parse a file which has no section headers. + + +.. exception:: ParsingError + + Exception raised when errors occur attempting to parse a file. + + +.. data:: MAX_INTERPOLATION_DEPTH + + The maximum depth for recursive interpolation for :meth:`get` when the *raw* + parameter is false. This is relevant only for the :class:`ConfigParser` class. + + +.. seealso:: + + Module :mod:`shlex` + Support for a creating Unix shell-like mini-languages which can be used as an + alternate format for application configuration files. + + +.. _rawconfigparser-objects: + +RawConfigParser Objects +----------------------- + +:class:`RawConfigParser` instances have the following methods: + + +.. method:: RawConfigParser.defaults() + + Return a dictionary containing the instance-wide defaults. + + +.. method:: RawConfigParser.sections() + + Return a list of the sections available; ``DEFAULT`` is not included in the + list. + + +.. method:: RawConfigParser.add_section(section) + + Add a section named *section* to the instance. If a section by the given name + already exists, :exc:`DuplicateSectionError` is raised. + + +.. method:: RawConfigParser.has_section(section) + + Indicates whether the named section is present in the configuration. The + ``DEFAULT`` section is not acknowledged. + + +.. method:: RawConfigParser.options(section) + + Returns a list of options available in the specified *section*. + + +.. method:: RawConfigParser.has_option(section, option) + + If the given section exists, and contains the given option, return + :const:`True`; otherwise return :const:`False`. + + .. versionadded:: 1.6 + + +.. method:: RawConfigParser.read(filenames) + + Attempt to read and parse a list of filenames, returning a list of filenames + which were successfully parsed. If *filenames* is a string or Unicode string, + it is treated as a single filename. If a file named in *filenames* cannot be + opened, that file will be ignored. This is designed so that you can specify a + list of potential configuration file locations (for example, the current + directory, the user's home directory, and some system-wide directory), and all + existing configuration files in the list will be read. If none of the named + files exist, the :class:`ConfigParser` instance will contain an empty dataset. + An application which requires initial values to be loaded from a file should + load the required file or files using :meth:`readfp` before calling :meth:`read` + for any optional files:: + + import ConfigParser, os + + config = ConfigParser.ConfigParser() + config.readfp(open('defaults.cfg')) + config.read(['site.cfg', os.path.expanduser('~/.myapp.cfg')]) + + .. versionchanged:: 2.4 + Returns list of successfully parsed filenames. + + +.. method:: RawConfigParser.readfp(fp[, filename]) + + Read and parse configuration data from the file or file-like object in *fp* + (only the :meth:`readline` method is used). If *filename* is omitted and *fp* + has a :attr:`name` attribute, that is used for *filename*; the default is + ````. + + +.. method:: RawConfigParser.get(section, option) + + Get an *option* value for the named *section*. + + +.. method:: RawConfigParser.getint(section, option) + + A convenience method which coerces the *option* in the specified *section* to an + integer. + + +.. method:: RawConfigParser.getfloat(section, option) + + A convenience method which coerces the *option* in the specified *section* to a + floating point number. + + +.. method:: RawConfigParser.getboolean(section, option) + + A convenience method which coerces the *option* in the specified *section* to a + Boolean value. Note that the accepted values for the option are ``"1"``, + ``"yes"``, ``"true"``, and ``"on"``, which cause this method to return ``True``, + and ``"0"``, ``"no"``, ``"false"``, and ``"off"``, which cause it to return + ``False``. These string values are checked in a case-insensitive manner. Any + other value will cause it to raise :exc:`ValueError`. + + +.. method:: RawConfigParser.items(section) + + Return a list of ``(name, value)`` pairs for each option in the given *section*. + + +.. method:: RawConfigParser.set(section, option, value) + + If the given section exists, set the given option to the specified value; + otherwise raise :exc:`NoSectionError`. While it is possible to use + :class:`RawConfigParser` (or :class:`ConfigParser` with *raw* parameters set to + true) for *internal* storage of non-string values, full functionality (including + interpolation and output to files) can only be achieved using string values. + + .. versionadded:: 1.6 + + +.. method:: RawConfigParser.write(fileobject) + + Write a representation of the configuration to the specified file object. This + representation can be parsed by a future :meth:`read` call. + + .. versionadded:: 1.6 + + +.. method:: RawConfigParser.remove_option(section, option) + + Remove the specified *option* from the specified *section*. If the section does + not exist, raise :exc:`NoSectionError`. If the option existed to be removed, + return :const:`True`; otherwise return :const:`False`. + + .. versionadded:: 1.6 + + +.. method:: RawConfigParser.remove_section(section) + + Remove the specified *section* from the configuration. If the section in fact + existed, return ``True``. Otherwise return ``False``. + + +.. method:: RawConfigParser.optionxform(option) + + Transforms the option name *option* as found in an input file or as passed in by + client code to the form that should be used in the internal structures. The + default implementation returns a lower-case version of *option*; subclasses may + override this or client code can set an attribute of this name on instances to + affect this behavior. Setting this to :func:`str`, for example, would make + option names case sensitive. + + +.. _configparser-objects: + +ConfigParser Objects +-------------------- + +The :class:`ConfigParser` class extends some methods of the +:class:`RawConfigParser` interface, adding some optional arguments. + + +.. method:: ConfigParser.get(section, option[, raw[, vars]]) + + Get an *option* value for the named *section*. All the ``'%'`` interpolations + are expanded in the return values, based on the defaults passed into the + constructor, as well as the options *vars* provided, unless the *raw* argument + is true. + + +.. method:: ConfigParser.items(section[, raw[, vars]]) + + Return a list of ``(name, value)`` pairs for each option in the given *section*. + Optional arguments have the same meaning as for the :meth:`get` method. + + .. versionadded:: 2.3 + + +.. _safeconfigparser-objects: + +SafeConfigParser Objects +------------------------ + +The :class:`SafeConfigParser` class implements the same extended interface as +:class:`ConfigParser`, with the following addition: + + +.. method:: SafeConfigParser.set(section, option, value) + + If the given section exists, set the given option to the specified value; + otherwise raise :exc:`NoSectionError`. *value* must be a string (:class:`str` + or :class:`unicode`); if not, :exc:`TypeError` is raised. + + .. versionadded:: 2.4 + diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst new file mode 100644 index 0000000..fecd836 --- /dev/null +++ b/Doc/library/constants.rst @@ -0,0 +1,42 @@ + +Built-in Constants +================== + +A small number of constants live in the built-in namespace. They are: + + +.. data:: False + + The false value of the :class:`bool` type. + + .. versionadded:: 2.3 + + +.. data:: True + + The true value of the :class:`bool` type. + + .. versionadded:: 2.3 + + +.. data:: None + + The sole value of :attr:`types.NoneType`. ``None`` is frequently used to + represent the absence of a value, as when default arguments are not passed to a + function. + + +.. data:: NotImplemented + + Special value which can be returned by the "rich comparison" special methods + (:meth:`__eq__`, :meth:`__lt__`, and friends), to indicate that the comparison + is not implemented with respect to the other type. + + +.. data:: Ellipsis + + The same as ``...``. Special value used mostly in conjunction with extended + slicing syntax for user-defined container data types. + + .. % XXX Someone who understands extended slicing should fill in here. + diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst new file mode 100644 index 0000000..fffb99c --- /dev/null +++ b/Doc/library/contextlib.rst @@ -0,0 +1,120 @@ + +:mod:`contextlib` --- Utilities for :keyword:`with`\ -statement contexts. +========================================================================= + +.. module:: contextlib + :synopsis: Utilities for with-statement contexts. + + +.. versionadded:: 2.5 + +This module provides utilities for common tasks involving the :keyword:`with` +statement. For more information see also :ref:`typecontextmanager` and +:ref:`context-managers`. + +Functions provided: + + +.. function:: contextmanager(func) + + This function is a decorator that can be used to define a factory function for + :keyword:`with` statement context managers, without needing to create a class or + separate :meth:`__enter__` and :meth:`__exit__` methods. + + A simple example (this is not recommended as a real way of generating HTML!):: + + from __future__ import with_statement + from contextlib import contextmanager + + @contextmanager + def tag(name): + print "<%s>" % name + yield + print "" % name + + >>> with tag("h1"): + ... print "foo" + ... +

+ foo +