Merged revisions 86134,86315-86316,86390,86424-86425,86428,86550,86561-86562,86564-86565,86705,86708,86713 via svnmerge from

svn+ssh://svn.python.org/python/branches/py3k

........
  r86134 | georg.brandl | 2010-11-03 08:41:00 +0100 (Mi, 03 Nov 2010) | 1 line

  A newline in lineno output breaks pyframe output.
........
  r86315 | georg.brandl | 2010-11-08 12:05:18 +0100 (Mo, 08 Nov 2010) | 1 line

  Fix latex conversion glitch in property/feature descriptions.
........
  r86316 | georg.brandl | 2010-11-08 12:08:35 +0100 (Mo, 08 Nov 2010) | 1 line

  Fix typo.
........
  r86390 | georg.brandl | 2010-11-10 08:57:10 +0100 (Mi, 10 Nov 2010) | 1 line

  Fix typo.
........
  r86424 | georg.brandl | 2010-11-12 07:19:48 +0100 (Fr, 12 Nov 2010) | 1 line

  Build a PDF of the FAQs too.
........
  r86425 | georg.brandl | 2010-11-12 07:20:12 +0100 (Fr, 12 Nov 2010) | 1 line

  #10008: Fix duplicate index entry.
........
  r86428 | georg.brandl | 2010-11-12 09:09:26 +0100 (Fr, 12 Nov 2010) | 1 line

  Fix weird line block in table.
........
  r86550 | georg.brandl | 2010-11-20 11:24:34 +0100 (Sa, 20 Nov 2010) | 1 line

  Fix rst markup errors.
........
  r86561 | georg.brandl | 2010-11-20 12:47:10 +0100 (Sa, 20 Nov 2010) | 1 line

  #10460: Update indent.pro to match PEP 7 better.
........
  r86562 | georg.brandl | 2010-11-20 14:44:41 +0100 (Sa, 20 Nov 2010) | 1 line

  #10439: document PyCodec C APIs.
........
  r86564 | georg.brandl | 2010-11-20 15:08:53 +0100 (Sa, 20 Nov 2010) | 1 line

  #10460: an even better indent.pro.
........
  r86565 | georg.brandl | 2010-11-20 15:16:17 +0100 (Sa, 20 Nov 2010) | 1 line

  socket.gethostbyname(socket.gethostname()) can fail when host name resolution is not set up correctly; do not fail test_socket if this is the case.
........
  r86705 | georg.brandl | 2010-11-23 08:54:19 +0100 (Di, 23 Nov 2010) | 1 line

  #10468: document Unicode exception creation and access functions.
........
  r86708 | georg.brandl | 2010-11-23 09:37:54 +0100 (Di, 23 Nov 2010) | 2 lines

  #10511: clarification of what heaps are; suggested by Johannes Hoff.
........
  r86713 | georg.brandl | 2010-11-23 19:14:57 +0100 (Di, 23 Nov 2010) | 1 line

  assert.h is also included. Thanks to Savio Sena.
........

10 files changed, 259 insertions, 51 deletions

diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst
new file mode 100644
index 0000000..2597d38
--- /dev/null
+++ b/Doc/c-api/codec.rst
@@ -0,0 +1,118 @@
+.. _codec-registry:
+
+Codec registry and support functions
+====================================
+
+.. cfunction:: int PyCodec_Register(PyObject *search_function)
+
+   Register a new codec search function.
+
+   As side effect, this tries to load the :mod:`encodings` package, if not yet
+   done, to make sure that it is always first in the list of search functions.
+
+.. cfunction:: int PyCodec_KnownEncoding(const char *encoding)
+
+   Return ``1`` or ``0`` depending on whether there is a registered codec for
+   the given *encoding*.
+
+.. cfunction:: PyObject* PyCodec_Encode(PyObject *object, const char *encoding, const char *errors)
+
+   Generic codec based encoding API.
+
+   *object* is passed through the encoder function found for the given
+   *encoding* using the error handling method defined by *errors*.  *errors* may
+   be *NULL* to use the default method defined for the codec.  Raises a
+   :exc:`LookupError` if no encoder can be found.
+
+.. cfunction:: PyObject* PyCodec_Decode(PyObject *object, const char *encoding, const char *errors)
+
+   Generic codec based decoding API.
+
+   *object* is passed through the decoder function found for the given
+   *encoding* using the error handling method defined by *errors*.  *errors* may
+   be *NULL* to use the default method defined for the codec.  Raises a
+   :exc:`LookupError` if no encoder can be found.
+
+
+Codec lookup API
+----------------
+
+In the following functions, the *encoding* string is looked up converted to all
+lower-case characters, which makes encodings looked up through this mechanism
+effectively case-insensitive.  If no codec is found, a :exc:`KeyError` is set
+and *NULL* returned.
+
+.. cfunction:: PyObject* PyCodec_Encoder(const char *encoding)
+
+   Get an encoder function for the given *encoding*.
+
+.. cfunction:: PyObject* PyCodec_Decoder(const char *encoding)
+
+   Get a decoder function for the given *encoding*.
+
+.. cfunction:: PyObject* PyCodec_IncrementalEncoder(const char *encoding, const char *errors)
+
+   Get an :class:`IncrementalEncoder` object for the given *encoding*.
+
+.. cfunction:: PyObject* PyCodec_IncrementalDecoder(const char *encoding, const char *errors)
+
+   Get an :class:`IncrementalDecoder` object for the given *encoding*.
+
+.. cfunction:: PyObject* PyCodec_StreamReader(const char *encoding, PyObject *stream, const char *errors)
+
+   Get a :class:`StreamReader` factory function for the given *encoding*.
+
+.. cfunction:: PyObject* PyCodec_StreamWriter(const char *encoding, PyObject *stream, const char *errors)
+
+   Get a :class:`StreamWriter` factory function for the given *encoding*.
+
+
+Registry API for Unicode encoding error handlers
+------------------------------------------------
+
+.. cfunction:: int PyCodec_RegisterError(const char *name, PyObject *error)
+
+   Register the error handling callback function *error* under the given *name*.
+   This callback function will be called by a codec when it encounters
+   unencodable characters/undecodable bytes and *name* is specified as the error
+   parameter in the call to the encode/decode function.
+
+   The callback gets a single argument, an instance of
+   :exc:`UnicodeEncodeError`, :exc:`UnicodeDecodeError` or
+   :exc:`UnicodeTranslateError` that holds information about the problematic
+   sequence of characters or bytes and their offset in the original string (see
+   :ref:`unicodeexceptions` for functions to extract this information).  The
+   callback must either raise the given exception, or return a two-item tuple
+   containing the replacement for the problematic sequence, and an integer
+   giving the offset in the original string at which encoding/decoding should be
+   resumed.
+
+   Return ``0`` on success, ``-1`` on error.
+
+.. cfunction:: PyObject* PyCodec_LookupError(const char *name)
+
+   Lookup the error handling callback function registered under *name*.  As a
+   special case *NULL* can be passed, in which case the error handling callback
+   for "strict" will be returned.
+
+.. cfunction:: PyObject* PyCodec_StrictErrors(PyObject *exc)
+
+   Raise *exc* as an exception.
+
+.. cfunction:: PyObject* PyCodec_IgnoreErrors(PyObject *exc)
+
+   Ignore the unicode error, skipping the faulty input.
+
+.. cfunction:: PyObject* PyCodec_ReplaceErrors(PyObject *exc)
+
+   Replace the unicode encode error with ``?`` or ``U+FFFD``.
+
+.. cfunction:: PyObject* PyCodec_XMLCharRefReplaceErrors(PyObject *exc)
+
+   Replace the unicode encode error with XML character references.
+
+.. cfunction:: PyObject* PyCodec_BackslashReplaceErrors(PyObject *exc)
+
+   Replace the unicode encode error with backslash escapes (``\x``, ``\u`` and
+   ``\U``).
+
diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
index 2214c4d..87891fb 100644
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -446,6 +446,83 @@ Exception Objects
    This steals a reference to *ctx*.
 
 
+.. _unicodeexceptions:
+
+Unicode Exception Objects
+=========================
+
+The following functions are used to create and modify Unicode exceptions from C.
+
+.. cfunction:: PyObject* PyUnicodeDecodeError_Create(const char *encoding, const char *object, Py_ssize_t length, Py_ssize_t start, Py_ssize_t end, const char *reason)
+
+   Create a :class:`UnicodeDecodeError` object with the attributes *encoding*,
+   *object*, *length*, *start*, *end* and *reason*.
+
+.. cfunction:: PyObject* PyUnicodeEncodeError_Create(const char *encoding, const Py_UNICODE *object, Py_ssize_t length, Py_ssize_t start, Py_ssize_t end, const char *reason)
+
+   Create a :class:`UnicodeEncodeError` object with the attributes *encoding*,
+   *object*, *length*, *start*, *end* and *reason*.
+
+.. cfunction:: PyObject* PyUnicodeTranslateError_Create(const Py_UNICODE *object, Py_ssize_t length, Py_ssize_t start, Py_ssize_t end, const char *reason)
+
+   Create a :class:`UnicodeTranslateError` object with the attributes *object*,
+   *length*, *start*, *end* and *reason*.
+
+.. cfunction:: PyObject* PyUnicodeDecodeError_GetEncoding(PyObject *exc)
+               PyObject* PyUnicodeEncodeError_GetEncoding(PyObject *exc)
+
+   Return the *encoding* attribute of the given exception object.
+
+.. cfunction:: PyObject* PyUnicodeDecodeError_GetObject(PyObject *exc)
+               PyObject* PyUnicodeEncodeError_GetObject(PyObject *exc)
+               PyObject* PyUnicodeTranslateError_GetObject(PyObject *exc)
+
+   Return the *object* attribute of the given exception object.
+
+.. cfunction:: int PyUnicodeDecodeError_GetStart(PyObject *exc, Py_ssize_t *start)
+               int PyUnicodeEncodeError_GetStart(PyObject *exc, Py_ssize_t *start)
+               int PyUnicodeTranslateError_GetStart(PyObject *exc, Py_ssize_t *start)
+
+   Get the *start* attribute of the given exception object and place it into
+   *\*start*.  *start* must not be *NULL*.  Return ``0`` on success, ``-1`` on
+   failure.
+
+.. cfunction:: int PyUnicodeDecodeError_SetStart(PyObject *exc, Py_ssize_t start)
+               int PyUnicodeEncodeError_SetStart(PyObject *exc, Py_ssize_t start)
+               int PyUnicodeTranslateError_SetStart(PyObject *exc, Py_ssize_t start)
+
+   Set the *start* attribute of the given exception object to *start*.  Return
+   ``0`` on success, ``-1`` on failure.
+
+.. cfunction:: int PyUnicodeDecodeError_GetEnd(PyObject *exc, Py_ssize_t *end)
+               int PyUnicodeEncodeError_GetEnd(PyObject *exc, Py_ssize_t *end)
+               int PyUnicodeTranslateError_GetEnd(PyObject *exc, Py_ssize_t *end)
+
+   Get the *end* attribute of the given exception object and place it into
+   *\*end*.  *end* must not be *NULL*.  Return ``0`` on success, ``-1`` on
+   failure.
+
+.. cfunction:: int PyUnicodeDecodeError_SetEnd(PyObject *exc, Py_ssize_t end)
+               int PyUnicodeEncodeError_SetEnd(PyObject *exc, Py_ssize_t end)
+               int PyUnicodeTranslateError_SetEnd(PyObject *exc, Py_ssize_t end)
+
+   Set the *end* attribute of the given exception object to *end*.  Return ``0``
+   on success, ``-1`` on failure.
+
+.. cfunction:: PyObject* PyUnicodeDecodeError_GetReason(PyObject *exc)
+               PyObject* PyUnicodeEncodeError_GetReason(PyObject *exc)
+               PyObject* PyUnicodeTranslateError_GetReason(PyObject *exc)
+
+   Return the *reason* attribute of the given exception object.
+
+.. cfunction:: int PyUnicodeDecodeError_SetReason(PyObject *exc, const char *reason)
+               int PyUnicodeEncodeError_SetReason(PyObject *exc, const char *reason)
+               int PyUnicodeTranslateError_SetReason(PyObject *exc, const char *reason)
+
+   Set the *reason* attribute of the given exception object to *reason*.  Return
+   ``0`` on success, ``-1`` on failure.
+
+
 Recursion Control
 =================
 
diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst
index 1708a85..e8b62dc 100644
--- a/Doc/c-api/intro.rst
+++ b/Doc/c-api/intro.rst
@@ -41,8 +41,8 @@ included in your code by the following line::
    #include "Python.h"
 
 This implies inclusion of the following standard headers: ``<stdio.h>``,
-``<string.h>``, ``<errno.h>``, ``<limits.h>``, and ``<stdlib.h>`` (if
-available).
+``<string.h>``, ``<errno.h>``, ``<limits.h>``, ``<assert.h>`` and ``<stdlib.h>``
+(if available).
 
 .. note::
 
diff --git a/Doc/c-api/utilities.rst b/Doc/c-api/utilities.rst
index 6cb4d0d..d4484fb 100644
--- a/Doc/c-api/utilities.rst
+++ b/Doc/c-api/utilities.rst
@@ -18,3 +18,4 @@ and parsing function arguments and constructing Python values from C values.
    arg.rst
    conversion.rst
    reflection.rst
+   codec.rst
diff --git a/Doc/conf.py b/Doc/conf.py
index 6a04a45..5939f7a 100644
--- a/Doc/conf.py
+++ b/Doc/conf.py
@@ -129,6 +129,8 @@ latex_documents = [
      'Python Tutorial', _stdauthor, 'manual'),
     ('using/index', 'using.tex',
      'Python Setup and Usage', _stdauthor, 'manual'),
+    ('faq/index', 'faq.tex',
+     'Python Frequently Asked Questions', _stdauthor, 'manual'),
     ('whatsnew/' + version, 'whatsnew.tex',
      'What\'s New in Python', 'A. M. Kuchling', 'howto'),
 ]
diff --git a/Doc/library/heapq.rst b/Doc/library/heapq.rst
index 78beee9..7735365 100644
--- a/Doc/library/heapq.rst
+++ b/Doc/library/heapq.rst
@@ -11,11 +11,12 @@
 This module provides an implementation of the heap queue algorithm, also known
 as the priority queue algorithm.
 
-Heaps are arrays for which ``heap[k] <= heap[2*k+1]`` and ``heap[k] <=
-heap[2*k+2]`` for all *k*, counting elements from zero.  For the sake of
-comparison, non-existing elements are considered to be infinite.  The
-interesting property of a heap is that ``heap[0]`` is always its smallest
-element.
+Heaps are binary trees for which every parent node has a value less than or
+equal to any of its children.  This implementation uses arrays for which
+``heap[k] <= heap[2*k+1]`` and ``heap[k] <= heap[2*k+2]`` for all *k*, counting
+elements from zero.  For the sake of comparison, non-existing elements are
+considered to be infinite.  The interesting property of a heap is that its
+smallest element is always the root, ``heap[0]``.
 
 The API below differs from textbook heap algorithms in two aspects: (a) We use
 zero-based indexing.  This makes the relationship between the index for a node
diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
index b4854c6..2f2617e 100644
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -67,7 +67,6 @@ Iterator                                         Arguments                  Resu
 :func:`permutations`                             p[, r]                     r-length tuples, all possible orderings, no repeated elements
 :func:`combinations`                             p, r                       r-length tuples, in sorted order, no repeated elements
 :func:`combinations_with_replacement`            p, r                       r-length tuples, in sorted order, with repeated elements
-|
 ``product('ABCD', repeat=2)``                                               ``AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD``
 ``permutations('ABCD', 2)``                                                 ``AB AC AD BA BC BD CA CB CD DA DB DC``
 ``combinations('ABCD', 2)``                                                 ``AB AC AD BC BD CD``
diff --git a/Doc/library/xml.sax.handler.rst b/Doc/library/xml.sax.handler.rst
index 0fa238d..1a6b391 100644
--- a/Doc/library/xml.sax.handler.rst
+++ b/Doc/library/xml.sax.handler.rst
@@ -49,52 +49,57 @@ for the feature and property names.
 
 .. data:: feature_namespaces
 
-   Value: ``"http://xml.org/sax/features/namespaces"`` ---  true: Perform Namespace
-   processing. ---  false: Optionally do not perform Namespace processing (implies
-   namespace-prefixes; default). ---  access: (parsing) read-only; (not parsing)
-   read/write
+   | value: ``"http://xml.org/sax/features/namespaces"``
+   | true: Perform Namespace processing.
+   | false: Optionally do not perform Namespace processing (implies
+     namespace-prefixes; default).
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: feature_namespace_prefixes
 
-   Value: ``"http://xml.org/sax/features/namespace-prefixes"`` --- true: Report
-   the original prefixed names and attributes used for Namespace
-   declarations. --- false: Do not report attributes used for Namespace
-   declarations, and optionally do not report original prefixed names
-   (default). --- access: (parsing) read-only; (not parsing) read/write
+   | value: ``"http://xml.org/sax/features/namespace-prefixes"``
+   | true: Report the original prefixed names and attributes used for Namespace
+     declarations.
+   | false: Do not report attributes used for Namespace declarations, and
+     optionally do not report original prefixed names (default).
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: feature_string_interning
 
-   Value: ``"http://xml.org/sax/features/string-interning"`` ---  true: All element
-   names, prefixes, attribute names, Namespace URIs, and local names are interned
-   using the built-in intern function. ---  false: Names are not necessarily
-   interned, although they may be (default). ---  access: (parsing) read-only; (not
-   parsing) read/write
+   | value: ``"http://xml.org/sax/features/string-interning"``
+   | true: All element names, prefixes, attribute names, Namespace URIs, and
+     local names are interned using the built-in intern function.
+   | false: Names are not necessarily interned, although they may be (default).
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: feature_validation
 
-   Value: ``"http://xml.org/sax/features/validation"`` --- true: Report all
-   validation errors (implies external-general-entities and
-   external-parameter-entities). --- false: Do not report validation errors. ---
-   access: (parsing) read-only; (not parsing) read/write
+   | value: ``"http://xml.org/sax/features/validation"``
+   | true: Report all validation errors (implies external-general-entities and
+     external-parameter-entities).
+   | false: Do not report validation errors.
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: feature_external_ges
 
-   Value: ``"http://xml.org/sax/features/external-general-entities"`` ---  true:
-   Include all external general (text) entities. ---  false: Do not include
-   external general entities. ---  access: (parsing) read-only; (not parsing)
-   read/write
+   | value: ``"http://xml.org/sax/features/external-general-entities"``
+   | true: Include all external general (text) entities.
+   | false: Do not include external general entities.
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: feature_external_pes
 
-   Value: ``"http://xml.org/sax/features/external-parameter-entities"`` ---  true:
-   Include all external parameter entities, including the external DTD subset. ---
-   false: Do not include any external parameter entities, even the external DTD
-   subset. ---  access: (parsing) read-only; (not parsing) read/write
+   | value: ``"http://xml.org/sax/features/external-parameter-entities"``
+   | true: Include all external parameter entities, including the external DTD
+     subset.
+   | false: Do not include any external parameter entities, even the external
+     DTD subset.
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: all_features
@@ -104,34 +109,38 @@ for the feature and property names.
 
 .. data:: property_lexical_handler
 
-   Value: ``"http://xml.org/sax/properties/lexical-handler"`` ---  data type:
-   xml.sax.sax2lib.LexicalHandler (not supported in Python 2) ---  description: An
-   optional extension handler for lexical events like comments. ---  access:
-   read/write
+   | value: ``"http://xml.org/sax/properties/lexical-handler"``
+   | data type: xml.sax.sax2lib.LexicalHandler (not supported in Python 2)
+   | description: An optional extension handler for lexical events like
+     comments.
+   | access: read/write
 
 
 .. data:: property_declaration_handler
 
-   Value: ``"http://xml.org/sax/properties/declaration-handler"`` ---  data type:
-   xml.sax.sax2lib.DeclHandler (not supported in Python 2) ---  description: An
-   optional extension handler for DTD-related events other than notations and
-   unparsed entities. ---  access: read/write
+   | value: ``"http://xml.org/sax/properties/declaration-handler"``
+   | data type: xml.sax.sax2lib.DeclHandler (not supported in Python 2)
+   | description: An optional extension handler for DTD-related events other
+     than notations and unparsed entities.
+   | access: read/write
 
 
 .. data:: property_dom_node
 
-   Value: ``"http://xml.org/sax/properties/dom-node"`` ---  data type:
-   org.w3c.dom.Node (not supported in Python 2)  ---  description: When parsing,
-   the current DOM node being visited if this is a DOM iterator; when not parsing,
-   the root DOM node for iteration. ---  access: (parsing) read-only; (not parsing)
-   read/write
+   | value: ``"http://xml.org/sax/properties/dom-node"``
+   | data type: org.w3c.dom.Node (not supported in Python 2)
+   | description: When parsing, the current DOM node being visited if this is
+     a DOM iterator; when not parsing, the root DOM node for iteration.
+   | access: (parsing) read-only; (not parsing) read/write
 
 
 .. data:: property_xml_string
 
-   Value: ``"http://xml.org/sax/properties/xml-string"`` ---  data type: String ---
-   description: The literal string of characters that was the source for the
-   current event. ---  access: read-only
+   | value: ``"http://xml.org/sax/properties/xml-string"``
+   | data type: String
+   | description: The literal string of characters that was the source for the
+     current event.
+   | access: read-only
 
 
 .. data:: all_properties
diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
index 5a168c0..a01735b 100644
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -36,6 +36,7 @@ The module defines the following items:
 
 
 .. class:: ZipFile
+   :noindex:
 
    The class for reading and writing ZIP files.  See section
    :ref:`zipfile-objects` for constructor details.
diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst
index 4f3d498..defb47c 100644
--- a/Doc/tutorial/datastructures.rst
+++ b/Doc/tutorial/datastructures.rst
@@ -379,7 +379,7 @@ Here is a brief demonstration::
 
    >>> basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'}
    >>> print(basket)                      # show that duplicates have been removed
-   {'orange', 'bananna', 'pear', 'apple'}
+   {'orange', 'banana', 'pear', 'apple'}
    >>> 'orange' in basket                 # fast membership testing
    True
    >>> 'crabgrass' in basket