Move the 3k reST doc tree in place.

1 files changed, 249 insertions, 0 deletions

diff --git a/Doc/library/email.charset.rst b/Doc/library/email.charset.rst
new file mode 100644
index 0000000..d16d281
--- /dev/null
+++ b/Doc/library/email.charset.rst
@@ -0,0 +1,249 @@
+:mod:`email`: Representing character sets
+-----------------------------------------
+
+.. module:: email.charset
+   :synopsis: Character Sets
+
+
+This module provides a class :class:`Charset` for representing character sets
+and character set conversions in email messages, as well as a character set
+registry and several convenience methods for manipulating this registry.
+Instances of :class:`Charset` are used in several other modules within the
+:mod:`email` package.
+
+Import this class from the :mod:`email.charset` module.
+
+.. versionadded:: 2.2.2
+
+
+.. class:: Charset([input_charset])
+
+   Map character sets to their email properties.
+
+   This class provides information about the requirements imposed on email for a
+   specific character set.  It also provides convenience routines for converting
+   between character sets, given the availability of the applicable codecs.  Given
+   a character set, it will do its best to provide information on how to use that
+   character set in an email message in an RFC-compliant way.
+
+   Certain character sets must be encoded with quoted-printable or base64 when used
+   in email headers or bodies.  Certain character sets must be converted outright,
+   and are not allowed in email.
+
+   Optional *input_charset* is as described below; it is always coerced to lower
+   case.  After being alias normalized it is also used as a lookup into the
+   registry of character sets to find out the header encoding, body encoding, and
+   output conversion codec to be used for the character set.  For example, if
+   *input_charset* is ``iso-8859-1``, then headers and bodies will be encoded using
+   quoted-printable and no output conversion codec is necessary.  If
+   *input_charset* is ``euc-jp``, then headers will be encoded with base64, bodies
+   will not be encoded, but output text will be converted from the ``euc-jp``
+   character set to the ``iso-2022-jp`` character set.
+
+:class:`Charset` instances have the following data attributes:
+
+
+.. data:: input_charset
+
+   The initial character set specified.  Common aliases are converted to their
+   *official* email names (e.g. ``latin_1`` is converted to ``iso-8859-1``).
+   Defaults to 7-bit ``us-ascii``.
+
+
+.. data:: header_encoding
+
+   If the character set must be encoded before it can be used in an email header,
+   this attribute will be set to ``Charset.QP`` (for quoted-printable),
+   ``Charset.BASE64`` (for base64 encoding), or ``Charset.SHORTEST`` for the
+   shortest of QP or BASE64 encoding. Otherwise, it will be ``None``.
+
+
+.. data:: body_encoding
+
+   Same as *header_encoding*, but describes the encoding for the mail message's
+   body, which indeed may be different than the header encoding.
+   ``Charset.SHORTEST`` is not allowed for *body_encoding*.
+
+
+.. data:: output_charset
+
+   Some character sets must be converted before they can be used in email headers
+   or bodies.  If the *input_charset* is one of them, this attribute will contain
+   the name of the character set output will be converted to.  Otherwise, it will
+   be ``None``.
+
+
+.. data:: input_codec
+
+   The name of the Python codec used to convert the *input_charset* to Unicode.  If
+   no conversion codec is necessary, this attribute will be ``None``.
+
+
+.. data:: output_codec
+
+   The name of the Python codec used to convert Unicode to the *output_charset*.
+   If no conversion codec is necessary, this attribute will have the same value as
+   the *input_codec*.
+
+:class:`Charset` instances also have the following methods:
+
+
+.. method:: Charset.get_body_encoding()
+
+   Return the content transfer encoding used for body encoding.
+
+   This is either the string ``quoted-printable`` or ``base64`` depending on the
+   encoding used, or it is a function, in which case you should call the function
+   with a single argument, the Message object being encoded.  The function should
+   then set the :mailheader:`Content-Transfer-Encoding` header itself to whatever
+   is appropriate.
+
+   Returns the string ``quoted-printable`` if *body_encoding* is ``QP``, returns
+   the string ``base64`` if *body_encoding* is ``BASE64``, and returns the string
+   ``7bit`` otherwise.
+
+
+.. method:: Charset.convert(s)
+
+   Convert the string *s* from the *input_codec* to the *output_codec*.
+
+
+.. method:: Charset.to_splittable(s)
+
+   Convert a possibly multibyte string to a safely splittable format. *s* is the
+   string to split.
+
+   Uses the *input_codec* to try and convert the string to Unicode, so it can be
+   safely split on character boundaries (even for multibyte characters).
+
+   Returns the string as-is if it isn't known how to convert *s* to Unicode with
+   the *input_charset*.
+
+   Characters that could not be converted to Unicode will be replaced with the
+   Unicode replacement character ``'U+FFFD'``.
+
+
+.. method:: Charset.from_splittable(ustr[, to_output])
+
+   Convert a splittable string back into an encoded string.  *ustr* is a Unicode
+   string to "unsplit".
+
+   This method uses the proper codec to try and convert the string from Unicode
+   back into an encoded format.  Return the string as-is if it is not Unicode, or
+   if it could not be converted from Unicode.
+
+   Characters that could not be converted from Unicode will be replaced with an
+   appropriate character (usually ``'?'``).
+
+   If *to_output* is ``True`` (the default), uses *output_codec* to convert to an
+   encoded format.  If *to_output* is ``False``, it uses *input_codec*.
+
+
+.. method:: Charset.get_output_charset()
+
+   Return the output character set.
+
+   This is the *output_charset* attribute if that is not ``None``, otherwise it is
+   *input_charset*.
+
+
+.. method:: Charset.encoded_header_len()
+
+   Return the length of the encoded header string, properly calculating for
+   quoted-printable or base64 encoding.
+
+
+.. method:: Charset.header_encode(s[, convert])
+
+   Header-encode the string *s*.
+
+   If *convert* is ``True``, the string will be converted from the input charset to
+   the output charset automatically.  This is not useful for multibyte character
+   sets, which have line length issues (multibyte characters must be split on a
+   character, not a byte boundary); use the higher-level :class:`Header` class to
+   deal with these issues (see :mod:`email.header`).  *convert* defaults to
+   ``False``.
+
+   The type of encoding (base64 or quoted-printable) will be based on the
+   *header_encoding* attribute.
+
+
+.. method:: Charset.body_encode(s[, convert])
+
+   Body-encode the string *s*.
+
+   If *convert* is ``True`` (the default), the string will be converted from the
+   input charset to output charset automatically. Unlike :meth:`header_encode`,
+   there are no issues with byte boundaries and multibyte charsets in email bodies,
+   so this is usually pretty safe.
+
+   The type of encoding (base64 or quoted-printable) will be based on the
+   *body_encoding* attribute.
+
+The :class:`Charset` class also provides a number of methods to support standard
+operations and built-in functions.
+
+
+.. method:: Charset.__str__()
+
+   Returns *input_charset* as a string coerced to lower case. :meth:`__repr__` is
+   an alias for :meth:`__str__`.
+
+
+.. method:: Charset.__eq__(other)
+
+   This method allows you to compare two :class:`Charset` instances for equality.
+
+
+.. method:: Header.__ne__(other)
+
+   This method allows you to compare two :class:`Charset` instances for inequality.
+
+The :mod:`email.charset` module also provides the following functions for adding
+new entries to the global character set, alias, and codec registries:
+
+
+.. function:: add_charset(charset[, header_enc[, body_enc[, output_charset]]])
+
+   Add character properties to the global registry.
+
+   *charset* is the input character set, and must be the canonical name of a
+   character set.
+
+   Optional *header_enc* and *body_enc* is either ``Charset.QP`` for
+   quoted-printable, ``Charset.BASE64`` for base64 encoding,
+   ``Charset.SHORTEST`` for the shortest of quoted-printable or base64 encoding,
+   or ``None`` for no encoding.  ``SHORTEST`` is only valid for
+   *header_enc*. The default is ``None`` for no encoding.
+
+   Optional *output_charset* is the character set that the output should be in.
+   Conversions will proceed from input charset, to Unicode, to the output charset
+   when the method :meth:`Charset.convert` is called.  The default is to output in
+   the same character set as the input.
+
+   Both *input_charset* and *output_charset* must have Unicode codec entries in the
+   module's character set-to-codec mapping; use :func:`add_codec` to add codecs the
+   module does not know about.  See the :mod:`codecs` module's documentation for
+   more information.
+
+   The global character set registry is kept in the module global dictionary
+   ``CHARSETS``.
+
+
+.. function:: add_alias(alias, canonical)
+
+   Add a character set alias.  *alias* is the alias name, e.g. ``latin-1``.
+   *canonical* is the character set's canonical name, e.g. ``iso-8859-1``.
+
+   The global charset alias registry is kept in the module global dictionary
+   ``ALIASES``.
+
+
+.. function:: add_codec(charset, codecname)
+
+   Add a codec that map characters in the given character set to and from Unicode.
+
+   *charset* is the canonical name of a character set. *codecname* is the name of a
+   Python codec, as appropriate for the second argument to the :func:`unicode`
+   built-in, or to the :meth:`encode` method of a Unicode string.
+