#1682942: add some ConfigParser features: alternate delimiters, alternate comments, empty lines in values. Also enhance the docs with more examples and mention SafeConfigParser before ConfigParser. Patch by Lukas Langa, review by myself, Eric and Ezio.

1 files changed, 148 insertions, 80 deletions

diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst
index 34a40ee..792784b 100644
--- a/Doc/library/configparser.rst
+++ b/Doc/library/configparser.rst
@@ -15,105 +15,168 @@
    single: ini file
    single: Windows ini file
 
-This module defines the class :class:`ConfigParser`.   The :class:`ConfigParser`
-class implements a basic configuration file parser language which provides a
-structure similar to what you would find on Microsoft Windows INI files.  You
-can use this to write Python programs which can be customized by end users
-easily.
+This module provides the classes :class:`RawConfigParser` and
+:class:`SafeConfigParser`.  They implement a basic configuration file parser
+language which provides a structure similar to what you would find in Microsoft
+Windows INI files.  You can use this to write Python programs which can be
+customized by end users easily.
 
 .. note::
 
    This library does *not* interpret or write the value-type prefixes used in
    the Windows Registry extended version of INI syntax.
 
-The configuration file consists of sections, led by a ``[section]`` header and
-followed by ``name: value`` entries, with continuations in the style of
-:rfc:`822` (see section 3.1.1, "LONG HEADER FIELDS"); ``name=value`` is also
-accepted.  Note that leading whitespace is removed from values. The optional
-values can contain format strings which refer to other values in the same
-section, or values in a special ``DEFAULT`` section.  Additional defaults can be
-provided on initialization and retrieval.  Lines beginning with ``'#'`` or
-``';'`` are ignored and may be used to provide comments.
+A configuration file consists of sections, each led by a ``[section]`` header,
+followed by name/value entries separated by a specific string (``=`` or ``:`` by
+default).  Note that leading whitespace is removed from values.  Values can be
+ommitted, in which case the key/value delimiter may also be left out.  Values
+can also span multiple lines, as long as they are indented deeper than the first
+line of the value.  Depending on the parser's mode, blank lines may be treated
+as parts of multiline values or ignored.
+
+Configuration files may include comments, prefixed by specific characters (``#``
+and ``;`` by default).  Comments may appear on their own in an otherwise empty
+line, or may be entered in lines holding values or spection names.  In the
+latter case, they need to be preceded by a whitespace character to be recognized
+as a comment.  (For backwards compatibility, by default only ``;`` starts an
+inline comment, while ``#`` does not.)
+
+On top of the core functionality, :class:`SafeConfigParser` supports
+interpolation.  This means values can contain format strings which refer to
+other values in the same section, or values in a special ``DEFAULT`` section.
+Additional defaults can be provided on initialization and retrieval.
 
 For example::
 
-   [My Section]
-   foodir: %(dir)s/whatever
-   dir=frob
-   long: this value continues
-      in the next line
-
-would resolve the ``%(dir)s`` to the value of ``dir`` (``frob`` in this case).
-All reference expansions are done on demand.
-
-Default values can be specified by passing them into the :class:`ConfigParser`
-constructor as a dictionary.  Additional defaults  may be passed into the
-:meth:`get` method which will override all others.
-
-Sections are normally stored in a built-in dictionary. An alternative dictionary
-type can be passed to the :class:`ConfigParser` constructor. For example, if a
-dictionary type is passed that sorts its keys, the sections will be sorted on
-write-back, as will be the keys within each section.
-
-
-.. class:: RawConfigParser(defaults=None, dict_type=collections.OrderedDict,
-                           allow_no_value=False)
+   [Paths]
+   home_dir: /Users
+   my_dir: %(home_dir)s/lumberjack
+   my_pictures: %(my_dir)s/Pictures
+
+   [Multiline Values]
+   chorus: I'm a lumberjack, and I'm okay
+      I sleep all night and I work all day
+
+   [No Values]
+   key_without_value
+   empty string value here =
+
+   [You can use comments] ; after a useful line
+   ; in an empty line
+   after: a_value ; here's another comment
+   inside: a         ;comment
+           multiline ;comment
+           value!    ;comment
+
+      [Sections Can Be Indented]
+         can_values_be_as_well = True
+         does_that_mean_anything_special = False
+         purpose = formatting for readability
+         multiline_values = are
+            handled just fine as
+            long as they are indented
+            deeper than the first line
+            of a value
+         # Did I mention we can indent comments, too?
+
+
+In the example above, :class:`SafeConfigParser` would resolve ``%(home_dir)s``
+to the value of ``home_dir`` (``/Users`` in this case).  ``%(my_dir)s`` in
+effect would resolve to ``/Users/lumberjack``.  All interpolations are done on
+demand so keys used in the chain of references do not have to be specified in
+any specific order in the configuration file.
+
+:class:`RawConfigParser` would simply return ``%(my_dir)s/Pictures`` as the
+value of ``my_pictures`` and ``%(home_dir)s/lumberjack`` as the value of
+``my_dir``.  Other features presented in the example are handled in the same
+manner by both parsers.
+
+Default values can be specified by passing them as a dictionary when
+constructing the :class:`SafeConfigParser`.  Additional defaults may be passed
+to the :meth:`get` method which will override all others.
+
+Sections are normally stored in an :class:`collections.OrderedDict` which
+maintains the order of all keys.  An alternative dictionary type can be passed
+to the :meth:`__init__` method.  For example, if a dictionary type is passed
+that sorts its keys, the sections will be sorted on write-back, as will be the
+keys within each section.
+
+
+.. class:: RawConfigParser(defaults=None, dict_type=collections.OrderedDict, delimiters=('=', ':'), comment_prefixes=_COMPATIBLE, empty_lines_in_values=True, allow_no_value=False)
 
    The basic configuration object.  When *defaults* is given, it is initialized
-   into the dictionary of intrinsic defaults.  When *dict_type* is given, it will
-   be used to create the dictionary objects for the list of sections, for the
-   options within a section, and for the default values.  When *allow_no_value*
-   is true (default: ``False``), options without values are accepted; the value
+   into the dictionary of intrinsic defaults.  When *dict_type* is given, it
+   will be used to create the dictionary objects for the list of sections, for
+   the options within a section, and for the default values.
+
+   When *delimiters* is given, it will be used as the set of substrings that
+   divide keys from values.  When *comment_prefixes* is given, it will be used
+   as the set of substrings that prefix comments in a line, both for the whole
+   line and inline comments.  For backwards compatibility, the default value for
+   *comment_prefixes* is a special value that indicates that ``;`` and ``#`` can
+   start whole line comments while only ``;`` can start inline comments.
+
+   When *empty_lines_in_values* is ``False`` (default: ``True``), each empty
+   line marks the end of an option.  Otherwise, internal empty lines of a
+   multiline option are kept as part of the value.  When *allow_no_value* is
+   true (default: ``False``), options without values are accepted; the value
    presented for these is ``None``.
 
-   This class does not
-   support the magical interpolation behavior.
+   This class does not support the magical interpolation behavior.
 
    .. versionchanged:: 3.1
       The default *dict_type* is :class:`collections.OrderedDict`.
 
    .. versionchanged:: 3.2
-      *allow_no_value* was added.
+      *delimiters*, *comment_prefixes*, *empty_lines_in_values* and
+      *allow_no_value* were added.
 
 
-.. class:: ConfigParser(defaults=None, dict_type=collections.OrderedDict,
-                        allow_no_value=False)
+.. class:: SafeConfigParser(defaults=None, dict_type=collections.OrderedDict, delimiters=('=', ':'), comment_prefixes=('#', ';'), empty_lines_in_values=True, allow_no_value=False)
 
-   Derived class of :class:`RawConfigParser` that implements the magical
-   interpolation feature and adds optional arguments to the :meth:`get` and
-   :meth:`items` methods.  The values in *defaults* must be appropriate for the
-   ``%()s`` string interpolation.  Note that *__name__* is an intrinsic default;
-   its value is the section name, and will override any value provided in
-   *defaults*.
+   Derived class of :class:`ConfigParser` that implements a sane variant of the
+   magical interpolation feature.  This implementation is more predictable as it
+   validates the interpolation syntax used within a configuration file.  This
+   class also enables escaping the interpolation character (e.g. a key can have
+   ``%`` as part of the value by specifying ``%%`` in the file).
 
-   All option names used in interpolation will be passed through the
-   :meth:`optionxform` method just like any other option name reference.  For
-   example, using the default implementation of :meth:`optionxform` (which converts
-   option names to lower case), the values ``foo %(bar)s`` and ``foo %(BAR)s`` are
-   equivalent.
+   Applications that don't require interpolation should use
+   :class:`RawConfigParser`, otherwise :class:`SafeConfigParser` is the best
+   option.
 
    .. versionchanged:: 3.1
       The default *dict_type* is :class:`collections.OrderedDict`.
 
    .. versionchanged:: 3.2
-      *allow_no_value* was added.
+      *delimiters*, *comment_prefixes*, *empty_lines_in_values* and
+      *allow_no_value* were added.
+
 
+.. class:: ConfigParser(defaults=None, dict_type=collections.OrderedDict, delimiters=('=', ':'), comment_prefixes=('#', ';'), empty_lines_in_values=True, allow_no_value=False)
+
+   Derived class of :class:`RawConfigParser` that implements the magical
+   interpolation feature and adds optional arguments to the :meth:`get` and
+   :meth:`items` methods.
 
-.. class:: SafeConfigParser(defaults=None, dict_type=collections.OrderedDict,
-                            allow_no_value=False)
+   :class:`SafeConfigParser` is generally recommended over this class if you
+   need interpolation.
 
-   Derived class of :class:`ConfigParser` that implements a more-sane variant of
-   the magical interpolation feature.  This implementation is more predictable as
-   well. New applications should prefer this version if they don't need to be
-   compatible with older versions of Python.
+   The values in *defaults* must be appropriate for the ``%()s`` string
+   interpolation.  Note that *__name__* is an intrinsic default; its value is
+   the section name, and will override any value provided in *defaults*.
 
-   .. XXX Need to explain what's safer/more predictable about it.
+   All option names used in interpolation will be passed through the
+   :meth:`optionxform` method just like any other option name reference.  For
+   example, using the default implementation of :meth:`optionxform` (which
+   converts option names to lower case), the values ``foo %(bar)s`` and ``foo
+   %(BAR)s`` are equivalent.
 
    .. versionchanged:: 3.1
       The default *dict_type* is :class:`collections.OrderedDict`.
 
    .. versionchanged:: 3.2
-      *allow_no_value* was added.
+      *delimiters*, *comment_prefixes*, *empty_lines_in_values* and
+      *allow_no_value* were added.
 
 
 .. exception:: NoSectionError
@@ -295,11 +358,13 @@ RawConfigParser Objects
    interpolation and output to files) can only be achieved using string values.
 
 
-.. method:: RawConfigParser.write(fileobject)
+.. method:: RawConfigParser.write(fileobject, space_around_delimiters=True)
 
    Write a representation of the configuration to the specified file object,
    which must be opened in text mode (accepting strings).  This representation
-   can be parsed by a future :meth:`read` call.
+   can be parsed by a future :meth:`read` call. If ``space_around_delimiters``
+   is ``True`` (the default), delimiters between keys and values are surrounded
+   by spaces.
 
 
 .. method:: RawConfigParser.remove_option(section, option)
@@ -342,21 +407,24 @@ ConfigParser Objects
 --------------------
 
 The :class:`ConfigParser` class extends some methods of the
-:class:`RawConfigParser` interface, adding some optional arguments.
+:class:`RawConfigParser` interface, adding some optional arguments. Whenever you
+can, consider using :class:`SafeConfigParser` which adds validation and escaping
+for the interpolation.
 
 
 .. method:: ConfigParser.get(section, option, raw=False, vars=None)
 
-   Get an *option* value for the named *section*.  All the ``'%'`` interpolations
-   are expanded in the return values, based on the defaults passed into the
-   constructor, as well as the options *vars* provided, unless the *raw* argument
-   is true.
+   Get an *option* value for the named *section*.  All the ``'%'``
+   interpolations are expanded in the return values, based on the defaults
+   passed into the :meth:`__init__` method, as well as the options *vars*
+   provided, unless the *raw* argument is true.
 
 
 .. method:: ConfigParser.items(section, raw=False, vars=None)
 
-   Return a list of ``(name, value)`` pairs for each option in the given *section*.
-   Optional arguments have the same meaning as for the :meth:`get` method.
+   Return a list of ``(name, value)`` pairs for each option in the given
+   *section*.  Optional arguments have the same meaning as for the :meth:`get`
+   method.
 
 
 .. _safeconfigparser-objects:
@@ -466,8 +534,8 @@ The function ``opt_move`` below can be used to move options between sections::
 
 Some configuration files are known to include settings without values, but which
 otherwise conform to the syntax supported by :mod:`configparser`.  The
-*allow_no_value* parameter to the constructor can be used to indicate that such
-values should be accepted:
+*allow_no_value* parameter to the :meth:`__init__` method can be used to
+indicate that such values should be accepted:
 
 .. doctest::
 
@@ -476,12 +544,12 @@ values should be accepted:
 
    >>> sample_config = """
    ... [mysqld]
-   ... user = mysql
-   ... pid-file = /var/run/mysqld/mysqld.pid
-   ... skip-external-locking
-   ... old_passwords = 1
-   ... skip-bdb
-   ... skip-innodb
+   ...   user = mysql
+   ...   pid-file = /var/run/mysqld/mysqld.pid
+   ...   skip-external-locking
+   ...   old_passwords = 1
+   ...   skip-bdb
+   ...   skip-innodb # we don't need ACID today
    ... """
    >>> config = configparser.RawConfigParser(allow_no_value=True)
    >>> config.readfp(io.BytesIO(sample_config))