Add more examples in pickle documentation.

1 files changed, 79 insertions, 27 deletions

diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst
index 9a31520..0a958a7 100644
--- a/Doc/library/pickle.rst
+++ b/Doc/library/pickle.rst
@@ -14,6 +14,7 @@
 .. sectionauthor:: Jim Kerr <jbkerr@sr.hp.com>.
 .. sectionauthor:: Barry Warsaw <barry@zope.com>
 
+
 The :mod:`pickle` module implements a fundamental, but powerful algorithm for
 serializing and de-serializing a Python object structure.  "Pickling" is the
 process whereby a Python object hierarchy is converted into a byte stream, and
@@ -282,23 +283,9 @@ The :mod:`pickle` module exports two classes, :class:`Pickler` and
 
    .. attribute:: memo
 
-      Dictionary holding previously pickled objects to allow shared or
-      recursive objects to pickled by reference as opposed to by value.
-
-
-.. XXX Move these comments to somewhere more appropriate.
-
-It is possible to make multiple calls to the :meth:`dump` method of the same
-:class:`Pickler` instance.  These must then be matched to the same number of
-calls to the :meth:`load` method of the corresponding :class:`Unpickler`
-instance.  If the same object is pickled by multiple :meth:`dump` calls, the
-:meth:`load` will all yield references to the same object.
-
-Please note, this is intended for pickling multiple objects without intervening
-modifications to the objects or their parts.  If you modify an object and then
-pickle it again using the same :class:`Pickler` instance, the object is not
-pickled again --- a reference to it is pickled and the :class:`Unpickler` will
-return the old value, not the modified one.
+      Dictionary-like object holding previously pickled objects to allow
+      shared or recursive objects to pickled by reference as opposed to
+      by value.
 
 
 .. class:: Unpickler(file, [\*, encoding="ASCII", errors="strict"])
@@ -345,6 +332,11 @@ return the old value, not the modified one.
       how they can be loaded, potentially reducing security risks. Refer to
       :ref:`pickle-restrict` for details.
 
+   .. attribute:: memo
+
+      Dictionary holding previously unpickled objects to allow shared or
+      recursive objects to unpickled by reference as opposed to by value.
+
 
 .. _pickle-picklable:
 
@@ -593,7 +585,7 @@ reading resumes from the last location. The :meth:`__setstate__` and
            line = self.file.readline()
            if not line:
                return None
-           if line.endswith("\n"):
+           if line.endswith('\n'):
                line = line[:-1]
            return "%i: %s" % (self.lineno, line)
 
@@ -707,25 +699,24 @@ A sample usage of our unpickler working has intended::
 
 As our examples shows, you have to be careful with what you allow to be
 unpickled.  Therefore if security is a concern, you may want to consider
-alternatives such as the marshalling API in :mod:`xmlrpc.client` or third-party
-solutions.
+alternatives such as the marshalling API in :mod:`xmlrpc.client` or
+third-party solutions.
 
 
 .. _pickle-example:
 
-Usage Examples
---------------
+Examples
+--------
 
-For the simplest code, use the :func:`dump` and :func:`load` functions.  Note
-that a self-referencing list is pickled and restored correctly. ::
+For the simplest code, use the :func:`dump` and :func:`load` functions. ::
 
    import pickle
 
    # An arbitrary collection of objects supported by pickle.
    data = {
-     'a': [1, 2.0, 3, 4+6j],
-     'b': ("character string", b"byte string"),
-     'c': set([None, True, False])
+       'a': [1, 2.0, 3, 4+6j],
+       'b': ("character string", b"byte string"),
+       'c': set([None, True, False])
    }
 
    with open('data.pickle', 'wb') as f:
@@ -743,11 +734,72 @@ The following example reads the resulting pickled data. ::
        data = pickle.load(f)
 
 
+Reusing Pickler Instances
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is possible to make multiple calls to the :meth:`dump` method of the same
+:class:`Pickler` instance.  These must then be matched to the same number of
+calls to the :meth:`load` method of the corresponding :class:`Unpickler`
+instance.  If the same object is pickled by multiple :meth:`dump` calls, the
+:meth:`load` will all yield references to the same object.
+
+Please note, this is intended for pickling multiple objects without intervening
+modifications to the objects or their parts.  If you modify an object and then
+pickle it again using the same :class:`Pickler` instance, the object is not
+pickled again --- a reference to it is pickled and the :class:`Unpickler` will
+return the old value, not the modified one. ::
+
+   import io
+   import pickle
+
+   data = {"hello": 0, "spam": 1}
+
+   # Create a binary file-like object to which the Pickler instance will
+   # write the pickles.
+   f = io.BytesIO()
+   p = pickle.Pickler(f)
+   p.dump(data)
+
+   # This second call appends a new pickle to the file. The modification we
+   # make is lost because objects are pickled by reference when seen again.
+   data["eggs"] = 2
+   p.dump(data)
+
+   # Now, we load the pickles saved in our file-like object.
+   f.seek(0)
+   u = pickle.Unpickler(f)
+   data1 = u.load()
+   data2 = u.load()
+
+   if data1 is data2:
+       print("data1 and data2 are the same object")
+   else:
+       print("data1 and data2 are not the same object")
+
+   if "eggs" in data2:
+       print("The modification was pickled.")
+   else:
+       print("The modification was not pickled.")
+
+Reusing a :class:`Pickler` instance like we shown can be a useful
+optimization. For example, a multi-process application could use this feature
+to reduce the size of the pickles transmitted across processes over time
+(assuming the pickles exchanged are containers sharing common immutable
+objects). However, you should take special care to regularly clear
+:attr:``Pickler.memo`` and :attr:``Unpickler.memo`` to avoid memory-leaks.
+
+.. XXX: Add examples showing how to optimize pickles for size (like using
+.. pickletools.optimize() or the gzip module).
+
+
 .. seealso::
 
    Module :mod:`copyreg`
       Pickle interface constructor registration for extension types.
 
+   Module :mod:`pickletools`
+      Tools for working with and analyzing pickled data.
+
    Module :mod:`shelve`
       Indexed databases of objects; uses :mod:`pickle`.