Move the 2.6 reST doc tree in place.

1 files changed, 868 insertions, 0 deletions

diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst
new file mode 100644
index 0000000..ab19ff8
--- /dev/null
+++ b/Doc/library/pickle.rst
@@ -0,0 +1,868 @@
+
+:mod:`pickle` --- Python object serialization
+=============================================
+
+.. index::
+   single: persistence
+   pair: persistent; objects
+   pair: serializing; objects
+   pair: marshalling; objects
+   pair: flattening; objects
+   pair: pickling; objects
+
+.. module:: pickle
+   :synopsis: Convert Python objects to streams of bytes and back.
+
+
+.. % Substantial improvements by Jim Kerr <jbkerr@sr.hp.com>.
+.. % Rewritten by Barry Warsaw <barry@zope.com>
+
+The :mod:`pickle` module implements a fundamental, but powerful algorithm for
+serializing and de-serializing a Python object structure.  "Pickling" is the
+process whereby a Python object hierarchy is converted into a byte stream, and
+"unpickling" is the inverse operation, whereby a byte stream is converted back
+into an object hierarchy.  Pickling (and unpickling) is alternatively known as
+"serialization", "marshalling," [#]_ or "flattening", however, to avoid
+confusion, the terms used here are "pickling" and "unpickling".
+
+This documentation describes both the :mod:`pickle` module and the
+:mod:`cPickle` module.
+
+
+Relationship to other Python modules
+------------------------------------
+
+The :mod:`pickle` module has an optimized cousin called the :mod:`cPickle`
+module.  As its name implies, :mod:`cPickle` is written in C, so it can be up to
+1000 times faster than :mod:`pickle`.  However it does not support subclassing
+of the :func:`Pickler` and :func:`Unpickler` classes, because in :mod:`cPickle`
+these are functions, not classes.  Most applications have no need for this
+functionality, and can benefit from the improved performance of :mod:`cPickle`.
+Other than that, the interfaces of the two modules are nearly identical; the
+common interface is described in this manual and differences are pointed out
+where necessary.  In the following discussions, we use the term "pickle" to
+collectively describe the :mod:`pickle` and :mod:`cPickle` modules.
+
+The data streams the two modules produce are guaranteed to be interchangeable.
+
+Python has a more primitive serialization module called :mod:`marshal`, but in
+general :mod:`pickle` should always be the preferred way to serialize Python
+objects.  :mod:`marshal` exists primarily to support Python's :file:`.pyc`
+files.
+
+The :mod:`pickle` module differs from :mod:`marshal` several significant ways:
+
+* The :mod:`pickle` module keeps track of the objects it has already serialized,
+  so that later references to the same object won't be serialized again.
+  :mod:`marshal` doesn't do this.
+
+  This has implications both for recursive objects and object sharing.  Recursive
+  objects are objects that contain references to themselves.  These are not
+  handled by marshal, and in fact, attempting to marshal recursive objects will
+  crash your Python interpreter.  Object sharing happens when there are multiple
+  references to the same object in different places in the object hierarchy being
+  serialized.  :mod:`pickle` stores such objects only once, and ensures that all
+  other references point to the master copy.  Shared objects remain shared, which
+  can be very important for mutable objects.
+
+* :mod:`marshal` cannot be used to serialize user-defined classes and their
+  instances.  :mod:`pickle` can save and restore class instances transparently,
+  however the class definition must be importable and live in the same module as
+  when the object was stored.
+
+* The :mod:`marshal` serialization format is not guaranteed to be portable
+  across Python versions.  Because its primary job in life is to support
+  :file:`.pyc` files, the Python implementers reserve the right to change the
+  serialization format in non-backwards compatible ways should the need arise.
+  The :mod:`pickle` serialization format is guaranteed to be backwards compatible
+  across Python releases.
+
+.. warning::
+
+   The :mod:`pickle` module is not intended to be secure against erroneous or
+   maliciously constructed data.  Never unpickle data received from an untrusted or
+   unauthenticated source.
+
+Note that serialization is a more primitive notion than persistence; although
+:mod:`pickle` reads and writes file objects, it does not handle the issue of
+naming persistent objects, nor the (even more complicated) issue of concurrent
+access to persistent objects.  The :mod:`pickle` module can transform a complex
+object into a byte stream and it can transform the byte stream into an object
+with the same internal structure.  Perhaps the most obvious thing to do with
+these byte streams is to write them onto a file, but it is also conceivable to
+send them across a network or store them in a database.  The module
+:mod:`shelve` provides a simple interface to pickle and unpickle objects on
+DBM-style database files.
+
+
+Data stream format
+------------------
+
+.. index::
+   single: XDR
+   single: External Data Representation
+
+The data format used by :mod:`pickle` is Python-specific.  This has the
+advantage that there are no restrictions imposed by external standards such as
+XDR (which can't represent pointer sharing); however it means that non-Python
+programs may not be able to reconstruct pickled Python objects.
+
+By default, the :mod:`pickle` data format uses a printable ASCII representation.
+This is slightly more voluminous than a binary representation.  The big
+advantage of using printable ASCII (and of some other characteristics of
+:mod:`pickle`'s representation) is that for debugging or recovery purposes it is
+possible for a human to read the pickled file with a standard text editor.
+
+There are currently 3 different protocols which can be used for pickling.
+
+* Protocol version 0 is the original ASCII protocol and is backwards compatible
+  with earlier versions of Python.
+
+* Protocol version 1 is the old binary format which is also compatible with
+  earlier versions of Python.
+
+* Protocol version 2 was introduced in Python 2.3.  It provides much more
+  efficient pickling of new-style classes.
+
+Refer to :pep:`307` for more information.
+
+If a *protocol* is not specified, protocol 0 is used. If *protocol* is specified
+as a negative value or :const:`HIGHEST_PROTOCOL`, the highest protocol version
+available will be used.
+
+.. versionchanged:: 2.3
+   Introduced the *protocol* parameter.
+
+A binary format, which is slightly more efficient, can be chosen by specifying a
+*protocol* version >= 1.
+
+
+Usage
+-----
+
+To serialize an object hierarchy, you first create a pickler, then you call the
+pickler's :meth:`dump` method.  To de-serialize a data stream, you first create
+an unpickler, then you call the unpickler's :meth:`load` method.  The
+:mod:`pickle` module provides the following constant:
+
+
+.. data:: HIGHEST_PROTOCOL
+
+   The highest protocol version available.  This value can be passed as a
+   *protocol* value.
+
+   .. versionadded:: 2.3
+
+.. note::
+
+   Be sure to always open pickle files created with protocols >= 1 in binary mode.
+   For the old ASCII-based pickle protocol 0 you can use either text mode or binary
+   mode as long as you stay consistent.
+
+   A pickle file written with protocol 0 in binary mode will contain lone linefeeds
+   as line terminators and therefore will look "funny" when viewed in Notepad or
+   other editors which do not support this format.
+
+The :mod:`pickle` module provides the following functions to make the pickling
+process more convenient:
+
+
+.. function:: dump(obj, file[, protocol])
+
+   Write a pickled representation of *obj* to the open file object *file*.  This is
+   equivalent to ``Pickler(file, protocol).dump(obj)``.
+
+   If the *protocol* parameter is omitted, protocol 0 is used. If *protocol* is
+   specified as a negative value or :const:`HIGHEST_PROTOCOL`, the highest protocol
+   version will be used.
+
+   .. versionchanged:: 2.3
+      Introduced the *protocol* parameter.
+
+   *file* must have a :meth:`write` method that accepts a single string argument.
+   It can thus be a file object opened for writing, a :mod:`StringIO` object, or
+   any other custom object that meets this interface.
+
+
+.. function:: load(file)
+
+   Read a string from the open file object *file* and interpret it as a pickle data
+   stream, reconstructing and returning the original object hierarchy.  This is
+   equivalent to ``Unpickler(file).load()``.
+
+   *file* must have two methods, a :meth:`read` method that takes an integer
+   argument, and a :meth:`readline` method that requires no arguments.  Both
+   methods should return a string.  Thus *file* can be a file object opened for
+   reading, a :mod:`StringIO` object, or any other custom object that meets this
+   interface.
+
+   This function automatically determines whether the data stream was written in
+   binary mode or not.
+
+
+.. function:: dumps(obj[, protocol])
+
+   Return the pickled representation of the object as a string, instead of writing
+   it to a file.
+
+   If the *protocol* parameter is omitted, protocol 0 is used. If *protocol* is
+   specified as a negative value or :const:`HIGHEST_PROTOCOL`, the highest protocol
+   version will be used.
+
+   .. versionchanged:: 2.3
+      The *protocol* parameter was added.
+
+
+.. function:: loads(string)
+
+   Read a pickled object hierarchy from a string.  Characters in the string past
+   the pickled object's representation are ignored.
+
+The :mod:`pickle` module also defines three exceptions:
+
+
+.. exception:: PickleError
+
+   A common base class for the other exceptions defined below.  This inherits from
+   :exc:`Exception`.
+
+
+.. exception:: PicklingError
+
+   This exception is raised when an unpicklable object is passed to the
+   :meth:`dump` method.
+
+
+.. exception:: UnpicklingError
+
+   This exception is raised when there is a problem unpickling an object. Note that
+   other exceptions may also be raised during unpickling, including (but not
+   necessarily limited to) :exc:`AttributeError`, :exc:`EOFError`,
+   :exc:`ImportError`, and :exc:`IndexError`.
+
+The :mod:`pickle` module also exports two callables [#]_, :class:`Pickler` and
+:class:`Unpickler`:
+
+
+.. class:: Pickler(file[, protocol])
+
+   This takes a file-like object to which it will write a pickle data stream.
+
+   If the *protocol* parameter is omitted, protocol 0 is used. If *protocol* is
+   specified as a negative value or :const:`HIGHEST_PROTOCOL`, the highest
+   protocol version will be used.
+
+   .. versionchanged:: 2.3
+      Introduced the *protocol* parameter.
+
+   *file* must have a :meth:`write` method that accepts a single string argument.
+   It can thus be an open file object, a :mod:`StringIO` object, or any other
+   custom object that meets this interface.
+
+:class:`Pickler` objects define one (or two) public methods:
+
+
+.. method:: Pickler.dump(obj)
+
+   Write a pickled representation of *obj* to the open file object given in the
+   constructor.  Either the binary or ASCII format will be used, depending on the
+   value of the *protocol* argument passed to the constructor.
+
+
+.. method:: Pickler.clear_memo()
+
+   Clears the pickler's "memo".  The memo is the data structure that remembers
+   which objects the pickler has already seen, so that shared or recursive objects
+   pickled by reference and not by value.  This method is useful when re-using
+   picklers.
+
+   .. note::
+
+      Prior to Python 2.3, :meth:`clear_memo` was only available on the picklers
+      created by :mod:`cPickle`.  In the :mod:`pickle` module, picklers have an
+      instance variable called :attr:`memo` which is a Python dictionary.  So to clear
+      the memo for a :mod:`pickle` module pickler, you could do the following::
+
+         mypickler.memo.clear()
+
+      Code that does not need to support older versions of Python should simply use
+      :meth:`clear_memo`.
+
+It is possible to make multiple calls to the :meth:`dump` method of the same
+:class:`Pickler` instance.  These must then be matched to the same number of
+calls to the :meth:`load` method of the corresponding :class:`Unpickler`
+instance.  If the same object is pickled by multiple :meth:`dump` calls, the
+:meth:`load` will all yield references to the same object. [#]_
+
+:class:`Unpickler` objects are defined as:
+
+
+.. class:: Unpickler(file)
+
+   This takes a file-like object from which it will read a pickle data stream.
+   This class automatically determines whether the data stream was written in
+   binary mode or not, so it does not need a flag as in the :class:`Pickler`
+   factory.
+
+   *file* must have two methods, a :meth:`read` method that takes an integer
+   argument, and a :meth:`readline` method that requires no arguments.  Both
+   methods should return a string.  Thus *file* can be a file object opened for
+   reading, a :mod:`StringIO` object, or any other custom object that meets this
+   interface.
+
+:class:`Unpickler` objects have one (or two) public methods:
+
+
+.. method:: Unpickler.load()
+
+   Read a pickled object representation from the open file object given in the
+   constructor, and return the reconstituted object hierarchy specified therein.
+
+   This method automatically determines whether the data stream was written in
+   binary mode or not.
+
+
+.. method:: Unpickler.noload()
+
+   This is just like :meth:`load` except that it doesn't actually create any
+   objects.  This is useful primarily for finding what's called "persistent ids"
+   that may be referenced in a pickle data stream.  See section
+   :ref:`pickle-protocol` below for more details.
+
+   **Note:** the :meth:`noload` method is currently only available on
+   :class:`Unpickler` objects created with the :mod:`cPickle` module.
+   :mod:`pickle` module :class:`Unpickler`\ s do not have the :meth:`noload`
+   method.
+
+
+What can be pickled and unpickled?
+----------------------------------
+
+The following types can be pickled:
+
+* ``None``, ``True``, and ``False``
+
+* integers, long integers, floating point numbers, complex numbers
+
+* normal and Unicode strings
+
+* tuples, lists, sets, and dictionaries containing only picklable objects
+
+* functions defined at the top level of a module
+
+* built-in functions defined at the top level of a module
+
+* classes that are defined at the top level of a module
+
+* instances of such classes whose :attr:`__dict__` or :meth:`__setstate__` is
+  picklable  (see section :ref:`pickle-protocol` for details)
+
+Attempts to pickle unpicklable objects will raise the :exc:`PicklingError`
+exception; when this happens, an unspecified number of bytes may have already
+been written to the underlying file. Trying to pickle a highly recursive data
+structure may exceed the maximum recursion depth, a :exc:`RuntimeError` will be
+raised in this case. You can carefully raise this limit with
+:func:`sys.setrecursionlimit`.
+
+Note that functions (built-in and user-defined) are pickled by "fully qualified"
+name reference, not by value.  This means that only the function name is
+pickled, along with the name of module the function is defined in.  Neither the
+function's code, nor any of its function attributes are pickled.  Thus the
+defining module must be importable in the unpickling environment, and the module
+must contain the named object, otherwise an exception will be raised. [#]_
+
+Similarly, classes are pickled by named reference, so the same restrictions in
+the unpickling environment apply.  Note that none of the class's code or data is
+pickled, so in the following example the class attribute ``attr`` is not
+restored in the unpickling environment::
+
+   class Foo:
+       attr = 'a class attr'
+
+   picklestring = pickle.dumps(Foo)
+
+These restrictions are why picklable functions and classes must be defined in
+the top level of a module.
+
+Similarly, when class instances are pickled, their class's code and data are not
+pickled along with them.  Only the instance data are pickled.  This is done on
+purpose, so you can fix bugs in a class or add methods to the class and still
+load objects that were created with an earlier version of the class.  If you
+plan to have long-lived objects that will see many versions of a class, it may
+be worthwhile to put a version number in the objects so that suitable
+conversions can be made by the class's :meth:`__setstate__` method.
+
+
+.. _pickle-protocol:
+
+The pickle protocol
+-------------------
+
+This section describes the "pickling protocol" that defines the interface
+between the pickler/unpickler and the objects that are being serialized.  This
+protocol provides a standard way for you to define, customize, and control how
+your objects are serialized and de-serialized.  The description in this section
+doesn't cover specific customizations that you can employ to make the unpickling
+environment slightly safer from untrusted pickle data streams; see section
+:ref:`pickle-sub` for more details.
+
+
+.. _pickle-inst:
+
+Pickling and unpickling normal class instances
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. index::
+   single: __getinitargs__() (copy protocol)
+   single: __init__() (instance constructor)
+
+When a pickled class instance is unpickled, its :meth:`__init__` method is
+normally *not* invoked.  If it is desirable that the :meth:`__init__` method be
+called on unpickling, an old-style class can define a method
+:meth:`__getinitargs__`, which should return a *tuple* containing the arguments
+to be passed to the class constructor (:meth:`__init__` for example).  The
+:meth:`__getinitargs__` method is called at pickle time; the tuple it returns is
+incorporated in the pickle for the instance.
+
+.. index:: single: __getnewargs__() (copy protocol)
+
+New-style types can provide a :meth:`__getnewargs__` method that is used for
+protocol 2.  Implementing this method is needed if the type establishes some
+internal invariants when the instance is created, or if the memory allocation is
+affected by the values passed to the :meth:`__new__` method for the type (as it
+is for tuples and strings).  Instances of a new-style type :class:`C` are
+created using ::
+
+   obj = C.__new__(C, *args)
+
+
+where *args* is the result of calling :meth:`__getnewargs__` on the original
+object; if there is no :meth:`__getnewargs__`, an empty tuple is assumed.
+
+.. index::
+   single: __getstate__() (copy protocol)
+   single: __setstate__() (copy protocol)
+   single: __dict__ (instance attribute)
+
+Classes can further influence how their instances are pickled; if the class
+defines the method :meth:`__getstate__`, it is called and the return state is
+pickled as the contents for the instance, instead of the contents of the
+instance's dictionary.  If there is no :meth:`__getstate__` method, the
+instance's :attr:`__dict__` is pickled.
+
+Upon unpickling, if the class also defines the method :meth:`__setstate__`, it
+is called with the unpickled state. [#]_  If there is no :meth:`__setstate__`
+method, the pickled state must be a dictionary and its items are assigned to the
+new instance's dictionary.  If a class defines both :meth:`__getstate__` and
+:meth:`__setstate__`, the state object needn't be a dictionary and these methods
+can do what they want. [#]_
+
+.. warning::
+
+   For new-style classes, if :meth:`__getstate__` returns a false value, the
+   :meth:`__setstate__` method will not be called.
+
+
+Pickling and unpickling extension types
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the :class:`Pickler` encounters an object of a type it knows nothing about
+--- such as an extension type --- it looks in two places for a hint of how to
+pickle it.  One alternative is for the object to implement a :meth:`__reduce__`
+method.  If provided, at pickling time :meth:`__reduce__` will be called with no
+arguments, and it must return either a string or a tuple.
+
+If a string is returned, it names a global variable whose contents are pickled
+as normal.  The string returned by :meth:`__reduce__` should be the object's
+local name relative to its module; the pickle module searches the module
+namespace to determine the object's module.
+
+When a tuple is returned, it must be between two and five elements long.
+Optional elements can either be omitted, or ``None`` can be provided  as their
+value.  The semantics of each element are:
+
+* A callable object that will be called to create the initial version of the
+  object.  The next element of the tuple will provide arguments for this callable,
+  and later elements provide additional state information that will subsequently
+  be used to fully reconstruct the pickled data.
+
+  In the unpickling environment this object must be either a class, a callable
+  registered as a "safe constructor" (see below), or it must have an attribute
+  :attr:`__safe_for_unpickling__` with a true value. Otherwise, an
+  :exc:`UnpicklingError` will be raised in the unpickling environment.  Note that
+  as usual, the callable itself is pickled by name.
+
+* A tuple of arguments for the callable object.
+
+  .. versionchanged:: 2.5
+     Formerly, this argument could also be ``None``.
+
+* Optionally, the object's state, which will be passed to the object's
+  :meth:`__setstate__` method as described in section :ref:`pickle-inst`.  If the
+  object has no :meth:`__setstate__` method, then, as above, the value must be a
+  dictionary and it will be added to the object's :attr:`__dict__`.
+
+* Optionally, an iterator (and not a sequence) yielding successive list items.
+  These list items will be pickled, and appended to the object using either
+  ``obj.append(item)`` or ``obj.extend(list_of_items)``.  This is primarily used
+  for list subclasses, but may be used by other classes as long as they have
+  :meth:`append` and :meth:`extend` methods with the appropriate signature.
+  (Whether :meth:`append` or :meth:`extend` is used depends on which pickle
+  protocol version is used as well as the number of items to append, so both must
+  be supported.)
+
+* Optionally, an iterator (not a sequence) yielding successive dictionary items,
+  which should be tuples of the form ``(key, value)``.  These items will be
+  pickled and stored to the object using ``obj[key] = value``. This is primarily
+  used for dictionary subclasses, but may be used by other classes as long as they
+  implement :meth:`__setitem__`.
+
+It is sometimes useful to know the protocol version when implementing
+:meth:`__reduce__`.  This can be done by implementing a method named
+:meth:`__reduce_ex__` instead of :meth:`__reduce__`. :meth:`__reduce_ex__`, when
+it exists, is called in preference over :meth:`__reduce__` (you may still
+provide :meth:`__reduce__` for backwards compatibility).  The
+:meth:`__reduce_ex__` method will be called with a single integer argument, the
+protocol version.
+
+The :class:`object` class implements both :meth:`__reduce__` and
+:meth:`__reduce_ex__`; however, if a subclass overrides :meth:`__reduce__` but
+not :meth:`__reduce_ex__`, the :meth:`__reduce_ex__` implementation detects this
+and calls :meth:`__reduce__`.
+
+An alternative to implementing a :meth:`__reduce__` method on the object to be
+pickled, is to register the callable with the :mod:`copy_reg` module.  This
+module provides a way for programs to register "reduction functions" and
+constructors for user-defined types.   Reduction functions have the same
+semantics and interface as the :meth:`__reduce__` method described above, except
+that they are called with a single argument, the object to be pickled.
+
+The registered constructor is deemed a "safe constructor" for purposes of
+unpickling as described above.
+
+
+Pickling and unpickling external objects
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For the benefit of object persistence, the :mod:`pickle` module supports the
+notion of a reference to an object outside the pickled data stream.  Such
+objects are referenced by a "persistent id", which is just an arbitrary string
+of printable ASCII characters. The resolution of such names is not defined by
+the :mod:`pickle` module; it will delegate this resolution to user defined
+functions on the pickler and unpickler. [#]_
+
+To define external persistent id resolution, you need to set the
+:attr:`persistent_id` attribute of the pickler object and the
+:attr:`persistent_load` attribute of the unpickler object.
+
+To pickle objects that have an external persistent id, the pickler must have a
+custom :func:`persistent_id` method that takes an object as an argument and
+returns either ``None`` or the persistent id for that object.  When ``None`` is
+returned, the pickler simply pickles the object as normal.  When a persistent id
+string is returned, the pickler will pickle that string, along with a marker so
+that the unpickler will recognize the string as a persistent id.
+
+To unpickle external objects, the unpickler must have a custom
+:func:`persistent_load` function that takes a persistent id string and returns
+the referenced object.
+
+Here's a silly example that *might* shed more light::
+
+   import pickle
+   from cStringIO import StringIO
+
+   src = StringIO()
+   p = pickle.Pickler(src)
+
+   def persistent_id(obj):
+       if hasattr(obj, 'x'):
+           return 'the value %d' % obj.x
+       else:
+           return None
+
+   p.persistent_id = persistent_id
+
+   class Integer:
+       def __init__(self, x):
+           self.x = x
+       def __str__(self):
+           return 'My name is integer %d' % self.x
+
+   i = Integer(7)
+   print i
+   p.dump(i)
+
+   datastream = src.getvalue()
+   print repr(datastream)
+   dst = StringIO(datastream)
+
+   up = pickle.Unpickler(dst)
+
+   class FancyInteger(Integer):
+       def __str__(self):
+           return 'I am the integer %d' % self.x
+
+   def persistent_load(persid):
+       if persid.startswith('the value '):
+           value = int(persid.split()[2])
+           return FancyInteger(value)
+       else:
+           raise pickle.UnpicklingError, 'Invalid persistent id'
+
+   up.persistent_load = persistent_load
+
+   j = up.load()
+   print j
+
+In the :mod:`cPickle` module, the unpickler's :attr:`persistent_load` attribute
+can also be set to a Python list, in which case, when the unpickler reaches a
+persistent id, the persistent id string will simply be appended to this list.
+This functionality exists so that a pickle data stream can be "sniffed" for
+object references without actually instantiating all the objects in a pickle.
+[#]_  Setting :attr:`persistent_load` to a list is usually used in conjunction
+with the :meth:`noload` method on the Unpickler.
+
+.. % BAW: Both pickle and cPickle support something called
+.. % inst_persistent_id() which appears to give unknown types a second
+.. % shot at producing a persistent id.  Since Jim Fulton can't remember
+.. % why it was added or what it's for, I'm leaving it undocumented.
+
+
+.. _pickle-sub:
+
+Subclassing Unpicklers
+----------------------
+
+By default, unpickling will import any class that it finds in the pickle data.
+You can control exactly what gets unpickled and what gets called by customizing
+your unpickler.  Unfortunately, exactly how you do this is different depending
+on whether you're using :mod:`pickle` or :mod:`cPickle`. [#]_
+
+In the :mod:`pickle` module, you need to derive a subclass from
+:class:`Unpickler`, overriding the :meth:`load_global` method.
+:meth:`load_global` should read two lines from the pickle data stream where the
+first line will the name of the module containing the class and the second line
+will be the name of the instance's class.  It then looks up the class, possibly
+importing the module and digging out the attribute, then it appends what it
+finds to the unpickler's stack.  Later on, this class will be assigned to the
+:attr:`__class__` attribute of an empty class, as a way of magically creating an
+instance without calling its class's :meth:`__init__`. Your job (should you
+choose to accept it), would be to have :meth:`load_global` push onto the
+unpickler's stack, a known safe version of any class you deem safe to unpickle.
+It is up to you to produce such a class.  Or you could raise an error if you
+want to disallow all unpickling of instances.  If this sounds like a hack,
+you're right.  Refer to the source code to make this work.
+
+Things are a little cleaner with :mod:`cPickle`, but not by much. To control
+what gets unpickled, you can set the unpickler's :attr:`find_global` attribute
+to a function or ``None``.  If it is ``None`` then any attempts to unpickle
+instances will raise an :exc:`UnpicklingError`.  If it is a function, then it
+should accept a module name and a class name, and return the corresponding class
+object.  It is responsible for looking up the class and performing any necessary
+imports, and it may raise an error to prevent instances of the class from being
+unpickled.
+
+The moral of the story is that you should be really careful about the source of
+the strings your application unpickles.
+
+
+.. _pickle-example:
+
+Example
+-------
+
+For the simplest code, use the :func:`dump` and :func:`load` functions.  Note
+that a self-referencing list is pickled and restored correctly. ::
+
+   import pickle
+
+   data1 = {'a': [1, 2.0, 3, 4+6j],
+            'b': ('string', u'Unicode string'),
+            'c': None}
+
+   selfref_list = [1, 2, 3]
+   selfref_list.append(selfref_list)
+
+   output = open('data.pkl', 'wb')
+
+   # Pickle dictionary using protocol 0.
+   pickle.dump(data1, output)
+
+   # Pickle the list using the highest protocol available.
+   pickle.dump(selfref_list, output, -1)
+
+   output.close()
+
+The following example reads the resulting pickled data.  When reading a
+pickle-containing file, you should open the file in binary mode because you
+can't be sure if the ASCII or binary format was used. ::
+
+   import pprint, pickle
+
+   pkl_file = open('data.pkl', 'rb')
+
+   data1 = pickle.load(pkl_file)
+   pprint.pprint(data1)
+
+   data2 = pickle.load(pkl_file)
+   pprint.pprint(data2)
+
+   pkl_file.close()
+
+Here's a larger example that shows how to modify pickling behavior for a class.
+The :class:`TextReader` class opens a text file, and returns the line number and
+line contents each time its :meth:`readline` method is called. If a
+:class:`TextReader` instance is pickled, all attributes *except* the file object
+member are saved. When the instance is unpickled, the file is reopened, and
+reading resumes from the last location. The :meth:`__setstate__` and
+:meth:`__getstate__` methods are used to implement this behavior. ::
+
+   #!/usr/local/bin/python
+
+   class TextReader:
+       """Print and number lines in a text file."""
+       def __init__(self, file):
+           self.file = file
+           self.fh = open(file)
+           self.lineno = 0
+
+       def readline(self):
+           self.lineno = self.lineno + 1
+           line = self.fh.readline()
+           if not line:
+               return None
+           if line.endswith("\n"):
+               line = line[:-1]
+           return "%d: %s" % (self.lineno, line)
+
+       def __getstate__(self):
+           odict = self.__dict__.copy() # copy the dict since we change it
+           del odict['fh']              # remove filehandle entry
+           return odict
+
+       def __setstate__(self, dict):
+           fh = open(dict['file'])      # reopen file
+           count = dict['lineno']       # read from file...
+           while count:                 # until line count is restored
+               fh.readline()
+               count = count - 1
+           self.__dict__.update(dict)   # update attributes
+           self.fh = fh                 # save the file object
+
+A sample usage might be something like this::
+
+   >>> import TextReader
+   >>> obj = TextReader.TextReader("TextReader.py")
+   >>> obj.readline()
+   '1: #!/usr/local/bin/python'
+   >>> obj.readline()
+   '2: '
+   >>> obj.readline()
+   '3: class TextReader:'
+   >>> import pickle
+   >>> pickle.dump(obj, open('save.p', 'wb'))
+
+If you want to see that :mod:`pickle` works across Python processes, start
+another Python session, before continuing.  What follows can happen from either
+the same process or a new process. ::
+
+   >>> import pickle
+   >>> reader = pickle.load(open('save.p', 'rb'))
+   >>> reader.readline()
+   '4:     """Print and number lines in a text file."""'
+
+
+.. seealso::
+
+   Module :mod:`copy_reg`
+      Pickle interface constructor registration for extension types.
+
+   Module :mod:`shelve`
+      Indexed databases of objects; uses :mod:`pickle`.
+
+   Module :mod:`copy`
+      Shallow and deep object copying.
+
+   Module :mod:`marshal`
+      High-performance serialization of built-in types.
+
+
+:mod:`cPickle` --- A faster :mod:`pickle`
+=========================================
+
+.. module:: cPickle
+   :synopsis: Faster version of pickle, but not subclassable.
+.. moduleauthor:: Jim Fulton <jim@zope.com>
+.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
+
+
+.. index:: module: pickle
+
+The :mod:`cPickle` module supports serialization and de-serialization of Python
+objects, providing an interface and functionality nearly identical to the
+:mod:`pickle` module.  There are several differences, the most important being
+performance and subclassability.
+
+First, :mod:`cPickle` can be up to 1000 times faster than :mod:`pickle` because
+the former is implemented in C.  Second, in the :mod:`cPickle` module the
+callables :func:`Pickler` and :func:`Unpickler` are functions, not classes.
+This means that you cannot use them to derive custom pickling and unpickling
+subclasses.  Most applications have no need for this functionality and should
+benefit from the greatly improved performance of the :mod:`cPickle` module.
+
+The pickle data stream produced by :mod:`pickle` and :mod:`cPickle` are
+identical, so it is possible to use :mod:`pickle` and :mod:`cPickle`
+interchangeably with existing pickles. [#]_
+
+There are additional minor differences in API between :mod:`cPickle` and
+:mod:`pickle`, however for most applications, they are interchangeable.  More
+documentation is provided in the :mod:`pickle` module documentation, which
+includes a list of the documented differences.
+
+.. rubric:: Footnotes
+
+.. [#] Don't confuse this with the :mod:`marshal` module
+
+.. [#] In the :mod:`pickle` module these callables are classes, which you could
+   subclass to customize the behavior.  However, in the :mod:`cPickle` module these
+   callables are factory functions and so cannot be subclassed.  One common reason
+   to subclass is to control what objects can actually be unpickled.  See section
+   :ref:`pickle-sub` for more details.
+
+.. [#] *Warning*: this is intended for pickling multiple objects without intervening
+   modifications to the objects or their parts.  If you modify an object and then
+   pickle it again using the same :class:`Pickler` instance, the object is not
+   pickled again --- a reference to it is pickled and the :class:`Unpickler` will
+   return the old value, not the modified one. There are two problems here: (1)
+   detecting changes, and (2) marshalling a minimal set of changes.  Garbage
+   Collection may also become a problem here.
+
+.. [#] The exception raised will likely be an :exc:`ImportError` or an
+   :exc:`AttributeError` but it could be something else.
+
+.. [#] These methods can also be used to implement copying class instances.
+
+.. [#] This protocol is also used by the shallow and deep copying operations defined in
+   the :mod:`copy` module.
+
+.. [#] The actual mechanism for associating these user defined functions is slightly
+   different for :mod:`pickle` and :mod:`cPickle`.  The description given here
+   works the same for both implementations.  Users of the :mod:`pickle` module
+   could also use subclassing to effect the same results, overriding the
+   :meth:`persistent_id` and :meth:`persistent_load` methods in the derived
+   classes.
+
+.. [#] We'll leave you with the image of Guido and Jim sitting around sniffing pickles
+   in their living rooms.
+
+.. [#] A word of caution: the mechanisms described here use internal attributes and
+   methods, which are subject to change in future versions of Python.  We intend to
+   someday provide a common interface for controlling this behavior, which will
+   work in either :mod:`pickle` or :mod:`cPickle`.
+
+.. [#] Since the pickle data format is actually a tiny stack-oriented programming
+   language, and some freedom is taken in the encodings of certain objects, it is
+   possible that the two modules produce different data streams for the same input
+   objects.  However it is guaranteed that they will always be able to read each
+   other's data streams.
+