diff options
| -rw-r--r-- | Doc/library/pickle.rst | 106 |
1 files changed, 79 insertions, 27 deletions
diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst index 9a31520..0a958a7 100644 --- a/Doc/library/pickle.rst +++ b/Doc/library/pickle.rst @@ -14,6 +14,7 @@ .. sectionauthor:: Jim Kerr <jbkerr@sr.hp.com>. .. sectionauthor:: Barry Warsaw <barry@zope.com> + The :mod:`pickle` module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. "Pickling" is the process whereby a Python object hierarchy is converted into a byte stream, and @@ -282,23 +283,9 @@ The :mod:`pickle` module exports two classes, :class:`Pickler` and .. attribute:: memo - Dictionary holding previously pickled objects to allow shared or - recursive objects to pickled by reference as opposed to by value. - - -.. XXX Move these comments to somewhere more appropriate. - -It is possible to make multiple calls to the :meth:`dump` method of the same -:class:`Pickler` instance. These must then be matched to the same number of -calls to the :meth:`load` method of the corresponding :class:`Unpickler` -instance. If the same object is pickled by multiple :meth:`dump` calls, the -:meth:`load` will all yield references to the same object. - -Please note, this is intended for pickling multiple objects without intervening -modifications to the objects or their parts. If you modify an object and then -pickle it again using the same :class:`Pickler` instance, the object is not -pickled again --- a reference to it is pickled and the :class:`Unpickler` will -return the old value, not the modified one. + Dictionary-like object holding previously pickled objects to allow + shared or recursive objects to pickled by reference as opposed to + by value. .. class:: Unpickler(file, [\*, encoding="ASCII", errors="strict"]) @@ -345,6 +332,11 @@ return the old value, not the modified one. how they can be loaded, potentially reducing security risks. Refer to :ref:`pickle-restrict` for details. + .. attribute:: memo + + Dictionary holding previously unpickled objects to allow shared or + recursive objects to unpickled by reference as opposed to by value. + .. _pickle-picklable: @@ -593,7 +585,7 @@ reading resumes from the last location. The :meth:`__setstate__` and line = self.file.readline() if not line: return None - if line.endswith("\n"): + if line.endswith('\n'): line = line[:-1] return "%i: %s" % (self.lineno, line) @@ -707,25 +699,24 @@ A sample usage of our unpickler working has intended:: As our examples shows, you have to be careful with what you allow to be unpickled. Therefore if security is a concern, you may want to consider -alternatives such as the marshalling API in :mod:`xmlrpc.client` or third-party -solutions. +alternatives such as the marshalling API in :mod:`xmlrpc.client` or +third-party solutions. .. _pickle-example: -Usage Examples --------------- +Examples +-------- -For the simplest code, use the :func:`dump` and :func:`load` functions. Note -that a self-referencing list is pickled and restored correctly. :: +For the simplest code, use the :func:`dump` and :func:`load` functions. :: import pickle # An arbitrary collection of objects supported by pickle. data = { - 'a': [1, 2.0, 3, 4+6j], - 'b': ("character string", b"byte string"), - 'c': set([None, True, False]) + 'a': [1, 2.0, 3, 4+6j], + 'b': ("character string", b"byte string"), + 'c': set([None, True, False]) } with open('data.pickle', 'wb') as f: @@ -743,11 +734,72 @@ The following example reads the resulting pickled data. :: data = pickle.load(f) +Reusing Pickler Instances +^^^^^^^^^^^^^^^^^^^^^^^^^ + +It is possible to make multiple calls to the :meth:`dump` method of the same +:class:`Pickler` instance. These must then be matched to the same number of +calls to the :meth:`load` method of the corresponding :class:`Unpickler` +instance. If the same object is pickled by multiple :meth:`dump` calls, the +:meth:`load` will all yield references to the same object. + +Please note, this is intended for pickling multiple objects without intervening +modifications to the objects or their parts. If you modify an object and then +pickle it again using the same :class:`Pickler` instance, the object is not +pickled again --- a reference to it is pickled and the :class:`Unpickler` will +return the old value, not the modified one. :: + + import io + import pickle + + data = {"hello": 0, "spam": 1} + + # Create a binary file-like object to which the Pickler instance will + # write the pickles. + f = io.BytesIO() + p = pickle.Pickler(f) + p.dump(data) + + # This second call appends a new pickle to the file. The modification we + # make is lost because objects are pickled by reference when seen again. + data["eggs"] = 2 + p.dump(data) + + # Now, we load the pickles saved in our file-like object. + f.seek(0) + u = pickle.Unpickler(f) + data1 = u.load() + data2 = u.load() + + if data1 is data2: + print("data1 and data2 are the same object") + else: + print("data1 and data2 are not the same object") + + if "eggs" in data2: + print("The modification was pickled.") + else: + print("The modification was not pickled.") + +Reusing a :class:`Pickler` instance like we shown can be a useful +optimization. For example, a multi-process application could use this feature +to reduce the size of the pickles transmitted across processes over time +(assuming the pickles exchanged are containers sharing common immutable +objects). However, you should take special care to regularly clear +:attr:``Pickler.memo`` and :attr:``Unpickler.memo`` to avoid memory-leaks. + +.. XXX: Add examples showing how to optimize pickles for size (like using +.. pickletools.optimize() or the gzip module). + + .. seealso:: Module :mod:`copyreg` Pickle interface constructor registration for extension types. + Module :mod:`pickletools` + Tools for working with and analyzing pickled data. + Module :mod:`shelve` Indexed databases of objects; uses :mod:`pickle`. |