diff options
Diffstat (limited to 'Doc/libpickle.tex')
| -rw-r--r-- | Doc/libpickle.tex | 290 |
1 files changed, 0 insertions, 290 deletions
diff --git a/Doc/libpickle.tex b/Doc/libpickle.tex deleted file mode 100644 index edd496f..0000000 --- a/Doc/libpickle.tex +++ /dev/null @@ -1,290 +0,0 @@ -\section{Standard Module \module{pickle}} -\label{module-pickle} -\stmodindex{pickle} -\index{persistency} -\indexii{persistent}{objects} -\indexii{serializing}{objects} -\indexii{marshalling}{objects} -\indexii{flattening}{objects} -\indexii{pickling}{objects} - - -The \module{pickle} module implements a basic but powerful algorithm for -``pickling'' (a.k.a.\ serializing, marshalling or flattening) nearly -arbitrary Python objects. This is the act of converting objects to a -stream of bytes (and back: ``unpickling''). -This is a more primitive notion than -persistency --- although \module{pickle} reads and writes file objects, -it does not handle the issue of naming persistent objects, nor the -(even more complicated) area of concurrent access to persistent -objects. The \module{pickle} module can transform a complex object into -a byte stream and it can transform the byte stream into an object with -the same internal structure. The most obvious thing to do with these -byte streams is to write them onto a file, but it is also conceivable -to send them across a network or store them in a database. The module -\module{shelve} provides a simple interface to pickle and unpickle -objects on ``dbm''-style database files. -\refstmodindex{shelve} - -\strong{Note:} The \module{pickle} module is rather slow. A -reimplementation of the same algorithm in \C{}, which is up to 1000 times -faster, is available as the \module{cPickle}\refbimodindex{cPickle} -module. This has the same interface except that \code{Pickler} and -\code{Unpickler} are factory functions, not classes (so they cannot be -used as base classes for inheritance). - -Unlike the built-in module \module{marshal}, \module{pickle} handles -the following correctly: -\refbimodindex{marshal} - -\begin{itemize} - -\item recursive objects (objects containing references to themselves) - -\item object sharing (references to the same object in different places) - -\item user-defined classes and their instances - -\end{itemize} - -The data format used by \module{pickle} is Python-specific. This has -the advantage that there are no restrictions imposed by external -standards such as XDR% -\index{XDR} -\index{External Data Representation} -(which can't represent pointer sharing); however -it means that non-Python programs may not be able to reconstruct -pickled Python objects. - -By default, the \module{pickle} data format uses a printable \ASCII{} -representation. This is slightly more voluminous than a binary -representation. The big advantage of using printable \ASCII{} (and of -some other characteristics of \module{pickle}'s representation) is that -for debugging or recovery purposes it is possible for a human to read -the pickled file with a standard text editor. - -A binary format, which is slightly more efficient, can be chosen by -specifying a nonzero (true) value for the \var{bin} argument to the -\class{Pickler} constructor or the \function{dump()} and \function{dumps()} -functions. The binary format is not the default because of backwards -compatibility with the Python 1.4 pickle module. In a future version, -the default may change to binary. - -The \module{pickle} module doesn't handle code objects, which the -\module{marshal} module does. I suppose \module{pickle} could, and maybe -it should, but there's probably no great need for it right now (as -long as \module{marshal} continues to be used for reading and writing -code objects), and at least this avoids the possibility of smuggling -Trojan horses into a program. -\refbimodindex{marshal} - -For the benefit of persistency modules written using \module{pickle}, it -supports the notion of a reference to an object outside the pickled -data stream. Such objects are referenced by a name, which is an -arbitrary string of printable \ASCII{} characters. The resolution of -such names is not defined by the \module{pickle} module --- the -persistent object module will have to implement a method -\method{persistent_load()}. To write references to persistent objects, -the persistent module must define a method \method{persistent_id()} which -returns either \code{None} or the persistent ID of the object. - -There are some restrictions on the pickling of class instances. - -First of all, the class must be defined at the top level in a module. -Furthermore, all its instance variables must be picklable. - -\setindexsubitem{(pickle protocol)} - -When a pickled class instance is unpickled, its \method{__init__()} method -is normally \emph{not} invoked. \strong{Note:} This is a deviation -from previous versions of this module; the change was introduced in -Python 1.5b2. The reason for the change is that in many cases it is -desirable to have a constructor that requires arguments; it is a -(minor) nuisance to have to provide a \method{__getinitargs__()} method. - -If it is desirable that the \method{__init__()} method be called on -unpickling, a class can define a method \method{__getinitargs__()}, -which should return a \emph{tuple} containing the arguments to be -passed to the class constructor (\method{__init__()}). This method is -called at pickle time; the tuple it returns is incorporated in the -pickle for the instance. -\ttindex{__getinitargs__()} -\ttindex{__init__()} - -Classes can further influence how their instances are pickled --- if the class -defines the method \method{__getstate__()}, it is called and the return -state is pickled as the contents for the instance, and if the class -defines the method \method{__setstate__()}, it is called with the -unpickled state. (Note that these methods can also be used to -implement copying class instances.) If there is no -\method{__getstate__()} method, the instance's \member{__dict__} is -pickled. If there is no \method{__setstate__()} method, the pickled -object must be a dictionary and its items are assigned to the new -instance's dictionary. (If a class defines both \method{__getstate__()} -and \method{__setstate__()}, the state object needn't be a dictionary ---- these methods can do what they want.) This protocol is also used -by the shallow and deep copying operations defined in the \module{copy} -module.\refstmodindex{copy} -\ttindex{__getstate__()} -\ttindex{__setstate__()} -\ttindex{__dict__} - -Note that when class instances are pickled, their class's code and -data are not pickled along with them. Only the instance data are -pickled. This is done on purpose, so you can fix bugs in a class or -add methods and still load objects that were created with an earlier -version of the class. If you plan to have long-lived objects that -will see many versions of a class, it may be worthwhile to put a version -number in the objects so that suitable conversions can be made by the -class's \method{__setstate__()} method. - -When a class itself is pickled, only its name is pickled --- the class -definition is not pickled, but re-imported by the unpickling process. -Therefore, the restriction that the class must be defined at the top -level in a module applies to pickled classes as well. - -\setindexsubitem{(in module pickle)} - -The interface can be summarized as follows. - -To pickle an object \code{x} onto a file \code{f}, open for writing: - -\begin{verbatim} -p = pickle.Pickler(f) -p.dump(x) -\end{verbatim} - -A shorthand for this is: - -\begin{verbatim} -pickle.dump(x, f) -\end{verbatim} - -To unpickle an object \code{x} from a file \code{f}, open for reading: - -\begin{verbatim} -u = pickle.Unpickler(f) -x = u.load() -\end{verbatim} - -A shorthand is: - -\begin{verbatim} -x = pickle.load(f) -\end{verbatim} - -The \class{Pickler} class only calls the method \code{f.write()} with a -string argument. The \class{Unpickler} calls the methods \code{f.read()} -(with an integer argument) and \code{f.readline()} (without argument), -both returning a string. It is explicitly allowed to pass non-file -objects here, as long as they have the right methods. -\ttindex{Unpickler} -\ttindex{Pickler} - -The constructor for the \class{Pickler} class has an optional second -argument, \var{bin}. If this is present and nonzero, the binary -pickle format is used; if it is zero or absent, the (less efficient, -but backwards compatible) text pickle format is used. The -\class{Unpickler} class does not have an argument to distinguish -between binary and text pickle formats; it accepts either format. - -The following types can be pickled: -\begin{itemize} - -\item \code{None} - -\item integers, long integers, floating point numbers - -\item strings - -\item tuples, lists and dictionaries containing only picklable objects - -\item classes that are defined at the top level in a module - -\item instances of such classes whose \member{__dict__} or -\method{__setstate__()} is picklable - -\end{itemize} - -Attempts to pickle unpicklable objects will raise the -\exception{PicklingError} exception; when this happens, an unspecified -number of bytes may have been written to the file. - -It is possible to make multiple calls to the \method{dump()} method of -the same \class{Pickler} instance. These must then be matched to the -same number of calls to the \method{load()} method of the -corresponding \class{Unpickler} instance. If the same object is -pickled by multiple \method{dump()} calls, the \method{load()} will all -yield references to the same object. \emph{Warning}: this is intended -for pickling multiple objects without intervening modifications to the -objects or their parts. If you modify an object and then pickle it -again using the same \class{Pickler} instance, the object is not -pickled again --- a reference to it is pickled and the -\class{Unpickler} will return the old value, not the modified one. -(There are two problems here: (a) detecting changes, and (b) -marshalling a minimal set of changes. I have no answers. Garbage -Collection may also become a problem here.) - -Apart from the \class{Pickler} and \class{Unpickler} classes, the -module defines the following functions, and an exception: - -\begin{funcdesc}{dump}{object, file\optional{, bin}} -Write a pickled representation of \var{obect} to the open file object -\var{file}. This is equivalent to -\samp{Pickler(\var{file}, \var{bin}).dump(\var{object})}. -If the optional \var{bin} argument is present and nonzero, the binary -pickle format is used; if it is zero or absent, the (less efficient) -text pickle format is used. -\end{funcdesc} - -\begin{funcdesc}{load}{file} -Read a pickled object from the open file object \var{file}. This is -equivalent to \samp{Unpickler(\var{file}).load()}. -\end{funcdesc} - -\begin{funcdesc}{dumps}{object\optional{, bin}} -Return the pickled representation of the object as a string, instead -of writing it to a file. If the optional \var{bin} argument is -present and nonzero, the binary pickle format is used; if it is zero -or absent, the (less efficient) text pickle format is used. -\end{funcdesc} - -\begin{funcdesc}{loads}{string} -Read a pickled object from a string instead of a file. Characters in -the string past the pickled object's representation are ignored. -\end{funcdesc} - -\begin{excdesc}{PicklingError} -This exception is raised when an unpicklable object is passed to -\code{Pickler.dump()}. -\end{excdesc} - - -\begin{seealso} -\seemodule[copyreg]{copy_reg}{pickle interface constructor -registration} - -\seemodule{shelve}{indexed databases of objects; uses \module{pickle}} - -\seemodule{copy}{shallow and deep object copying} - -\seemodule{marshal}{high-performance serialization of built-in types} -\end{seealso} - - -\section{Built-in Module \module{cPickle}} -\bimodindex{cPickle} -\label{module-cPickle} - -% This section was written by Fred L. Drake, Jr. <fdrake@acm.org> - -The \module{cPickle} module provides a similar interface and identical -functionality as the \module{pickle} module, but can be up to 1000 -times faster since it is implemented in \C{}. The only other -important difference to note is that \function{Pickler()} and -\function{Unpickler()} are functions and not classes, and so cannot be -subclassed. This should not be an issue in most cases. - -The format of the pickle data is identical to that produced using the -\module{pickle} module, so it is possible to use \module{pickle} and -\module{cPickle} interchangably with existing pickles. |