diff options
Diffstat (limited to 'Doc')
| -rw-r--r-- | Doc/Makefile.deps | 7 | ||||
| -rw-r--r-- | Doc/lib/compiler.tex | 353 | ||||
| -rw-r--r-- | Doc/lib/lib.tex | 9 | ||||
| -rw-r--r-- | Doc/lib/libbastion.tex | 57 | ||||
| -rw-r--r-- | Doc/lib/libjpeg.tex | 80 | ||||
| -rw-r--r-- | Doc/lib/libmd5.tex | 92 | ||||
| -rw-r--r-- | Doc/lib/libpanel.tex | 74 | ||||
| -rw-r--r-- | Doc/lib/librestricted.tex | 66 | ||||
| -rw-r--r-- | Doc/lib/librexec.tex | 275 | ||||
| -rw-r--r-- | Doc/lib/libsha.tex | 83 | ||||
| -rw-r--r-- | Doc/ref/ref3.tex | 9 |
11 files changed, 3 insertions, 1102 deletions
diff --git a/Doc/Makefile.deps b/Doc/Makefile.deps index 426c7ea1..0246653 100644 --- a/Doc/Makefile.deps +++ b/Doc/Makefile.deps @@ -88,7 +88,6 @@ LIBFILES= $(MANSTYLES) $(INDEXSTYLES) $(COMMONTEX) \ commontex/reportingbugs.tex \ lib/lib.tex \ lib/asttable.tex \ - lib/compiler.tex \ lib/distutils.tex \ lib/email.tex \ lib/emailencoders.tex \ @@ -200,21 +199,15 @@ LIBFILES= $(MANSTYLES) $(INDEXSTYLES) $(COMMONTEX) \ lib/libaudioop.tex \ lib/libimageop.tex \ lib/libaifc.tex \ - lib/libjpeg.tex \ lib/librgbimg.tex \ lib/libossaudiodev.tex \ lib/libcrypto.tex \ lib/libhashlib.tex \ - lib/libmd5.tex \ - lib/libsha.tex \ lib/libhmac.tex \ lib/libstdwin.tex \ lib/libsun.tex \ lib/libxdrlib.tex \ lib/libimghdr.tex \ - lib/librestricted.tex \ - lib/librexec.tex \ - lib/libbastion.tex \ lib/libformatter.tex \ lib/liboperator.tex \ lib/libresource.tex \ diff --git a/Doc/lib/compiler.tex b/Doc/lib/compiler.tex deleted file mode 100644 index d4f4124..0000000 --- a/Doc/lib/compiler.tex +++ /dev/null @@ -1,353 +0,0 @@ -\chapter{Python compiler package \label{compiler}} - -\sectionauthor{Jeremy Hylton}{jeremy@zope.com} - - -The Python compiler package is a tool for analyzing Python source code -and generating Python bytecode. The compiler contains libraries to -generate an abstract syntax tree from Python source code and to -generate Python bytecode from the tree. - -The \refmodule{compiler} package is a Python source to bytecode -translator written in Python. It uses the built-in parser and -standard \refmodule{parser} module to generated a concrete syntax -tree. This tree is used to generate an abstract syntax tree (AST) and -then Python bytecode. - -The full functionality of the package duplicates the builtin compiler -provided with the Python interpreter. It is intended to match its -behavior almost exactly. Why implement another compiler that does the -same thing? The package is useful for a variety of purposes. It can -be modified more easily than the builtin compiler. The AST it -generates is useful for analyzing Python source code. - -This chapter explains how the various components of the -\refmodule{compiler} package work. It blends reference material with -a tutorial. - -The following modules are part of the \refmodule{compiler} package: - -\localmoduletable - - -\section{The basic interface} - -\declaremodule{}{compiler} - -The top-level of the package defines four functions. If you import -\module{compiler}, you will get these functions and a collection of -modules contained in the package. - -\begin{funcdesc}{parse}{buf} -Returns an abstract syntax tree for the Python source code in \var{buf}. -The function raises \exception{SyntaxError} if there is an error in the -source code. The return value is a \class{compiler.ast.Module} instance -that contains the tree. -\end{funcdesc} - -\begin{funcdesc}{parseFile}{path} -Return an abstract syntax tree for the Python source code in the file -specified by \var{path}. It is equivalent to -\code{parse(open(\var{path}).read())}. -\end{funcdesc} - -\begin{funcdesc}{walk}{ast, visitor\optional{, verbose}} -Do a pre-order walk over the abstract syntax tree \var{ast}. Call the -appropriate method on the \var{visitor} instance for each node -encountered. -\end{funcdesc} - -\begin{funcdesc}{compile}{source, filename, mode, flags=None, - dont_inherit=None} -Compile the string \var{source}, a Python module, statement or -expression, into a code object that can be executed by the exec -statement or \function{eval()}. This function is a replacement for the -built-in \function{compile()} function. - -The \var{filename} will be used for run-time error messages. - -The \var{mode} must be 'exec' to compile a module, 'single' to compile a -single (interactive) statement, or 'eval' to compile an expression. - -The \var{flags} and \var{dont_inherit} arguments affect future-related -statements, but are not supported yet. -\end{funcdesc} - -\begin{funcdesc}{compileFile}{source} -Compiles the file \var{source} and generates a .pyc file. -\end{funcdesc} - -The \module{compiler} package contains the following modules: -\refmodule[compiler.ast]{ast}, \module{consts}, \module{future}, -\module{misc}, \module{pyassem}, \module{pycodegen}, \module{symbols}, -\module{transformer}, and \refmodule[compiler.visitor]{visitor}. - -\section{Limitations} - -There are some problems with the error checking of the compiler -package. The interpreter detects syntax errors in two distinct -phases. One set of errors is detected by the interpreter's parser, -the other set by the compiler. The compiler package relies on the -interpreter's parser, so it get the first phases of error checking for -free. It implements the second phase itself, and that implementation is -incomplete. For example, the compiler package does not raise an error -if a name appears more than once in an argument list: -\code{def f(x, x): ...} - -A future version of the compiler should fix these problems. - -\section{Python Abstract Syntax} - -The \module{compiler.ast} module defines an abstract syntax for -Python. In the abstract syntax tree, each node represents a syntactic -construct. The root of the tree is \class{Module} object. - -The abstract syntax offers a higher level interface to parsed Python -source code. The \refmodule{parser} -module and the compiler written in C for the Python interpreter use a -concrete syntax tree. The concrete syntax is tied closely to the -grammar description used for the Python parser. Instead of a single -node for a construct, there are often several levels of nested nodes -that are introduced by Python's precedence rules. - -The abstract syntax tree is created by the -\module{compiler.transformer} module. The transformer relies on the -builtin Python parser to generate a concrete syntax tree. It -generates an abstract syntax tree from the concrete tree. - -The \module{transformer} module was created by Greg -Stein\index{Stein, Greg} and Bill Tutt\index{Tutt, Bill} for an -experimental Python-to-C compiler. The current version contains a -number of modifications and improvements, but the basic form of the -abstract syntax and of the transformer are due to Stein and Tutt. - -\subsection{AST Nodes} - -\declaremodule{}{compiler.ast} - -The \module{compiler.ast} module is generated from a text file that -describes each node type and its elements. Each node type is -represented as a class that inherits from the abstract base class -\class{compiler.ast.Node} and defines a set of named attributes for -child nodes. - -\begin{classdesc}{Node}{} - - The \class{Node} instances are created automatically by the parser - generator. The recommended interface for specific \class{Node} - instances is to use the public attributes to access child nodes. A - public attribute may be bound to a single node or to a sequence of - nodes, depending on the \class{Node} type. For example, the - \member{bases} attribute of the \class{Class} node, is bound to a - list of base class nodes, and the \member{doc} attribute is bound to - a single node. - - Each \class{Node} instance has a \member{lineno} attribute which may - be \code{None}. XXX Not sure what the rules are for which nodes - will have a useful lineno. -\end{classdesc} - -All \class{Node} objects offer the following methods: - -\begin{methoddesc}{getChildren}{} - Returns a flattened list of the child nodes and objects in the - order they occur. Specifically, the order of the nodes is the - order in which they appear in the Python grammar. Not all of the - children are \class{Node} instances. The names of functions and - classes, for example, are plain strings. -\end{methoddesc} - -\begin{methoddesc}{getChildNodes}{} - Returns a flattened list of the child nodes in the order they - occur. This method is like \method{getChildren()}, except that it - only returns those children that are \class{Node} instances. -\end{methoddesc} - -Two examples illustrate the general structure of \class{Node} -classes. The \keyword{while} statement is defined by the following -grammar production: - -\begin{verbatim} -while_stmt: "while" expression ":" suite - ["else" ":" suite] -\end{verbatim} - -The \class{While} node has three attributes: \member{test}, -\member{body}, and \member{else_}. (If the natural name for an -attribute is also a Python reserved word, it can't be used as an -attribute name. An underscore is appended to the word to make it a -legal identifier, hence \member{else_} instead of \keyword{else}.) - -The \keyword{if} statement is more complicated because it can include -several tests. - -\begin{verbatim} -if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] -\end{verbatim} - -The \class{If} node only defines two attributes: \member{tests} and -\member{else_}. The \member{tests} attribute is a sequence of test -expression, consequent body pairs. There is one pair for each -\keyword{if}/\keyword{elif} clause. The first element of the pair is -the test expression. The second elements is a \class{Stmt} node that -contains the code to execute if the test is true. - -The \method{getChildren()} method of \class{If} returns a flat list of -child nodes. If there are three \keyword{if}/\keyword{elif} clauses -and no \keyword{else} clause, then \method{getChildren()} will return -a list of six elements: the first test expression, the first -\class{Stmt}, the second text expression, etc. - -The following table lists each of the \class{Node} subclasses defined -in \module{compiler.ast} and each of the public attributes available -on their instances. The values of most of the attributes are -themselves \class{Node} instances or sequences of instances. When the -value is something other than an instance, the type is noted in the -comment. The attributes are listed in the order in which they are -returned by \method{getChildren()} and \method{getChildNodes()}. - -\input{asttable} - - -\subsection{Assignment nodes} - -There is a collection of nodes used to represent assignments. Each -assignment statement in the source code becomes a single -\class{Assign} node in the AST. The \member{nodes} attribute is a -list that contains a node for each assignment target. This is -necessary because assignment can be chained, e.g. \code{a = b = 2}. -Each \class{Node} in the list will be one of the following classes: -\class{AssAttr}, \class{AssList}, \class{AssName}, or -\class{AssTuple}. - -Each target assignment node will describe the kind of object being -assigned to: \class{AssName} for a simple name, e.g. \code{a = 1}. -\class{AssAttr} for an attribute assigned, e.g. \code{a.x = 1}. -\class{AssList} and \class{AssTuple} for list and tuple expansion -respectively, e.g. \code{a, b, c = a_tuple}. - -The target assignment nodes also have a \member{flags} attribute that -indicates whether the node is being used for assignment or in a delete -statement. The \class{AssName} is also used to represent a delete -statement, e.g. \class{del x}. - -When an expression contains several attribute references, an -assignment or delete statement will contain only one \class{AssAttr} -node -- for the final attribute reference. The other attribute -references will be represented as \class{Getattr} nodes in the -\member{expr} attribute of the \class{AssAttr} instance. - -\subsection{Examples} - -This section shows several simple examples of ASTs for Python source -code. The examples demonstrate how to use the \function{parse()} -function, what the repr of an AST looks like, and how to access -attributes of an AST node. - -The first module defines a single function. Assume it is stored in -\file{/tmp/doublelib.py}. - -\begin{verbatim} -"""This is an example module. - -This is the docstring. -""" - -def double(x): - "Return twice the argument" - return x * 2 -\end{verbatim} - -In the interactive interpreter session below, I have reformatted the -long AST reprs for readability. The AST reprs use unqualified class -names. If you want to create an instance from a repr, you must import -the class names from the \module{compiler.ast} module. - -\begin{verbatim} ->>> import compiler ->>> mod = compiler.parseFile("/tmp/doublelib.py") ->>> mod -Module('This is an example module.\n\nThis is the docstring.\n', - Stmt([Function(None, 'double', ['x'], [], 0, - 'Return twice the argument', - Stmt([Return(Mul((Name('x'), Const(2))))]))])) ->>> from compiler.ast import * ->>> Module('This is an example module.\n\nThis is the docstring.\n', -... Stmt([Function(None, 'double', ['x'], [], 0, -... 'Return twice the argument', -... Stmt([Return(Mul((Name('x'), Const(2))))]))])) -Module('This is an example module.\n\nThis is the docstring.\n', - Stmt([Function(None, 'double', ['x'], [], 0, - 'Return twice the argument', - Stmt([Return(Mul((Name('x'), Const(2))))]))])) ->>> mod.doc -'This is an example module.\n\nThis is the docstring.\n' ->>> for node in mod.node.nodes: -... print node -... -Function(None, 'double', ['x'], [], 0, 'Return twice the argument', - Stmt([Return(Mul((Name('x'), Const(2))))])) ->>> func = mod.node.nodes[0] ->>> func.code -Stmt([Return(Mul((Name('x'), Const(2))))]) -\end{verbatim} - -\section{Using Visitors to Walk ASTs} - -\declaremodule{}{compiler.visitor} - -The visitor pattern is ... The \refmodule{compiler} package uses a -variant on the visitor pattern that takes advantage of Python's -introspection features to eliminate the need for much of the visitor's -infrastructure. - -The classes being visited do not need to be programmed to accept -visitors. The visitor need only define visit methods for classes it -is specifically interested in; a default visit method can handle the -rest. - -XXX The magic \method{visit()} method for visitors. - -\begin{funcdesc}{walk}{tree, visitor\optional{, verbose}} -\end{funcdesc} - -\begin{classdesc}{ASTVisitor}{} - -The \class{ASTVisitor} is responsible for walking over the tree in the -correct order. A walk begins with a call to \method{preorder()}. For -each node, it checks the \var{visitor} argument to \method{preorder()} -for a method named `visitNodeType,' where NodeType is the name of the -node's class, e.g. for a \class{While} node a \method{visitWhile()} -would be called. If the method exists, it is called with the node as -its first argument. - -The visitor method for a particular node type can control how child -nodes are visited during the walk. The \class{ASTVisitor} modifies -the visitor argument by adding a visit method to the visitor; this -method can be used to visit a particular child node. If no visitor is -found for a particular node type, the \method{default()} method is -called. -\end{classdesc} - -\class{ASTVisitor} objects have the following methods: - -XXX describe extra arguments - -\begin{methoddesc}{default}{node\optional{, \moreargs}} -\end{methoddesc} - -\begin{methoddesc}{dispatch}{node\optional{, \moreargs}} -\end{methoddesc} - -\begin{methoddesc}{preorder}{tree, visitor} -\end{methoddesc} - - -\section{Bytecode Generation} - -The code generator is a visitor that emits bytecodes. Each visit method -can call the \method{emit()} method to emit a new bytecode. The basic -code generator is specialized for modules, classes, and functions. An -assembler converts that emitted instructions to the low-level bytecode -format. It handles things like generator of constant lists of code -objects and calculation of jump offsets. diff --git a/Doc/lib/lib.tex b/Doc/lib/lib.tex index aa4d3e8..d87cd5e 100644 --- a/Doc/lib/lib.tex +++ b/Doc/lib/lib.tex @@ -182,8 +182,6 @@ and how to embed it in other applications. \input{libcrypto} % Cryptographic Services \input{libhashlib} \input{libhmac} -\input{libmd5} -\input{libsha} % ============= % FILE & DATABASE STORAGE @@ -388,9 +386,6 @@ and how to embed it in other applications. \input{custominterp} % Custom interpreter \input{libcode} \input{libcodeop} -\input{librestricted} % Restricted Execution -\input{librexec} -\input{libbastion} \input{modules} % Importing Modules @@ -419,7 +414,6 @@ and how to embed it in other applications. \input{libpickletools} \input{distutils} -\input{compiler} % compiler package \input{libast} \input{libmisc} % Miscellaneous Services @@ -434,9 +428,6 @@ and how to embed it in other applications. %\input{libstdwin} % STDWIN ONLY -\input{libjpeg} -%\input{libpanel} - \input{libsun} % SUNOS ONLY \input{libsunaudio} % XXX(nnorwitz): the modules below this comment should be kept. diff --git a/Doc/lib/libbastion.tex b/Doc/lib/libbastion.tex deleted file mode 100644 index 9f45c47..0000000 --- a/Doc/lib/libbastion.tex +++ /dev/null @@ -1,57 +0,0 @@ -\section{\module{Bastion} --- - Restricting access to objects} - -\declaremodule{standard}{Bastion} -\modulesynopsis{Providing restricted access to objects.} -\moduleauthor{Barry Warsaw}{bwarsaw@python.org} -\versionchanged[Disabled module]{2.3} - -\begin{notice}[warning] - The documentation has been left in place to help in reading old code - that uses the module. -\end{notice} - -% I'm concerned that the word 'bastion' won't be understood by people -% for whom English is a second language, making the module name -% somewhat mysterious. Thus, the brief definition... --amk - -According to the dictionary, a bastion is ``a fortified area or -position'', or ``something that is considered a stronghold.'' It's a -suitable name for this module, which provides a way to forbid access -to certain attributes of an object. It must always be used with the -\refmodule{rexec} module, in order to allow restricted-mode programs -access to certain safe attributes of an object, while denying access -to other, unsafe attributes. - -% I've punted on the issue of documenting keyword arguments for now. - -\begin{funcdesc}{Bastion}{object\optional{, filter\optional{, - name\optional{, class}}}} -Protect the object \var{object}, returning a bastion for the -object. Any attempt to access one of the object's attributes will -have to be approved by the \var{filter} function; if the access is -denied an \exception{AttributeError} exception will be raised. - -If present, \var{filter} must be a function that accepts a string -containing an attribute name, and returns true if access to that -attribute will be permitted; if \var{filter} returns false, the access -is denied. The default filter denies access to any function beginning -with an underscore (\character{_}). The bastion's string representation -will be \samp{<Bastion for \var{name}>} if a value for -\var{name} is provided; otherwise, \samp{repr(\var{object})} will be -used. - -\var{class}, if present, should be a subclass of \class{BastionClass}; -see the code in \file{bastion.py} for the details. Overriding the -default \class{BastionClass} will rarely be required. -\end{funcdesc} - - -\begin{classdesc}{BastionClass}{getfunc, name} -Class which actually implements bastion objects. This is the default -class used by \function{Bastion()}. The \var{getfunc} parameter is a -function which returns the value of an attribute which should be -exposed to the restricted execution environment when called with the -name of the attribute as the only parameter. \var{name} is used to -construct the \function{repr()} of the \class{BastionClass} instance. -\end{classdesc} diff --git a/Doc/lib/libjpeg.tex b/Doc/lib/libjpeg.tex deleted file mode 100644 index a10e06c..0000000 --- a/Doc/lib/libjpeg.tex +++ /dev/null @@ -1,80 +0,0 @@ -\section{\module{jpeg} --- - Read and write JPEG files} - -\declaremodule{builtin}{jpeg} - \platform{IRIX} -\modulesynopsis{Read and write image files in compressed JPEG format.} - - -The module \module{jpeg} provides access to the jpeg compressor and -decompressor written by the Independent JPEG Group -\index{Independent JPEG Group}(IJG). JPEG is a standard for -compressing pictures; it is defined in ISO 10918. For details on JPEG -or the Independent JPEG Group software refer to the JPEG standard or -the documentation provided with the software. - -A portable interface to JPEG image files is available with the Python -Imaging Library (PIL) by Fredrik Lundh. Information on PIL is -available at \url{http://www.pythonware.com/products/pil/}. -\index{Python Imaging Library} -\index{PIL (the Python Imaging Library)} -\index{Lundh, Fredrik} - -The \module{jpeg} module defines an exception and some functions. - -\begin{excdesc}{error} -Exception raised by \function{compress()} and \function{decompress()} -in case of errors. -\end{excdesc} - -\begin{funcdesc}{compress}{data, w, h, b} -Treat data as a pixmap of width \var{w} and height \var{h}, with -\var{b} bytes per pixel. The data is in SGI GL order, so the first -pixel is in the lower-left corner. This means that \function{gl.lrectread()} -return data can immediately be passed to \function{compress()}. -Currently only 1 byte and 4 byte pixels are allowed, the former being -treated as greyscale and the latter as RGB color. -\function{compress()} returns a string that contains the compressed -picture, in JFIF\index{JFIF} format. -\end{funcdesc} - -\begin{funcdesc}{decompress}{data} -Data is a string containing a picture in JFIF\index{JFIF} format. It -returns a tuple \code{(\var{data}, \var{width}, \var{height}, -\var{bytesperpixel})}. Again, the data is suitable to pass to -\function{gl.lrectwrite()}. -\end{funcdesc} - -\begin{funcdesc}{setoption}{name, value} -Set various options. Subsequent \function{compress()} and -\function{decompress()} calls will use these options. The following -options are available: - -\begin{tableii}{l|p{3in}}{code}{Option}{Effect} - \lineii{'forcegray'}{% - Force output to be grayscale, even if input is RGB.} - \lineii{'quality'}{% - Set the quality of the compressed image to a value between - \code{0} and \code{100} (default is \code{75}). This only affects - compression.} - \lineii{'optimize'}{% - Perform Huffman table optimization. Takes longer, but results in - smaller compressed image. This only affects compression.} - \lineii{'smooth'}{% - Perform inter-block smoothing on uncompressed image. Only useful - for low-quality images. This only affects decompression.} -\end{tableii} -\end{funcdesc} - - -\begin{seealso} - \seetitle{JPEG Still Image Data Compression Standard}{The - canonical reference for the JPEG image format, by - Pennebaker and Mitchell.} - - \seetitle[http://www.w3.org/Graphics/JPEG/itu-t81.pdf]{Information - Technology - Digital Compression and Coding of - Continuous-tone Still Images - Requirements and - Guidelines}{The ISO standard for JPEG is also published as - ITU T.81. This is available online in PDF form.} -\end{seealso} diff --git a/Doc/lib/libmd5.tex b/Doc/lib/libmd5.tex deleted file mode 100644 index 38105ae..0000000 --- a/Doc/lib/libmd5.tex +++ /dev/null @@ -1,92 +0,0 @@ -\section{\module{md5} --- - MD5 message digest algorithm} - -\declaremodule{builtin}{md5} -\modulesynopsis{RSA's MD5 message digest algorithm.} - -\deprecated{2.5}{Use the \refmodule{hashlib} module instead.} - -This module implements the interface to RSA's MD5 message digest -\index{message digest, MD5} -algorithm (see also Internet \rfc{1321}). Its use is quite -straightforward:\ use \function{new()} to create an md5 object. -You can now feed this object with arbitrary strings using the -\method{update()} method, and at any point you can ask it for the -\dfn{digest} (a strong kind of 128-bit checksum, -a.k.a. ``fingerprint'') of the concatenation of the strings fed to it -so far using the \method{digest()} method. -\index{checksum!MD5} - -For example, to obtain the digest of the string \code{'Nobody inspects -the spammish repetition'}: - -\begin{verbatim} ->>> import md5 ->>> m = md5.new() ->>> m.update("Nobody inspects") ->>> m.update(" the spammish repetition") ->>> m.digest() -'\xbbd\x9c\x83\xdd\x1e\xa5\xc9\xd9\xde\xc9\xa1\x8d\xf0\xff\xe9' -\end{verbatim} - -More condensed: - -\begin{verbatim} ->>> md5.new("Nobody inspects the spammish repetition").digest() -'\xbbd\x9c\x83\xdd\x1e\xa5\xc9\xd9\xde\xc9\xa1\x8d\xf0\xff\xe9' -\end{verbatim} - -The following values are provided as constants in the module and as -attributes of the md5 objects returned by \function{new()}: - -\begin{datadesc}{digest_size} - The size of the resulting digest in bytes. This is always - \code{16}. -\end{datadesc} - -The md5 module provides the following functions: - -\begin{funcdesc}{new}{\optional{arg}} -Return a new md5 object. If \var{arg} is present, the method call -\code{update(\var{arg})} is made. -\end{funcdesc} - -\begin{funcdesc}{md5}{\optional{arg}} -For backward compatibility reasons, this is an alternative name for the -\function{new()} function. -\end{funcdesc} - -An md5 object has the following methods: - -\begin{methoddesc}[md5]{update}{arg} -Update the md5 object with the string \var{arg}. Repeated calls are -equivalent to a single call with the concatenation of all the -arguments: \code{m.update(a); m.update(b)} is equivalent to -\code{m.update(a+b)}. -\end{methoddesc} - -\begin{methoddesc}[md5]{digest}{} -Return the digest of the strings passed to the \method{update()} -method so far. This is a 16-byte string which may contain -non-\ASCII{} characters, including null bytes. -\end{methoddesc} - -\begin{methoddesc}[md5]{hexdigest}{} -Like \method{digest()} except the digest is returned as a string of -length 32, containing only hexadecimal digits. This may -be used to exchange the value safely in email or other non-binary -environments. -\end{methoddesc} - -\begin{methoddesc}[md5]{copy}{} -Return a copy (``clone'') of the md5 object. This can be used to -efficiently compute the digests of strings that share a common initial -substring. -\end{methoddesc} - - -\begin{seealso} - \seemodule{sha}{Similar module implementing the Secure Hash - Algorithm (SHA). The SHA algorithm is considered a - more secure hash.} -\end{seealso} diff --git a/Doc/lib/libpanel.tex b/Doc/lib/libpanel.tex deleted file mode 100644 index f2db0b0..0000000 --- a/Doc/lib/libpanel.tex +++ /dev/null @@ -1,74 +0,0 @@ -\section{\module{panel} --- - None} -\declaremodule{standard}{panel} - -\modulesynopsis{None} - - -\strong{Please note:} The FORMS library, to which the -\code{fl}\refbimodindex{fl} module described above interfaces, is a -simpler and more accessible user interface library for use with GL -than the \code{panel} module (besides also being by a Dutch author). - -This module should be used instead of the built-in module -\code{pnl}\refbimodindex{pnl} -to interface with the -\emph{Panel Library}. - -The module is too large to document here in its entirety. -One interesting function: - -\begin{funcdesc}{defpanellist}{filename} -Parses a panel description file containing S-expressions written by the -\emph{Panel Editor} -that accompanies the Panel Library and creates the described panels. -It returns a list of panel objects. -\end{funcdesc} - -\warning{The Python interpreter will dump core if you don't create a -GL window before calling -\code{panel.mkpanel()} -or -\code{panel.defpanellist()}.} - -\section{\module{panelparser} --- - None} -\declaremodule{standard}{panelparser} - -\modulesynopsis{None} - - -This module defines a self-contained parser for S-expressions as output -by the Panel Editor (which is written in Scheme so it can't help writing -S-expressions). -The relevant function is -\code{panelparser.parse_file(\var{file})} -which has a file object (not a filename!) as argument and returns a list -of parsed S-expressions. -Each S-expression is converted into a Python list, with atoms converted -to Python strings and sub-expressions (recursively) to Python lists. -For more details, read the module file. -% XXXXJH should be funcdesc, I think - -\section{\module{pnl} --- - None} -\declaremodule{builtin}{pnl} - -\modulesynopsis{None} - - -This module provides access to the -\emph{Panel Library} -built by NASA Ames\index{NASA} (to get it, send email to -\code{panel-request@nas.nasa.gov}). -All access to it should be done through the standard module -\code{panel}\refstmodindex{panel}, -which transparently exports most functions from -\code{pnl} -but redefines -\code{pnl.dopanel()}. - -\warning{The Python interpreter will dump core if you don't create a -GL window before calling \code{pnl.mkpanel()}.} - -The module is too large to document here in its entirety. diff --git a/Doc/lib/librestricted.tex b/Doc/lib/librestricted.tex deleted file mode 100644 index 5d4b157..0000000 --- a/Doc/lib/librestricted.tex +++ /dev/null @@ -1,66 +0,0 @@ -\chapter{Restricted Execution \label{restricted}} - -\begin{notice}[warning] - In Python 2.3 these modules have been disabled due to various known - and not readily fixable security holes. The modules are still - documented here to help in reading old code that uses the - \module{rexec} and \module{Bastion} modules. -\end{notice} - -\emph{Restricted execution} is the basic framework in Python that allows -for the segregation of trusted and untrusted code. The framework is based on the -notion that trusted Python code (a \emph{supervisor}) can create a -``padded cell' (or environment) with limited permissions, and run the -untrusted code within this cell. The untrusted code cannot break out -of its cell, and can only interact with sensitive system resources -through interfaces defined and managed by the trusted code. The term -``restricted execution'' is favored over ``safe-Python'' -since true safety is hard to define, and is determined by the way the -restricted environment is created. Note that the restricted -environments can be nested, with inner cells creating subcells of -lesser, but never greater, privilege. - -An interesting aspect of Python's restricted execution model is that -the interfaces presented to untrusted code usually have the same names -as those presented to trusted code. Therefore no special interfaces -need to be learned to write code designed to run in a restricted -environment. And because the exact nature of the padded cell is -determined by the supervisor, different restrictions can be imposed, -depending on the application. For example, it might be deemed -``safe'' for untrusted code to read any file within a specified -directory, but never to write a file. In this case, the supervisor -may redefine the built-in \function{open()} function so that it raises -an exception whenever the \var{mode} parameter is \code{'w'}. It -might also perform a \cfunction{chroot()}-like operation on the -\var{filename} parameter, such that root is always relative to some -safe ``sandbox'' area of the filesystem. In this case, the untrusted -code would still see an built-in \function{open()} function in its -environment, with the same calling interface. The semantics would be -identical too, with \exception{IOError}s being raised when the -supervisor determined that an unallowable parameter is being used. - -The Python run-time determines whether a particular code block is -executing in restricted execution mode based on the identity of the -\code{__builtins__} object in its global variables: if this is (the -dictionary of) the standard \refmodule[builtin]{__builtin__} module, -the code is deemed to be unrestricted, else it is deemed to be -restricted. - -Python code executing in restricted mode faces a number of limitations -that are designed to prevent it from escaping from the padded cell. -For instance, the function object attribute \member{func_globals} and -the class and instance object attribute \member{__dict__} are -unavailable. - -Two modules provide the framework for setting up restricted execution -environments: - -\localmoduletable - -\begin{seealso} - \seetitle[http://grail.sourceforge.net/]{Grail Home Page} - {Grail, an Internet browser written in Python, uses these - modules to support Python applets. More - information on the use of Python's restricted execution - mode in Grail is available on the Web site.} -\end{seealso} diff --git a/Doc/lib/librexec.tex b/Doc/lib/librexec.tex deleted file mode 100644 index 3104004..0000000 --- a/Doc/lib/librexec.tex +++ /dev/null @@ -1,275 +0,0 @@ -\section{\module{rexec} --- - Restricted execution framework} - -\declaremodule{standard}{rexec} -\modulesynopsis{Basic restricted execution framework.} -\versionchanged[Disabled module]{2.3} - -\begin{notice}[warning] - The documentation has been left in place to help in reading old code - that uses the module. -\end{notice} - -This module contains the \class{RExec} class, which supports -\method{r_exec()}, \method{r_eval()}, \method{r_execfile()}, and -\method{r_import()} methods, which are restricted versions of the standard -Python functions \method{exec()}, \method{eval()}, \method{execfile()} and -the \keyword{import} statement. -Code executed in this restricted environment will -only have access to modules and functions that are deemed safe; you -can subclass \class{RExec} to add or remove capabilities as desired. - -\begin{notice}[warning] - While the \module{rexec} module is designed to perform as described - below, it does have a few known vulnerabilities which could be - exploited by carefully written code. Thus it should not be relied - upon in situations requiring ``production ready'' security. In such - situations, execution via sub-processes or very careful - ``cleansing'' of both code and data to be processed may be - necessary. Alternatively, help in patching known \module{rexec} - vulnerabilities would be welcomed. -\end{notice} - -\begin{notice} - The \class{RExec} class can prevent code from performing unsafe - operations like reading or writing disk files, or using TCP/IP - sockets. However, it does not protect against code using extremely - large amounts of memory or processor time. -\end{notice} - -\begin{classdesc}{RExec}{\optional{hooks\optional{, verbose}}} -Returns an instance of the \class{RExec} class. - -\var{hooks} is an instance of the \class{RHooks} class or a subclass of it. -If it is omitted or \code{None}, the default \class{RHooks} class is -instantiated. -Whenever the \module{rexec} module searches for a module (even a -built-in one) or reads a module's code, it doesn't actually go out to -the file system itself. Rather, it calls methods of an \class{RHooks} -instance that was passed to or created by its constructor. (Actually, -the \class{RExec} object doesn't make these calls --- they are made by -a module loader object that's part of the \class{RExec} object. This -allows another level of flexibility, which can be useful when changing -the mechanics of \keyword{import} within the restricted environment.) - -By providing an alternate \class{RHooks} object, we can control the -file system accesses made to import a module, without changing the -actual algorithm that controls the order in which those accesses are -made. For instance, we could substitute an \class{RHooks} object that -passes all filesystem requests to a file server elsewhere, via some -RPC mechanism such as ILU. Grail's applet loader uses this to support -importing applets from a URL for a directory. - -If \var{verbose} is true, additional debugging output may be sent to -standard output. -\end{classdesc} - -It is important to be aware that code running in a restricted -environment can still call the \function{sys.exit()} function. To -disallow restricted code from exiting the interpreter, always protect -calls that cause restricted code to run with a -\keyword{try}/\keyword{except} statement that catches the -\exception{SystemExit} exception. Removing the \function{sys.exit()} -function from the restricted environment is not sufficient --- the -restricted code could still use \code{raise SystemExit}. Removing -\exception{SystemExit} is not a reasonable option; some library code -makes use of this and would break were it not available. - - -\begin{seealso} - \seetitle[http://grail.sourceforge.net/]{Grail Home Page}{Grail is a - Web browser written entirely in Python. It uses the - \module{rexec} module as a foundation for supporting - Python applets, and can be used as an example usage of - this module.} -\end{seealso} - - -\subsection{RExec Objects \label{rexec-objects}} - -\class{RExec} instances support the following methods: - -\begin{methoddesc}[RExec]{r_eval}{code} -\var{code} must either be a string containing a Python expression, or -a compiled code object, which will be evaluated in the restricted -environment's \module{__main__} module. The value of the expression or -code object will be returned. -\end{methoddesc} - -\begin{methoddesc}[RExec]{r_exec}{code} -\var{code} must either be a string containing one or more lines of -Python code, or a compiled code object, which will be executed in the -restricted environment's \module{__main__} module. -\end{methoddesc} - -\begin{methoddesc}[RExec]{r_execfile}{filename} -Execute the Python code contained in the file \var{filename} in the -restricted environment's \module{__main__} module. -\end{methoddesc} - -Methods whose names begin with \samp{s_} are similar to the functions -beginning with \samp{r_}, but the code will be granted access to -restricted versions of the standard I/O streams \code{sys.stdin}, -\code{sys.stderr}, and \code{sys.stdout}. - -\begin{methoddesc}[RExec]{s_eval}{code} -\var{code} must be a string containing a Python expression, which will -be evaluated in the restricted environment. -\end{methoddesc} - -\begin{methoddesc}[RExec]{s_exec}{code} -\var{code} must be a string containing one or more lines of Python code, -which will be executed in the restricted environment. -\end{methoddesc} - -\begin{methoddesc}[RExec]{s_execfile}{code} -Execute the Python code contained in the file \var{filename} in the -restricted environment. -\end{methoddesc} - -\class{RExec} objects must also support various methods which will be -implicitly called by code executing in the restricted environment. -Overriding these methods in a subclass is used to change the policies -enforced by a restricted environment. - -\begin{methoddesc}[RExec]{r_import}{modulename\optional{, globals\optional{, - locals\optional{, fromlist}}}} -Import the module \var{modulename}, raising an \exception{ImportError} -exception if the module is considered unsafe. -\end{methoddesc} - -\begin{methoddesc}[RExec]{r_open}{filename\optional{, mode\optional{, bufsize}}} -Method called when \function{open()} is called in the restricted -environment. The arguments are identical to those of \function{open()}, -and a file object (or a class instance compatible with file objects) -should be returned. \class{RExec}'s default behaviour is allow opening -any file for reading, but forbidding any attempt to write a file. See -the example below for an implementation of a less restrictive -\method{r_open()}. -\end{methoddesc} - -\begin{methoddesc}[RExec]{r_reload}{module} -Reload the module object \var{module}, re-parsing and re-initializing it. -\end{methoddesc} - -\begin{methoddesc}[RExec]{r_unload}{module} -Unload the module object \var{module} (remove it from the -restricted environment's \code{sys.modules} dictionary). -\end{methoddesc} - -And their equivalents with access to restricted standard I/O streams: - -\begin{methoddesc}[RExec]{s_import}{modulename\optional{, globals\optional{, - locals\optional{, fromlist}}}} -Import the module \var{modulename}, raising an \exception{ImportError} -exception if the module is considered unsafe. -\end{methoddesc} - -\begin{methoddesc}[RExec]{s_reload}{module} -Reload the module object \var{module}, re-parsing and re-initializing it. -\end{methoddesc} - -\begin{methoddesc}[RExec]{s_unload}{module} -Unload the module object \var{module}. -% XXX what are the semantics of this? -\end{methoddesc} - - -\subsection{Defining restricted environments \label{rexec-extension}} - -The \class{RExec} class has the following class attributes, which are -used by the \method{__init__()} method. Changing them on an existing -instance won't have any effect; instead, create a subclass of -\class{RExec} and assign them new values in the class definition. -Instances of the new class will then use those new values. All these -attributes are tuples of strings. - -\begin{memberdesc}[RExec]{nok_builtin_names} -Contains the names of built-in functions which will \emph{not} be -available to programs running in the restricted environment. The -value for \class{RExec} is \code{('open', 'reload', '__import__')}. -(This gives the exceptions, because by far the majority of built-in -functions are harmless. A subclass that wants to override this -variable should probably start with the value from the base class and -concatenate additional forbidden functions --- when new dangerous -built-in functions are added to Python, they will also be added to -this module.) -\end{memberdesc} - -\begin{memberdesc}[RExec]{ok_builtin_modules} -Contains the names of built-in modules which can be safely imported. -The value for \class{RExec} is \code{('audioop', 'array', 'binascii', -'cmath', 'errno', 'imageop', 'marshal', 'math', 'md5', 'operator', -'parser', 'regex', 'select', 'sha', '_sre', 'strop', -'struct', 'time')}. A similar remark about overriding this variable -applies --- use the value from the base class as a starting point. -\end{memberdesc} - -\begin{memberdesc}[RExec]{ok_path} -Contains the directories which will be searched when an \keyword{import} -is performed in the restricted environment. -The value for \class{RExec} is the same as \code{sys.path} (at the time -the module is loaded) for unrestricted code. -\end{memberdesc} - -\begin{memberdesc}[RExec]{ok_posix_names} -% Should this be called ok_os_names? -Contains the names of the functions in the \refmodule{os} module which will be -available to programs running in the restricted environment. The -value for \class{RExec} is \code{('error', 'fstat', 'listdir', -'lstat', 'readlink', 'stat', 'times', 'uname', 'getpid', 'getppid', -'getcwd', 'getuid', 'getgid', 'geteuid', 'getegid')}. -\end{memberdesc} - -\begin{memberdesc}[RExec]{ok_sys_names} -Contains the names of the functions and variables in the \refmodule{sys} -module which will be available to programs running in the restricted -environment. The value for \class{RExec} is \code{('ps1', 'ps2', -'copyright', 'version', 'platform', 'exit', 'maxint')}. -\end{memberdesc} - -\begin{memberdesc}[RExec]{ok_file_types} -Contains the file types from which modules are allowed to be loaded. -Each file type is an integer constant defined in the \refmodule{imp} module. -The meaningful values are \constant{PY_SOURCE}, \constant{PY_COMPILED}, and -\constant{C_EXTENSION}. The value for \class{RExec} is \code{(C_EXTENSION, -PY_SOURCE)}. Adding \constant{PY_COMPILED} in subclasses is not recommended; -an attacker could exit the restricted execution mode by putting a forged -byte-compiled file (\file{.pyc}) anywhere in your file system, for example -by writing it to \file{/tmp} or uploading it to the \file{/incoming} -directory of your public FTP server. -\end{memberdesc} - - -\subsection{An example} - -Let us say that we want a slightly more relaxed policy than the -standard \class{RExec} class. For example, if we're willing to allow -files in \file{/tmp} to be written, we can subclass the \class{RExec} -class: - -\begin{verbatim} -class TmpWriterRExec(rexec.RExec): - def r_open(self, file, mode='r', buf=-1): - if mode in ('r', 'rb'): - pass - elif mode in ('w', 'wb', 'a', 'ab'): - # check filename : must begin with /tmp/ - if file[:5]!='/tmp/': - raise IOError, "can't write outside /tmp" - elif (string.find(file, '/../') >= 0 or - file[:3] == '../' or file[-3:] == '/..'): - raise IOError, "'..' in filename forbidden" - else: raise IOError, "Illegal open() mode" - return open(file, mode, buf) -\end{verbatim} -% -Notice that the above code will occasionally forbid a perfectly valid -filename; for example, code in the restricted environment won't be -able to open a file called \file{/tmp/foo/../bar}. To fix this, the -\method{r_open()} method would have to simplify the filename to -\file{/tmp/bar}, which would require splitting apart the filename and -performing various operations on it. In cases where security is at -stake, it may be preferable to write simple code which is sometimes -overly restrictive, instead of more general code that is also more -complex and may harbor a subtle security hole. diff --git a/Doc/lib/libsha.tex b/Doc/lib/libsha.tex deleted file mode 100644 index 6d1da68..0000000 --- a/Doc/lib/libsha.tex +++ /dev/null @@ -1,83 +0,0 @@ -\section{\module{sha} --- - SHA-1 message digest algorithm} - -\declaremodule{builtin}{sha} -\modulesynopsis{NIST's secure hash algorithm, SHA.} -\sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org} - -\deprecated{2.5}{Use the \refmodule{hashlib} module instead.} - - -This module implements the interface to NIST's\index{NIST} secure hash -algorithm,\index{Secure Hash Algorithm} known as SHA-1. SHA-1 is an -improved version of the original SHA hash algorithm. It is used in -the same way as the \refmodule{md5} module:\ use \function{new()} -to create an sha object, then feed this object with arbitrary strings -using the \method{update()} method, and at any point you can ask it -for the \dfn{digest} of the concatenation of the strings fed to it -so far.\index{checksum!SHA} SHA-1 digests are 160 bits instead of -MD5's 128 bits. - - -\begin{funcdesc}{new}{\optional{string}} - Return a new sha object. If \var{string} is present, the method - call \code{update(\var{string})} is made. -\end{funcdesc} - - -The following values are provided as constants in the module and as -attributes of the sha objects returned by \function{new()}: - -\begin{datadesc}{blocksize} - Size of the blocks fed into the hash function; this is always - \code{1}. This size is used to allow an arbitrary string to be - hashed. -\end{datadesc} - -\begin{datadesc}{digest_size} - The size of the resulting digest in bytes. This is always - \code{20}. -\end{datadesc} - - -An sha object has the same methods as md5 objects: - -\begin{methoddesc}[sha]{update}{arg} -Update the sha object with the string \var{arg}. Repeated calls are -equivalent to a single call with the concatenation of all the -arguments: \code{m.update(a); m.update(b)} is equivalent to -\code{m.update(a+b)}. -\end{methoddesc} - -\begin{methoddesc}[sha]{digest}{} -Return the digest of the strings passed to the \method{update()} -method so far. This is a 20-byte string which may contain -non-\ASCII{} characters, including null bytes. -\end{methoddesc} - -\begin{methoddesc}[sha]{hexdigest}{} -Like \method{digest()} except the digest is returned as a string of -length 40, containing only hexadecimal digits. This may -be used to exchange the value safely in email or other non-binary -environments. -\end{methoddesc} - -\begin{methoddesc}[sha]{copy}{} -Return a copy (``clone'') of the sha object. This can be used to -efficiently compute the digests of strings that share a common initial -substring. -\end{methoddesc} - -\begin{seealso} - \seetitle[http://csrc.nist.gov/publications/fips/fips180-2/fips180-2withchangenotice.pdf] - {Secure Hash Standard} - {The Secure Hash Algorithm is defined by NIST document FIPS - PUB 180-2: - \citetitle[http://csrc.nist.gov/publications/fips/fips180-2/fips180-2withchangenotice.pdf] - {Secure Hash Standard}, published in August 2002.} - - \seetitle[http://csrc.nist.gov/encryption/tkhash.html] - {Cryptographic Toolkit (Secure Hashing)} - {Links from NIST to various information on secure hashing.} -\end{seealso} - diff --git a/Doc/ref/ref3.tex b/Doc/ref/ref3.tex index 8340e17..3f82a8c 100644 --- a/Doc/ref/ref3.tex +++ b/Doc/ref/ref3.tex @@ -941,18 +941,15 @@ stack frame; \member{f_code} is the code object being executed in this frame; \member{f_locals} is the dictionary used to look up local variables; \member{f_globals} is used for global variables; \member{f_builtins} is used for built-in (intrinsic) names; -\member{f_restricted} is a flag indicating whether the function is -executing in restricted execution mode; \member{f_lasti} gives the -precise instruction (this is an index into the bytecode string of -the code object). + \member{f_lasti} gives the precise instruction (this is an index into + the bytecode string of the code object). \withsubitem{(frame attribute)}{ \ttindex{f_back} \ttindex{f_code} \ttindex{f_globals} \ttindex{f_locals} \ttindex{f_lasti} - \ttindex{f_builtins} - \ttindex{f_restricted}} + \ttindex{f_builtins}} Special writable attributes: \member{f_trace}, if not \code{None}, is a function called at the start of each source code line (this is used |