Move reference material on PyArg_Parse*() out of the Extending & Embedding

document to the C API reference.  Move some instructional text from the API
reference to the Extending & Embedding manual.

Fix the descriptions of the es and es# formats for PyArg_Parse*().
This closes SF bug #536516.

1 files changed, 5 insertions, 372 deletions

diff --git a/Doc/ext/extending.tex b/Doc/ext/extending.tex
index 716696b..7eafc05 100644
--- a/Doc/ext/extending.tex
+++ b/Doc/ext/extending.tex
@@ -602,12 +602,12 @@ int PyArg_ParseTuple(PyObject *arg, char *format, ...);
 
 The \var{arg} argument must be a tuple object containing an argument
 list passed from Python to a C function.  The \var{format} argument
-must be a format string, whose syntax is explained below.  The
+must be a format string, whose syntax is explained in
+``\ulink{Parsing arguments and building
+values}{../api/arg-parsing.html}'' in the
+\citetitle[../api/api.html]{Python/C API Reference Manual}.  The
 remaining arguments must be addresses of variables whose type is
-determined by the format string.  For the conversion to succeed, the
-\var{arg} object must match the format and the format must be
-exhausted.  On success, \cfunction{PyArg_ParseTuple()} returns true,
-otherwise it returns false and raises an appropriate exception.
+determined by the format string.
 
 Note that while \cfunction{PyArg_ParseTuple()} checks that the Python
 arguments have the required types, it cannot check the validity of the
@@ -615,263 +615,10 @@ addresses of C variables passed to the call: if you make mistakes
 there, your code will probably crash or at least overwrite random bits
 in memory.  So be careful!
 
-A format string consists of zero or more ``format units''.  A format
-unit describes one Python object; it is usually a single character or
-a parenthesized sequence of format units.  With a few exceptions, a
-format unit that is not a parenthesized sequence normally corresponds
-to a single address argument to \cfunction{PyArg_ParseTuple()}.  In the
-following description, the quoted form is the format unit; the entry
-in (round) parentheses is the Python object type that matches the
-format unit; and the entry in [square] brackets is the type of the C
-variable(s) whose address should be passed.  (Use the \samp{\&}
-operator to pass a variable's address.)
-
 Note that any Python object references which are provided to the
 caller are \emph{borrowed} references; do not decrement their
 reference count!
 
-\begin{description}
-
-\item[\samp{s} (string or Unicode object) {[char *]}]
-Convert a Python string or Unicode object to a C pointer to a
-character string.  You must not provide storage for the string
-itself; a pointer to an existing string is stored into the character
-pointer variable whose address you pass.  The C string is
-null-terminated.  The Python string must not contain embedded null
-bytes; if it does, a \exception{TypeError} exception is raised.
-Unicode objects are converted to C strings using the default
-encoding. If this conversion fails, an \exception{UnicodeError} is
-raised.
-
-\item[\samp{s\#} (string, Unicode or any read buffer compatible object) 
-{[char *, int]}]
-This variant on \samp{s} stores into two C variables, the first one a
-pointer to a character string, the second one its length.  In this
-case the Python string may contain embedded null bytes.  Unicode
-objects pass back a pointer to the default encoded string version of the
-object if such a conversion is possible. All other read buffer
-compatible objects pass back a reference to the raw internal data
-representation.
-
-\item[\samp{z} (string or \code{None}) {[char *]}]
-Like \samp{s}, but the Python object may also be \code{None}, in which
-case the C pointer is set to \NULL.
-
-\item[\samp{z\#} (string or \code{None} or any read buffer compatible object) 
-{[char *, int]}]
-This is to \samp{s\#} as \samp{z} is to \samp{s}.
-
-\item[\samp{u} (Unicode object) {[Py_UNICODE *]}]
-Convert a Python Unicode object to a C pointer to a null-terminated
-buffer of 16-bit Unicode (UTF-16) data.  As with \samp{s}, there is no
-need to provide storage for the Unicode data buffer; a pointer to the
-existing Unicode data is stored into the \ctype{Py_UNICODE} pointer
-variable whose address you pass.  
-
-\item[\samp{u\#} (Unicode object) {[Py_UNICODE *, int]}]
-This variant on \samp{u} stores into two C variables, the first one
-a pointer to a Unicode data buffer, the second one its length.
-Non-Unicode objects are handled by interpreting their read buffer
-pointer as pointer to a \ctype{Py_UNICODE} array.
-
-\item[\samp{es} (string, Unicode object or character buffer compatible
-object) {[const char *encoding, char **buffer]}]
-This variant on \samp{s} is used for encoding Unicode and objects
-convertible to Unicode into a character buffer. It only works for
-encoded data without embedded \NULL{} bytes.
-
-The variant reads one C variable and stores into two C variables, the
-first one a pointer to an encoding name string (\var{encoding}), and the
-second a pointer to a pointer to a character buffer (\var{**buffer},
-the buffer used for storing the encoded data).
-
-The encoding name must map to a registered codec. If set to \NULL,
-the default encoding is used.
-
-\cfunction{PyArg_ParseTuple()} will allocate a buffer of the needed
-size using \cfunction{PyMem_NEW()}, copy the encoded data into this
-buffer and adjust \var{*buffer} to reference the newly allocated
-storage. The caller is responsible for calling
-\cfunction{PyMem_Free()} to free the allocated buffer after usage.
-
-\item[\samp{et} (string, Unicode object or character buffer compatible
-object) {[const char *encoding, char **buffer]}]
-Same as \samp{es} except that string objects are passed through without
-recoding them. Instead, the implementation assumes that the string
-object uses the encoding passed in as parameter.
-
-\item[\samp{es\#} (string, Unicode object or character buffer compatible
-object) {[const char *encoding, char **buffer, int *buffer_length]}]
-This variant on \samp{s\#} is used for encoding Unicode and objects
-convertible to Unicode into a character buffer. It reads one C
-variable and stores into three C variables, the first one a pointer to
-an encoding name string (\var{encoding}), the second a pointer to a
-pointer to a character buffer (\var{**buffer}, the buffer used for
-storing the encoded data) and the third one a pointer to an integer
-(\var{*buffer_length}, the buffer length).
-
-The encoding name must map to a registered codec. If set to \NULL,
-the default encoding is used.
-
-There are two modes of operation: 
-
-If \var{*buffer} points a \NULL{} pointer,
-\cfunction{PyArg_ParseTuple()} will allocate a buffer of the needed
-size using \cfunction{PyMem_NEW()}, copy the encoded data into this
-buffer and adjust \var{*buffer} to reference the newly allocated
-storage. The caller is responsible for calling
-\cfunction{PyMem_Free()} to free the allocated buffer after usage.
-
-If \var{*buffer} points to a non-\NULL{} pointer (an already allocated
-buffer), \cfunction{PyArg_ParseTuple()} will use this location as
-buffer and interpret \var{*buffer_length} as buffer size. It will then
-copy the encoded data into the buffer and 0-terminate it. Buffer
-overflow is signalled with an exception.
-
-In both cases, \var{*buffer_length} is set to the length of the
-encoded data without the trailing 0-byte.
-
-\item[\samp{et\#} (string, Unicode object or character buffer compatible
-object) {[const char *encoding, char **buffer]}]
-Same as \samp{es\#} except that string objects are passed through without
-recoding them. Instead, the implementation assumes that the string
-object uses the encoding passed in as parameter.
-
-\item[\samp{b} (integer) {[char]}]
-Convert a Python integer to a tiny int, stored in a C \ctype{char}.
-
-\item[\samp{h} (integer) {[short int]}]
-Convert a Python integer to a C \ctype{short int}.
-
-\item[\samp{i} (integer) {[int]}]
-Convert a Python integer to a plain C \ctype{int}.
-
-\item[\samp{l} (integer) {[long int]}]
-Convert a Python integer to a C \ctype{long int}.
-
-\item[\samp{L} (integer) {[LONG_LONG]}]
-Convert a Python integer to a C \ctype{long long}.  This format is only
-available on platforms that support \ctype{long long} (or \ctype{_int64}
-on Windows).
-
-\item[\samp{c} (string of length 1) {[char]}]
-Convert a Python character, represented as a string of length 1, to a
-C \ctype{char}.
-
-\item[\samp{f} (float) {[float]}]
-Convert a Python floating point number to a C \ctype{float}.
-
-\item[\samp{d} (float) {[double]}]
-Convert a Python floating point number to a C \ctype{double}.
-
-\item[\samp{D} (complex) {[Py_complex]}]
-Convert a Python complex number to a C \ctype{Py_complex} structure.
-
-\item[\samp{O} (object) {[PyObject *]}]
-Store a Python object (without any conversion) in a C object pointer.
-The C program thus receives the actual object that was passed.  The
-object's reference count is not increased.  The pointer stored is not
-\NULL.
-
-\item[\samp{O!} (object) {[\var{typeobject}, PyObject *]}]
-Store a Python object in a C object pointer.  This is similar to
-\samp{O}, but takes two C arguments: the first is the address of a
-Python type object, the second is the address of the C variable (of
-type \ctype{PyObject *}) into which the object pointer is stored.
-If the Python object does not have the required type,
-\exception{TypeError} is raised.
-
-\item[\samp{O\&} (object) {[\var{converter}, \var{anything}]}]
-Convert a Python object to a C variable through a \var{converter}
-function.  This takes two arguments: the first is a function, the
-second is the address of a C variable (of arbitrary type), converted
-to \ctype{void *}.  The \var{converter} function in turn is called as
-follows:
-
-\var{status}\code{ = }\var{converter}\code{(}\var{object}, \var{address}\code{);}
-
-where \var{object} is the Python object to be converted and
-\var{address} is the \ctype{void *} argument that was passed to
-\cfunction{PyArg_ParseTuple()}.  The returned \var{status} should be
-\code{1} for a successful conversion and \code{0} if the conversion
-has failed.  When the conversion fails, the \var{converter} function
-should raise an exception.
-
-\item[\samp{S} (string) {[PyStringObject *]}]
-Like \samp{O} but requires that the Python object is a string object.
-Raises \exception{TypeError} if the object is not a string object.
-The C variable may also be declared as \ctype{PyObject *}.
-
-\item[\samp{U} (Unicode string) {[PyUnicodeObject *]}]
-Like \samp{O} but requires that the Python object is a Unicode object.
-Raises \exception{TypeError} if the object is not a Unicode object.
-The C variable may also be declared as \ctype{PyObject *}.
-
-\item[\samp{t\#} (read-only character buffer) {[char *, int]}]
-Like \samp{s\#}, but accepts any object which implements the read-only 
-buffer interface.  The \ctype{char *} variable is set to point to the
-first byte of the buffer, and the \ctype{int} is set to the length of
-the buffer.  Only single-segment buffer objects are accepted;
-\exception{TypeError} is raised for all others.
-
-\item[\samp{w} (read-write character buffer) {[char *]}]
-Similar to \samp{s}, but accepts any object which implements the
-read-write buffer interface.  The caller must determine the length of
-the buffer by other means, or use \samp{w\#} instead.  Only
-single-segment buffer objects are accepted; \exception{TypeError} is
-raised for all others.
-
-\item[\samp{w\#} (read-write character buffer) {[char *, int]}]
-Like \samp{s\#}, but accepts any object which implements the
-read-write buffer interface.  The \ctype{char *} variable is set to
-point to the first byte of the buffer, and the \ctype{int} is set to
-the length of the buffer.  Only single-segment buffer objects are
-accepted; \exception{TypeError} is raised for all others.
-
-\item[\samp{(\var{items})} (tuple) {[\var{matching-items}]}]
-The object must be a Python sequence whose length is the number of
-format units in \var{items}.  The C arguments must correspond to the
-individual format units in \var{items}.  Format units for sequences
-may be nested.
-
-\note{Prior to Python version 1.5.2, this format specifier
-only accepted a tuple containing the individual parameters, not an
-arbitrary sequence.  Code which previously caused
-\exception{TypeError} to be raised here may now proceed without an
-exception.  This is not expected to be a problem for existing code.}
-
-\end{description}
-
-It is possible to pass Python long integers where integers are
-requested; however no proper range checking is done --- the most
-significant bits are silently truncated when the receiving field is
-too small to receive the value (actually, the semantics are inherited
-from downcasts in C --- your mileage may vary).
-
-A few other characters have a meaning in a format string.  These may
-not occur inside nested parentheses.  They are:
-
-\begin{description}
-
-\item[\samp{|}]
-Indicates that the remaining arguments in the Python argument list are
-optional.  The C variables corresponding to optional arguments should
-be initialized to their default value --- when an optional argument is
-not specified, \cfunction{PyArg_ParseTuple()} does not touch the contents
-of the corresponding C variable(s).
-
-\item[\samp{:}]
-The list of format units ends here; the string after the colon is used
-as the function name in error messages (the ``associated value'' of
-the exception that \cfunction{PyArg_ParseTuple()} raises).
-
-\item[\samp{;}]
-The list of format units ends here; the string after the semicolon is
-used as the error message \emph{instead} of the default error message.
-Clearly, \samp{:} and \samp{;} mutually exclude each other.
-
-\end{description}
-
 Some example calls:
 
 \begin{verbatim}
@@ -1042,120 +789,6 @@ exactly one format unit, it returns whatever object is described by
 that format unit.  To force it to return a tuple of size 0 or one,
 parenthesize the format string.
 
-When memory buffers are passed as parameters to supply data to build
-objects, as for the \samp{s} and \samp{s\#} formats, the required data
-is copied.  Buffers provided by the caller are never referenced by the
-objects created by \cfunction{Py_BuildValue()}.  In other words, if
-your code invokes \cfunction{malloc()} and passes the allocated memory
-to \cfunction{Py_BuildValue()}, your code is responsible for
-calling \cfunction{free()} for that memory once
-\cfunction{Py_BuildValue()} returns.
-
-In the following description, the quoted form is the format unit; the
-entry in (round) parentheses is the Python object type that the format
-unit will return; and the entry in [square] brackets is the type of
-the C value(s) to be passed.
-
-The characters space, tab, colon and comma are ignored in format
-strings (but not within format units such as \samp{s\#}).  This can be
-used to make long format strings a tad more readable.
-
-\begin{description}
-
-\item[\samp{s} (string) {[char *]}]
-Convert a null-terminated C string to a Python object.  If the C
-string pointer is \NULL, \code{None} is used.
-
-\item[\samp{s\#} (string) {[char *, int]}]
-Convert a C string and its length to a Python object.  If the C string
-pointer is \NULL, the length is ignored and \code{None} is
-returned.
-
-\item[\samp{z} (string or \code{None}) {[char *]}]
-Same as \samp{s}.
-
-\item[\samp{z\#} (string or \code{None}) {[char *, int]}]
-Same as \samp{s\#}.
-
-\item[\samp{u} (Unicode string) {[Py_UNICODE *]}]
-Convert a null-terminated buffer of Unicode (UCS-2) data to a Python
-Unicode object.  If the Unicode buffer pointer is \NULL,
-\code{None} is returned.
-
-\item[\samp{u\#} (Unicode string) {[Py_UNICODE *, int]}]
-Convert a Unicode (UCS-2) data buffer and its length to a Python
-Unicode object.   If the Unicode buffer pointer is \NULL, the length
-is ignored and \code{None} is returned.
-
-\item[\samp{i} (integer) {[int]}]
-Convert a plain C \ctype{int} to a Python integer object.
-
-\item[\samp{b} (integer) {[char]}]
-Same as \samp{i}.
-
-\item[\samp{h} (integer) {[short int]}]
-Same as \samp{i}.
-
-\item[\samp{l} (integer) {[long int]}]
-Convert a C \ctype{long int} to a Python integer object.
-
-\item[\samp{c} (string of length 1) {[char]}]
-Convert a C \ctype{int} representing a character to a Python string of
-length 1.
-
-\item[\samp{d} (float) {[double]}]
-Convert a C \ctype{double} to a Python floating point number.
-
-\item[\samp{f} (float) {[float]}]
-Same as \samp{d}.
-
-\item[\samp{D} (complex) {[Py_complex *]}]
-Convert a C \ctype{Py_complex} structure to a Python complex number.
-
-\item[\samp{O} (object) {[PyObject *]}]
-Pass a Python object untouched (except for its reference count, which
-is incremented by one).  If the object passed in is a \NULL{}
-pointer, it is assumed that this was caused because the call producing
-the argument found an error and set an exception.  Therefore,
-\cfunction{Py_BuildValue()} will return \NULL{} but won't raise an
-exception.  If no exception has been raised yet,
-\cdata{PyExc_SystemError} is set.
-
-\item[\samp{S} (object) {[PyObject *]}]
-Same as \samp{O}.
-
-\item[\samp{U} (object) {[PyObject *]}]
-Same as \samp{O}.
-
-\item[\samp{N} (object) {[PyObject *]}]
-Same as \samp{O}, except it doesn't increment the reference count on
-the object.  Useful when the object is created by a call to an object
-constructor in the argument list.
-
-\item[\samp{O\&} (object) {[\var{converter}, \var{anything}]}]
-Convert \var{anything} to a Python object through a \var{converter}
-function.  The function is called with \var{anything} (which should be
-compatible with \ctype{void *}) as its argument and should return a
-``new'' Python object, or \NULL{} if an error occurred.
-
-\item[\samp{(\var{items})} (tuple) {[\var{matching-items}]}]
-Convert a sequence of C values to a Python tuple with the same number
-of items.
-
-\item[\samp{[\var{items}]} (list) {[\var{matching-items}]}]
-Convert a sequence of C values to a Python list with the same number
-of items.
-
-\item[\samp{\{\var{items}\}} (dictionary) {[\var{matching-items}]}]
-Convert a sequence of C values to a Python dictionary.  Each pair of
-consecutive C values adds one item to the dictionary, serving as key
-and value, respectively.
-
-\end{description}
-
-If there is an error in the format string, the
-\cdata{PyExc_SystemError} exception is raised and \NULL{} returned.
-
 Examples (to the left the call, to the right the resulting Python value):
 
 \begin{verbatim}