diff options
Diffstat (limited to 'Doc/c-api')
31 files changed, 2013 insertions, 727 deletions
diff --git a/Doc/c-api/allocation.rst b/Doc/c-api/allocation.rst index e8f60bf..25a867f 100644 --- a/Doc/c-api/allocation.rst +++ b/Doc/c-api/allocation.rst @@ -32,7 +32,7 @@ Allocating Objects on the Heap Allocate a new Python object using the C structure type *TYPE* and the Python type object *type*. Fields not defined by the Python object header are not initialized; the object's reference count will be one. The size of - the memory allocation is determined from the :attr:`tp_basicsize` field of + the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of the type object. @@ -41,7 +41,7 @@ Allocating Objects on the Heap Allocate a new Python object using the C structure type *TYPE* and the Python type object *type*. Fields not defined by the Python object header are not initialized. The allocated memory allows for the *TYPE* structure - plus *size* fields of the size given by the :attr:`tp_itemsize` field of + plus *size* fields of the size given by the :c:member:`~PyTypeObject.tp_itemsize` field of *type*. This is useful for implementing objects like tuples, which are able to determine their size at construction time. Embedding the array of fields into the same allocation decreases the number of allocations, @@ -52,7 +52,7 @@ Allocating Objects on the Heap Releases memory allocated to an object using :c:func:`PyObject_New` or :c:func:`PyObject_NewVar`. This is normally called from the - :attr:`tp_dealloc` handler specified in the object's type. The fields of + :c:member:`~PyTypeObject.tp_dealloc` handler specified in the object's type. The fields of the object should not be accessed after this call as the memory is no longer a valid Python object. diff --git a/Doc/c-api/apiabiversion.rst b/Doc/c-api/apiabiversion.rst new file mode 100644 index 0000000..890a038 --- /dev/null +++ b/Doc/c-api/apiabiversion.rst @@ -0,0 +1,39 @@ +.. highlightlang:: c + +.. _apiabiversion: + +*********************** +API and ABI Versioning +*********************** + +``PY_VERSION_HEX`` is the Python version number encoded in a single integer. + +For example if the ``PY_VERSION_HEX`` is set to ``0x030401a2``, the underlying +version information can be found by treating it as a 32 bit number in +the following manner: + + +-------+-------------------------+------------------------------------------------+ + | Bytes | Bits (big endian order) | Meaning | + +=======+=========================+================================================+ + | ``1`` | ``1-8`` | ``PY_MAJOR_VERSION`` (the ``3`` in | + | | | ``3.4.1a2``) | + +-------+-------------------------+------------------------------------------------+ + | ``2`` | ``9-16`` | ``PY_MINOR_VERSION`` (the ``4`` in | + | | | ``3.4.1a2``) | + +-------+-------------------------+------------------------------------------------+ + | ``3`` | ``17-24`` | ``PY_MICRO_VERSION`` (the ``1`` in | + | | | ``3.4.1a2``) | + +-------+-------------------------+------------------------------------------------+ + | ``4`` | ``25-28`` | ``PY_RELEASE_LEVEL`` (``0xA`` for alpha, | + | | | ``0xB`` for beta, ``0xC`` for release | + | | | candidate and ``0xF`` for final), in this | + | | | case it is alpha. | + +-------+-------------------------+------------------------------------------------+ + | | ``29-32`` | ``PY_RELEASE_SERIAL`` (the ``2`` in | + | | | ``3.4.1a2``, zero for final releases) | + +-------+-------------------------+------------------------------------------------+ + +Thus ``3.4.1a2`` is hexversion ``0x030401a2``. + +All the given macros are defined in :source:`Include/patchlevel.h`. + diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index d4dda7c..58c0f3d 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -70,8 +70,7 @@ Unless otherwise stated, buffers are not NUL-terminated. as *converter*. ``s*`` (:class:`str`, :class:`bytes`, :class:`bytearray` or buffer compatible object) [Py_buffer] - This format accepts Unicode objects as well as objects supporting the - buffer protocol. + This format accepts Unicode objects as well as :term:`bytes-like object`\ s. It fills a :c:type:`Py_buffer` structure provided by the caller. In this case the resulting C string may contain embedded NUL bytes. Unicode objects are converted to C strings using ``'utf-8'`` encoding. @@ -101,14 +100,14 @@ Unless otherwise stated, buffers are not NUL-terminated. contain embedded NUL bytes; if it does, a :exc:`TypeError` exception is raised. -``y*`` (:class:`bytes`, :class:`bytearray` or buffer compatible object) [Py_buffer] - This variant on ``s*`` doesn't accept Unicode objects, only objects - supporting the buffer protocol. **This is the recommended way to accept +``y*`` (:class:`bytes`, :class:`bytearray` or :term:`bytes-like object`) [Py_buffer] + This variant on ``s*`` doesn't accept Unicode objects, only + :term:`bytes-like object`\ s. **This is the recommended way to accept binary data.** ``y#`` (:class:`bytes`) [const char \*, int] - This variant on ``s#`` doesn't accept Unicode objects, only bytes-like - objects. + This variant on ``s#`` doesn't accept Unicode objects, only :term:`bytes-like + object`\ s. ``S`` (:class:`bytes`) [PyBytesObject \*] Requires that the Python object is a :class:`bytes` object, without @@ -146,7 +145,7 @@ Unless otherwise stated, buffers are not NUL-terminated. Like ``u#``, but the Python object may also be ``None``, in which case the :c:type:`Py_UNICODE` pointer is set to *NULL*. -``U`` (:class:`str`) [PyUnicodeObject \*] +``U`` (:class:`str`) [PyObject \*] Requires that the Python object is a Unicode object, without attempting any conversion. Raises :exc:`TypeError` if the object is not a Unicode object. The C variable may also be declared as :c:type:`PyObject\*`. @@ -260,9 +259,12 @@ Numbers ``n`` (:class:`int`) [Py_ssize_t] Convert a Python integer to a C :c:type:`Py_ssize_t`. -``c`` (:class:`bytes` of length 1) [char] - Convert a Python byte, represented as a :class:`bytes` object of length 1, - to a C :c:type:`char`. +``c`` (:class:`bytes` or :class:`bytearray` of length 1) [char] + Convert a Python byte, represented as a :class:`bytes` or + :class:`bytearray` object of length 1, to a C :c:type:`char`. + + .. versionchanged:: 3.3 + Allow :class:`bytearray` objects. ``C`` (:class:`str` of length 1) [int] Convert a Python character, represented as a :class:`str` object of @@ -315,6 +317,15 @@ Other objects .. versionchanged:: 3.1 ``Py_CLEANUP_SUPPORTED`` was added. +``p`` (:class:`bool`) [int] + Tests the value passed in for truth (a boolean **p**\redicate) and converts + the result to its equivalent C true/false integer value. + Sets the int to 1 if the expression was true and 0 if it was false. + This accepts any valid Python value. See :ref:`truth` for more + information about how Python tests values for truth. + + .. versionadded:: 3.3 + ``(items)`` (:class:`tuple`) [*matching-items*] The object must be a Python sequence whose length is the number of format units in *items*. The C arguments must correspond to the individual format units in @@ -336,6 +347,15 @@ inside nested parentheses. They are: :c:func:`PyArg_ParseTuple` does not touch the contents of the corresponding C variable(s). +``$`` + :c:func:`PyArg_ParseTupleAndKeywords` only: + Indicates that the remaining arguments in the Python argument list are + keyword-only. Currently, all keyword-only arguments must also be optional + arguments, so ``|`` must always be specified before ``$`` in the format + string. + + .. versionadded:: 3.3 + ``:`` The list of format units ends here; the string after the colon is used as the function name in error messages (the "associated value" of the exception that @@ -493,7 +513,7 @@ Building values ``None`` is returned. ``y`` (:class:`bytes`) [char \*] - This converts a C string to a Python :func:`bytes` object. If the C + This converts a C string to a Python :class:`bytes` object. If the C string pointer is *NULL*, ``None`` is returned. ``y#`` (:class:`bytes`) [char \*, int] diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 740b575..f703e9c 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -12,6 +12,7 @@ Buffer Protocol .. sectionauthor:: Greg Stein <gstein@lyra.org> .. sectionauthor:: Benjamin Peterson +.. sectionauthor:: Stefan Krah Certain objects available in Python wrap access to an underlying memory @@ -22,7 +23,7 @@ as image processing or numeric analysis. While each of these types have their own semantics, they share the common characteristic of being backed by a possibly large memory buffer. It is -then desireable, in some situations, to access that buffer directly and +then desirable, in some situations, to access that buffer directly and without intermediate copying. Python provides such a facility at the C level in the form of the :ref:`buffer @@ -62,8 +63,10 @@ isn't needed anymore. Failure to do so could lead to various issues such as resource leaks. -The buffer structure -==================== +.. _buffer-structure: + +Buffer structure +================ Buffer structures (or simply "buffers") are useful as a way to expose the binary data from another object to the Python programmer. They can also be @@ -80,249 +83,414 @@ allows them to be created and copied very simply. When a generic wrapper around a buffer is needed, a :ref:`memoryview <memoryview-objects>` object can be created. +For short instructions how to write an exporting object, see +:ref:`Buffer Object Structures <buffer-structs>`. For obtaining +a buffer, see :c:func:`PyObject_GetBuffer`. .. c:type:: Py_buffer - .. c:member:: void *buf + .. c:member:: void \*obj + + A new reference to the exporting object. The reference is owned by + the consumer and automatically decremented and set to *NULL* by + :c:func:`PyBuffer_Release`. The field is the equivalent of the return + value of any standard C-API function. + + As a special case, for *temporary* buffers that are wrapped by + :c:func:`PyMemoryView_FromBuffer` or :c:func:`PyBuffer_FillInfo` + this field is *NULL*. In general, exporting objects MUST NOT + use this scheme. + + .. c:member:: void \*buf + + A pointer to the start of the logical structure described by the buffer + fields. This can be any location within the underlying physical memory + block of the exporter. For example, with negative :c:member:`~Py_buffer.strides` + the value may point to the end of the memory block. - A pointer to the start of the memory for the object. + For contiguous arrays, the value points to the beginning of the memory + block. .. c:member:: Py_ssize_t len - :noindex: - The total length of the memory in bytes. + ``product(shape) * itemsize``. For contiguous arrays, this is the length + of the underlying memory block. For non-contiguous arrays, it is the length + that the logical structure would have if it were copied to a contiguous + representation. + + Accessing ``((char *)buf)[0] up to ((char *)buf)[len-1]`` is only valid + if the buffer has been obtained by a request that guarantees contiguity. In + most cases such a request will be :c:macro:`PyBUF_SIMPLE` or :c:macro:`PyBUF_WRITABLE`. .. c:member:: int readonly - An indicator of whether the buffer is read only. + An indicator of whether the buffer is read-only. This field is controlled + by the :c:macro:`PyBUF_WRITABLE` flag. + + .. c:member:: Py_ssize_t itemsize + + Item size in bytes of a single element. Same as the value of :func:`struct.calcsize` + called on non-NULL :c:member:`~Py_buffer.format` values. + + Important exception: If a consumer requests a buffer without the + :c:macro:`PyBUF_FORMAT` flag, :c:member:`~Py_Buffer.format` will + be set to *NULL*, but :c:member:`~Py_buffer.itemsize` still has + the value for the original format. + + If :c:member:`~Py_Buffer.shape` is present, the equality + ``product(shape) * itemsize == len`` still holds and the consumer + can use :c:member:`~Py_buffer.itemsize` to navigate the buffer. - .. c:member:: const char *format - :noindex: + If :c:member:`~Py_Buffer.shape` is *NULL* as a result of a :c:macro:`PyBUF_SIMPLE` + or a :c:macro:`PyBUF_WRITABLE` request, the consumer must disregard + :c:member:`~Py_buffer.itemsize` and assume ``itemsize == 1``. - A *NULL* terminated string in :mod:`struct` module style syntax giving - the contents of the elements available through the buffer. If this is - *NULL*, ``"B"`` (unsigned bytes) is assumed. + .. c:member:: const char \*format + + A *NUL* terminated string in :mod:`struct` module style syntax describing + the contents of a single item. If this is *NULL*, ``"B"`` (unsigned bytes) + is assumed. + + This field is controlled by the :c:macro:`PyBUF_FORMAT` flag. .. c:member:: int ndim - The number of dimensions the memory represents as a multi-dimensional - array. If it is 0, :c:data:`strides` and :c:data:`suboffsets` must be - *NULL*. - - .. c:member:: Py_ssize_t *shape - - An array of :c:type:`Py_ssize_t`\s the length of :c:data:`ndim` giving the - shape of the memory as a multi-dimensional array. Note that - ``((*shape)[0] * ... * (*shape)[ndims-1])*itemsize`` should be equal to - :c:data:`len`. - - .. c:member:: Py_ssize_t *strides - - An array of :c:type:`Py_ssize_t`\s the length of :c:data:`ndim` giving the - number of bytes to skip to get to a new element in each dimension. - - .. c:member:: Py_ssize_t *suboffsets - - An array of :c:type:`Py_ssize_t`\s the length of :c:data:`ndim`. If these - suboffset numbers are greater than or equal to 0, then the value stored - along the indicated dimension is a pointer and the suboffset value - dictates how many bytes to add to the pointer after de-referencing. A - suboffset value that it negative indicates that no de-referencing should - occur (striding in a contiguous memory block). - - Here is a function that returns a pointer to the element in an N-D array - pointed to by an N-dimensional index when there are both non-NULL strides - and suboffsets:: - - void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides, - Py_ssize_t *suboffsets, Py_ssize_t *indices) { - char *pointer = (char*)buf; - int i; - for (i = 0; i < ndim; i++) { - pointer += strides[i] * indices[i]; - if (suboffsets[i] >=0 ) { - pointer = *((char**)pointer) + suboffsets[i]; - } - } - return (void*)pointer; - } + The number of dimensions the memory represents as an n-dimensional array. + If it is 0, :c:member:`~Py_Buffer.buf` points to a single item representing + a scalar. In this case, :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides` + and :c:member:`~Py_buffer.suboffsets` MUST be *NULL*. + The macro :c:macro:`PyBUF_MAX_NDIM` limits the maximum number of dimensions + to 64. Exporters MUST respect this limit, consumers of multi-dimensional + buffers SHOULD be able to handle up to :c:macro:`PyBUF_MAX_NDIM` dimensions. - .. c:member:: Py_ssize_t itemsize + .. c:member:: Py_ssize_t \*shape + + An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim` + indicating the shape of the memory as an n-dimensional array. Note that + ``shape[0] * ... * shape[ndim-1] * itemsize`` MUST be equal to + :c:member:`~Py_buffer.len`. + + Shape values are restricted to ``shape[n] >= 0``. The case + ``shape[n] == 0`` requires special attention. See `complex arrays`_ + for further information. + + The shape array is read-only for the consumer. + + .. c:member:: Py_ssize_t \*strides + + An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim` + giving the number of bytes to skip to get to a new element in each + dimension. + + Stride values can be any integer. For regular arrays, strides are + usually positive, but a consumer MUST be able to handle the case + ``strides[n] <= 0``. See `complex arrays`_ for further information. + + The strides array is read-only for the consumer. - This is a storage for the itemsize (in bytes) of each element of the - shared memory. It is technically un-necessary as it can be obtained - using :c:func:`PyBuffer_SizeFromFormat`, however an exporter may know - this information without parsing the format string and it is necessary - to know the itemsize for proper interpretation of striding. Therefore, - storing it is more convenient and faster. + .. c:member:: Py_ssize_t \*suboffsets - .. c:member:: void *internal + An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim`. + If ``suboffsets[n] >= 0``, the values stored along the nth dimension are + pointers and the suboffset value dictates how many bytes to add to each + pointer after de-referencing. A suboffset value that is negative + indicates that no de-referencing should occur (striding in a contiguous + memory block). + + This type of array representation is used by the Python Imaging Library + (PIL). See `complex arrays`_ for further information how to access elements + of such an array. + + The suboffsets array is read-only for the consumer. + + .. c:member:: void \*internal This is for use internally by the exporting object. For example, this might be re-cast as an integer by the exporter and used to store flags about whether or not the shape, strides, and suboffsets arrays must be - freed when the buffer is released. The consumer should never alter this + freed when the buffer is released. The consumer MUST NOT alter this value. +.. _buffer-request-types: -Buffer-related functions -======================== +Buffer request types +==================== +Buffers are usually obtained by sending a buffer request to an exporting +object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical +structure of the memory can vary drastically, the consumer uses the *flags* +argument to specify the exact buffer type it can handle. -.. c:function:: int PyObject_CheckBuffer(PyObject *obj) +All :c:data:`Py_buffer` fields are unambiguously defined by the request +type. - Return 1 if *obj* supports the buffer interface otherwise 0. When 1 is - returned, it doesn't guarantee that :c:func:`PyObject_GetBuffer` will - succeed. +request-independent fields +~~~~~~~~~~~~~~~~~~~~~~~~~~ +The following fields are not influenced by *flags* and must always be filled in +with the correct values: :c:member:`~Py_buffer.obj`, :c:member:`~Py_buffer.buf`, +:c:member:`~Py_buffer.len`, :c:member:`~Py_buffer.itemsize`, :c:member:`~Py_buffer.ndim`. -.. c:function:: int PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags) +readonly, format +~~~~~~~~~~~~~~~~ - Export a view over some internal data from the target object *obj*. - *obj* must not be NULL, and *view* must point to an existing - :c:type:`Py_buffer` structure allocated by the caller (most uses of - this function will simply declare a local variable of type - :c:type:`Py_buffer`). The *flags* argument is a bit field indicating - what kind of buffer is requested. The buffer interface allows - for complicated memory layout possibilities; however, some callers - won't want to handle all the complexity and instead request a simple - view of the target object (using :c:macro:`PyBUF_SIMPLE` for a read-only - view and :c:macro:`PyBUF_WRITABLE` for a read-write view). + .. c:macro:: PyBUF_WRITABLE - Some exporters may not be able to share memory in every possible way and - may need to raise errors to signal to some consumers that something is - just not possible. These errors should be a :exc:`BufferError` unless - there is another error that is actually causing the problem. The - exporter can use flags information to simplify how much of the - :c:data:`Py_buffer` structure is filled in with non-default values and/or - raise an error if the object can't support a simpler view of its memory. + Controls the :c:member:`~Py_buffer.readonly` field. If set, the exporter + MUST provide a writable buffer or else report failure. Otherwise, the + exporter MAY provide either a read-only or writable buffer, but the choice + MUST be consistent for all consumers. - On success, 0 is returned and the *view* structure is filled with useful - values. On error, -1 is returned and an exception is raised; the *view* - is left in an undefined state. + .. c:macro:: PyBUF_FORMAT - The following are the possible values to the *flags* arguments. + Controls the :c:member:`~Py_buffer.format` field. If set, this field MUST + be filled in correctly. Otherwise, this field MUST be *NULL*. - .. c:macro:: PyBUF_SIMPLE - This is the default flag. The returned buffer exposes a read-only - memory area. The format of data is assumed to be raw unsigned bytes, - without any particular structure. This is a "stand-alone" flag - constant. It never needs to be '|'d to the others. The exporter will - raise an error if it cannot provide such a contiguous buffer of bytes. +:c:macro:`PyBUF_WRITABLE` can be \|'d to any of the flags in the next section. +Since :c:macro:`PyBUF_SIMPLE` is defined as 0, :c:macro:`PyBUF_WRITABLE` +can be used as a stand-alone flag to request a simple writable buffer. - .. c:macro:: PyBUF_WRITABLE +:c:macro:`PyBUF_FORMAT` can be \|'d to any of the flags except :c:macro:`PyBUF_SIMPLE`. +The latter already implies format ``B`` (unsigned bytes). - Like :c:macro:`PyBUF_SIMPLE`, but the returned buffer is writable. If - the exporter doesn't support writable buffers, an error is raised. - .. c:macro:: PyBUF_STRIDES +shape, strides, suboffsets +~~~~~~~~~~~~~~~~~~~~~~~~~~ - This implies :c:macro:`PyBUF_ND`. The returned buffer must provide - strides information (i.e. the strides cannot be NULL). This would be - used when the consumer can handle strided, discontiguous arrays. - Handling strides automatically assumes you can handle shape. The - exporter can raise an error if a strided representation of the data is - not possible (i.e. without the suboffsets). +The flags that control the logical structure of the memory are listed +in decreasing order of complexity. Note that each flag contains all bits +of the flags below it. - .. c:macro:: PyBUF_ND +.. tabularcolumns:: |p{0.35\linewidth}|l|l|l| - The returned buffer must provide shape information. The memory will be - assumed C-style contiguous (last dimension varies the fastest). The - exporter may raise an error if it cannot provide this kind of - contiguous buffer. If this is not given then shape will be *NULL*. ++-----------------------------+-------+---------+------------+ +| Request | shape | strides | suboffsets | ++=============================+=======+=========+============+ +| .. c:macro:: PyBUF_INDIRECT | yes | yes | if needed | ++-----------------------------+-------+---------+------------+ +| .. c:macro:: PyBUF_STRIDES | yes | yes | NULL | ++-----------------------------+-------+---------+------------+ +| .. c:macro:: PyBUF_ND | yes | NULL | NULL | ++-----------------------------+-------+---------+------------+ +| .. c:macro:: PyBUF_SIMPLE | NULL | NULL | NULL | ++-----------------------------+-------+---------+------------+ - .. c:macro:: PyBUF_C_CONTIGUOUS - PyBUF_F_CONTIGUOUS - PyBUF_ANY_CONTIGUOUS - These flags indicate that the contiguity returned buffer must be - respectively, C-contiguous (last dimension varies the fastest), Fortran - contiguous (first dimension varies the fastest) or either one. All of - these flags imply :c:macro:`PyBUF_STRIDES` and guarantee that the - strides buffer info structure will be filled in correctly. +contiguity requests +~~~~~~~~~~~~~~~~~~~ - .. c:macro:: PyBUF_INDIRECT +C or Fortran contiguity can be explicitly requested, with and without stride +information. Without stride information, the buffer must be C-contiguous. - This flag indicates the returned buffer must have suboffsets - information (which can be NULL if no suboffsets are needed). This can - be used when the consumer can handle indirect array referencing implied - by these suboffsets. This implies :c:macro:`PyBUF_STRIDES`. +.. tabularcolumns:: |p{0.35\linewidth}|l|l|l|l| - .. c:macro:: PyBUF_FORMAT ++-----------------------------------+-------+---------+------------+--------+ +| Request | shape | strides | suboffsets | contig | ++===================================+=======+=========+============+========+ +| .. c:macro:: PyBUF_C_CONTIGUOUS | yes | yes | NULL | C | ++-----------------------------------+-------+---------+------------+--------+ +| .. c:macro:: PyBUF_F_CONTIGUOUS | yes | yes | NULL | F | ++-----------------------------------+-------+---------+------------+--------+ +| .. c:macro:: PyBUF_ANY_CONTIGUOUS | yes | yes | NULL | C or F | ++-----------------------------------+-------+---------+------------+--------+ +| .. c:macro:: PyBUF_ND | yes | NULL | NULL | C | ++-----------------------------------+-------+---------+------------+--------+ - The returned buffer must have true format information if this flag is - provided. This would be used when the consumer is going to be checking - for what 'kind' of data is actually stored. An exporter should always - be able to provide this information if requested. If format is not - explicitly requested then the format must be returned as *NULL* (which - means ``'B'``, or unsigned bytes). - .. c:macro:: PyBUF_STRIDED +compound requests +~~~~~~~~~~~~~~~~~ - This is equivalent to ``(PyBUF_STRIDES | PyBUF_WRITABLE)``. +All possible requests are fully defined by some combination of the flags in +the previous section. For convenience, the buffer protocol provides frequently +used combinations as single flags. - .. c:macro:: PyBUF_STRIDED_RO +In the following table *U* stands for undefined contiguity. The consumer would +have to call :c:func:`PyBuffer_IsContiguous` to determine contiguity. - This is equivalent to ``(PyBUF_STRIDES)``. +.. tabularcolumns:: |p{0.35\linewidth}|l|l|l|l|l|l| - .. c:macro:: PyBUF_RECORDS ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| Request | shape | strides | suboffsets | contig | readonly | format | ++===============================+=======+=========+============+========+==========+========+ +| .. c:macro:: PyBUF_FULL | yes | yes | if needed | U | 0 | yes | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_FULL_RO | yes | yes | if needed | U | 1 or 0 | yes | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_RECORDS | yes | yes | NULL | U | 0 | yes | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_RECORDS_RO | yes | yes | NULL | U | 1 or 0 | yes | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_STRIDED | yes | yes | NULL | U | 0 | NULL | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_STRIDED_RO | yes | yes | NULL | U | 1 or 0 | NULL | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_CONTIG | yes | NULL | NULL | C | 0 | NULL | ++-------------------------------+-------+---------+------------+--------+----------+--------+ +| .. c:macro:: PyBUF_CONTIG_RO | yes | NULL | NULL | C | 1 or 0 | NULL | ++-------------------------------+-------+---------+------------+--------+----------+--------+ - This is equivalent to ``(PyBUF_STRIDES | PyBUF_FORMAT | - PyBUF_WRITABLE)``. - .. c:macro:: PyBUF_RECORDS_RO +Complex arrays +============== + +NumPy-style: shape and strides +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The logical structure of NumPy-style arrays is defined by :c:member:`~Py_buffer.itemsize`, +:c:member:`~Py_buffer.ndim`, :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides`. + +If ``ndim == 0``, the memory location pointed to by :c:member:`~Py_buffer.buf` is +interpreted as a scalar of size :c:member:`~Py_buffer.itemsize`. In that case, +both :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides` are *NULL*. + +If :c:member:`~Py_buffer.strides` is *NULL*, the array is interpreted as +a standard n-dimensional C-array. Otherwise, the consumer must access an +n-dimensional array as follows: + + ``ptr = (char *)buf + indices[0] * strides[0] + ... + indices[n-1] * strides[n-1]`` + ``item = *((typeof(item) *)ptr);`` + + +As noted above, :c:member:`~Py_buffer.buf` can point to any location within +the actual memory block. An exporter can check the validity of a buffer with +this function: + +.. code-block:: python + + def verify_structure(memlen, itemsize, ndim, shape, strides, offset): + """Verify that the parameters represent a valid array within + the bounds of the allocated memory: + char *mem: start of the physical memory block + memlen: length of the physical memory block + offset: (char *)buf - mem + """ + if offset % itemsize: + return False + if offset < 0 or offset+itemsize > memlen: + return False + if any(v % itemsize for v in strides): + return False + + if ndim <= 0: + return ndim == 0 and not shape and not strides + if 0 in shape: + return True + + imin = sum(strides[j]*(shape[j]-1) for j in range(ndim) + if strides[j] <= 0) + imax = sum(strides[j]*(shape[j]-1) for j in range(ndim) + if strides[j] > 0) + + return 0 <= offset+imin and offset+imax+itemsize <= memlen + + +PIL-style: shape, strides and suboffsets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In addition to the regular items, PIL-style arrays can contain pointers +that must be followed in order to get to the next element in a dimension. +For example, the regular three-dimensional C-array ``char v[2][2][3]`` can +also be viewed as an array of 2 pointers to 2 two-dimensional arrays: +``char (*v[2])[2][3]``. In suboffsets representation, those two pointers +can be embedded at the start of :c:member:`~Py_buffer.buf`, pointing +to two ``char x[2][3]`` arrays that can be located anywhere in memory. + + +Here is a function that returns a pointer to the element in an N-D array +pointed to by an N-dimensional index when there are both non-NULL strides +and suboffsets:: + + void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides, + Py_ssize_t *suboffsets, Py_ssize_t *indices) { + char *pointer = (char*)buf; + int i; + for (i = 0; i < ndim; i++) { + pointer += strides[i] * indices[i]; + if (suboffsets[i] >=0 ) { + pointer = *((char**)pointer) + suboffsets[i]; + } + } + return (void*)pointer; + } - This is equivalent to ``(PyBUF_STRIDES | PyBUF_FORMAT)``. - .. c:macro:: PyBUF_FULL +Buffer-related functions +======================== - This is equivalent to ``(PyBUF_INDIRECT | PyBUF_FORMAT | - PyBUF_WRITABLE)``. +.. c:function:: int PyObject_CheckBuffer(PyObject *obj) - .. c:macro:: PyBUF_FULL_RO + Return 1 if *obj* supports the buffer interface otherwise 0. When 1 is + returned, it doesn't guarantee that :c:func:`PyObject_GetBuffer` will + succeed. - This is equivalent to ``(PyBUF_INDIRECT | PyBUF_FORMAT)``. - .. c:macro:: PyBUF_CONTIG +.. c:function:: int PyObject_GetBuffer(PyObject *exporter, Py_buffer *view, int flags) - This is equivalent to ``(PyBUF_ND | PyBUF_WRITABLE)``. + Send a request to *exporter* to fill in *view* as specified by *flags*. + If the exporter cannot provide a buffer of the exact type, it MUST raise + :c:data:`PyExc_BufferError`, set :c:member:`view->obj` to *NULL* and + return -1. - .. c:macro:: PyBUF_CONTIG_RO + On success, fill in *view*, set :c:member:`view->obj` to a new reference + to *exporter* and return 0. In the case of chained buffer providers + that redirect requests to a single object, :c:member:`view->obj` MAY + refer to this object instead of *exporter* (See :ref:`Buffer Object Structures <buffer-structs>`). - This is equivalent to ``(PyBUF_ND)``. + Successful calls to :c:func:`PyObject_GetBuffer` must be paired with calls + to :c:func:`PyBuffer_Release`, similar to :c:func:`malloc` and :c:func:`free`. + Thus, after the consumer is done with the buffer, :c:func:`PyBuffer_Release` + must be called exactly once. .. c:function:: void PyBuffer_Release(Py_buffer *view) - Release the buffer *view*. This should be called when the buffer is no - longer being used as it may free memory from it. + Release the buffer *view* and decrement the reference count for + :c:member:`view->obj`. This function MUST be called when the buffer + is no longer being used, otherwise reference leaks may occur. + + It is an error to call this function on a buffer that was not obtained via + :c:func:`PyObject_GetBuffer`. .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *) - Return the implied :c:data:`~Py_buffer.itemsize` from the struct-stype - :c:data:`~Py_buffer.format`. + Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`. + This function is not yet implemented. -.. c:function:: int PyBuffer_IsContiguous(Py_buffer *view, char fortran) +.. c:function:: int PyBuffer_IsContiguous(Py_buffer *view, char order) - Return 1 if the memory defined by the *view* is C-style (*fortran* is - ``'C'``) or Fortran-style (*fortran* is ``'F'``) contiguous or either one - (*fortran* is ``'A'``). Return 0 otherwise. + Return 1 if the memory defined by the *view* is C-style (*order* is + ``'C'``) or Fortran-style (*order* is ``'F'``) contiguous or either one + (*order* is ``'A'``). Return 0 otherwise. -.. c:function:: void PyBuffer_FillContiguousStrides(int ndim, Py_ssize_t *shape, Py_ssize_t *strides, Py_ssize_t itemsize, char fortran) +.. c:function:: void PyBuffer_FillContiguousStrides(int ndim, Py_ssize_t *shape, Py_ssize_t *strides, Py_ssize_t itemsize, char order) Fill the *strides* array with byte-strides of a contiguous (C-style if - *fortran* is ``'C'`` or Fortran-style if *fortran* is ``'F'``) array of the + *order* is ``'C'`` or Fortran-style if *order* is ``'F'``) array of the given shape with the given number of bytes per element. -.. c:function:: int PyBuffer_FillInfo(Py_buffer *view, PyObject *obj, void *buf, Py_ssize_t len, int readonly, int infoflags) +.. c:function:: int PyBuffer_FillInfo(Py_buffer *view, PyObject *exporter, void *buf, Py_ssize_t len, int readonly, int flags) + + Handle buffer requests for an exporter that wants to expose *buf* of size *len* + with writability set according to *readonly*. *buf* is interpreted as a sequence + of unsigned bytes. + + The *flags* argument indicates the request type. This function always fills in + *view* as specified by flags, unless *buf* has been designated as read-only + and :c:macro:`PyBUF_WRITABLE` is set in *flags*. + + On success, set :c:member:`view->obj` to a new reference to *exporter* and + return 0. Otherwise, raise :c:data:`PyExc_BufferError`, set + :c:member:`view->obj` to *NULL* and return -1; + + If this function is used as part of a :ref:`getbufferproc <buffer-structs>`, + *exporter* MUST be set to the exporting object. Otherwise, *exporter* MUST + be NULL. + - Fill in a buffer-info structure, *view*, correctly for an exporter that can - only share a contiguous chunk of memory of "unsigned bytes" of the given - length. Return 0 on success and -1 (with raising an error) on error. diff --git a/Doc/c-api/bytearray.rst b/Doc/c-api/bytearray.rst index 95ded96..8202205 100644 --- a/Doc/c-api/bytearray.rst +++ b/Doc/c-api/bytearray.rst @@ -40,7 +40,7 @@ Direct API functions .. c:function:: PyObject* PyByteArray_FromObject(PyObject *o) Return a new bytearray object from any object, *o*, that implements the - buffer protocol. + :ref:`buffer protocol <bufferobjects>`. .. XXX expand about the buffer protocol, at least somewhere diff --git a/Doc/c-api/bytes.rst b/Doc/c-api/bytes.rst index 12ec80c..5666dac 100644 --- a/Doc/c-api/bytes.rst +++ b/Doc/c-api/bytes.rst @@ -62,6 +62,8 @@ called with a non-bytes parameter. .. % because not all compilers support the %z width modifier -- we fake it .. % when necessary via interpolating PY_FORMAT_SIZE_T. + .. tabularcolumns:: |l|l|L| + +-------------------+---------------+--------------------------------+ | Format Characters | Type | Comment | +===================+===============+================================+ diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index 6932bb1..57e8072 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -31,11 +31,11 @@ bound into a function. Return true if *co* is a :class:`code` object -.. c:function:: int PyCode_GetNumFree(PyObject *co) +.. c:function:: int PyCode_GetNumFree(PyCodeObject *co) Return the number of free variables in *co*. -.. c:function:: PyCodeObject *PyCode_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *lnotab) +.. c:function:: PyCodeObject* PyCode_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *lnotab) Return a new code object. If you need a dummy code object to create a frame, use :c:func:`PyCode_NewEmpty` instead. Calling @@ -43,7 +43,7 @@ bound into a function. version since the definition of the bytecode changes often. -.. c:function:: int PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) +.. c:function:: PyCodeObject* PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) Return a new empty code object with the specified filename, function name, and first line number. It is illegal to diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst index 8207ae0..83252af 100644 --- a/Doc/c-api/codec.rst +++ b/Doc/c-api/codec.rst @@ -52,19 +52,19 @@ and *NULL* returned. .. c:function:: PyObject* PyCodec_IncrementalEncoder(const char *encoding, const char *errors) - Get an :class:`IncrementalEncoder` object for the given *encoding*. + Get an :class:`~codecs.IncrementalEncoder` object for the given *encoding*. .. c:function:: PyObject* PyCodec_IncrementalDecoder(const char *encoding, const char *errors) - Get an :class:`IncrementalDecoder` object for the given *encoding*. + Get an :class:`~codecs.IncrementalDecoder` object for the given *encoding*. .. c:function:: PyObject* PyCodec_StreamReader(const char *encoding, PyObject *stream, const char *errors) - Get a :class:`StreamReader` factory function for the given *encoding*. + Get a :class:`~codecs.StreamReader` factory function for the given *encoding*. .. c:function:: PyObject* PyCodec_StreamWriter(const char *encoding, PyObject *stream, const char *errors) - Get a :class:`StreamWriter` factory function for the given *encoding*. + Get a :class:`~codecs.StreamWriter` factory function for the given *encoding*. Registry API for Unicode encoding error handlers diff --git a/Doc/c-api/conversion.rst b/Doc/c-api/conversion.rst index dfc0a3a..9578f98 100644 --- a/Doc/c-api/conversion.rst +++ b/Doc/c-api/conversion.rst @@ -119,13 +119,13 @@ The following functions provide locale-independent string to number conversions. .. versionadded:: 3.1 -.. c:function:: char* PyOS_stricmp(char *s1, char *s2) +.. c:function:: int PyOS_stricmp(char *s1, char *s2) Case insensitive comparison of strings. The function works almost identically to :c:func:`strcmp` except that it ignores the case. -.. c:function:: char* PyOS_strnicmp(char *s1, char *s2, Py_ssize_t size) +.. c:function:: int PyOS_strnicmp(char *s1, char *s2, Py_ssize_t size) Case insensitive comparison of strings. The function works almost identically to :c:func:`strncmp` except that it ignores the case. diff --git a/Doc/c-api/datetime.rst b/Doc/c-api/datetime.rst index fcd1395..39542bd 100644 --- a/Doc/c-api/datetime.rst +++ b/Doc/c-api/datetime.rst @@ -170,6 +170,31 @@ and the type is not checked: Return the microsecond, as an int from 0 through 999999. +Macros to extract fields from time delta objects. The argument must be an +instance of :c:data:`PyDateTime_Delta`, including subclasses. The argument must +not be *NULL*, and the type is not checked: + +.. c:function:: int PyDateTime_DELTA_GET_DAYS(PyDateTime_Delta *o) + + Return the number of days, as an int from -999999999 to 999999999. + + .. versionadded:: 3.3 + + +.. c:function:: int PyDateTime_DELTA_GET_SECONDS(PyDateTime_Delta *o) + + Return the number of seconds, as an int from 0 through 86399. + + .. versionadded:: 3.3 + + +.. c:function:: int PyDateTime_DELTA_GET_MICROSECOND(PyDateTime_Delta *o) + + Return the number of microseconds, as an int from 0 through 999999. + + .. versionadded:: 3.3 + + Macros for the convenience of modules implementing the DB API: .. c:function:: PyObject* PyDateTime_FromTimestamp(PyObject *args) diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst index 6df84e0..6bacc32 100644 --- a/Doc/c-api/dict.rst +++ b/Doc/c-api/dict.rst @@ -36,11 +36,11 @@ Dictionary Objects Return a new empty dictionary, or *NULL* on failure. -.. c:function:: PyObject* PyDictProxy_New(PyObject *dict) +.. c:function:: PyObject* PyDictProxy_New(PyObject *mapping) - Return a proxy object for a mapping which enforces read-only behavior. - This is normally used to create a proxy to prevent modification of the - dictionary for non-dynamic class types. + Return a :class:`types.MappingProxyType` object for a mapping which + enforces read-only behavior. This is normally used to create a view to + prevent modification of the dictionary for non-dynamic class types. .. c:function:: void PyDict_Clear(PyObject *p) @@ -209,3 +209,10 @@ Dictionary Objects for key, value in seq2: if override or key not in a: a[key] = value + + +.. c:function:: int PyDict_ClearFreeList() + + Clear the free list. Return the total number of freed items. + + .. versionadded:: 3.3 diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 6f13c80..0aa892d 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -129,6 +129,41 @@ in various ways. There is a separate error indicator for each thread. exception state. +.. c:function:: void PyErr_GetExcInfo(PyObject **ptype, PyObject **pvalue, PyObject **ptraceback) + + Retrieve the exception info, as known from ``sys.exc_info()``. This refers + to an exception that was already caught, not to an exception that was + freshly raised. Returns new references for the three objects, any of which + may be *NULL*. Does not modify the exception info state. + + .. note:: + + This function is not normally used by code that wants to handle exceptions. + Rather, it can be used when code needs to save and restore the exception + state temporarily. Use :c:func:`PyErr_SetExcInfo` to restore or clear the + exception state. + + .. versionadded:: 3.3 + + +.. c:function:: void PyErr_SetExcInfo(PyObject *type, PyObject *value, PyObject *traceback) + + Set the exception info, as known from ``sys.exc_info()``. This refers + to an exception that was already caught, not to an exception that was + freshly raised. This function steals the references of the arguments. + To clear the exception state, pass *NULL* for all three arguments. + For general rules about the three arguments, see :c:func:`PyErr_Restore`. + + .. note:: + + This function is not normally used by code that wants to handle exceptions. + Rather, it can be used when code needs to save and restore the exception + state temporarily. Use :c:func:`PyErr_GetExcInfo` to read the exception + state. + + .. versionadded:: 3.3 + + .. c:function:: void PyErr_SetString(PyObject *type, const char *message) This is the most common way to set the error indicator. The first argument @@ -187,13 +222,19 @@ in various ways. There is a separate error indicator for each thread. when the system call returns an error. -.. c:function:: PyObject* PyErr_SetFromErrnoWithFilename(PyObject *type, const char *filename) +.. c:function:: PyObject* PyErr_SetFromErrnoWithFilenameObject(PyObject *type, PyObject *filenameObject) Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if - *filename* is not *NULL*, it is passed to the constructor of *type* as a third - parameter. In the case of exceptions such as :exc:`IOError` and :exc:`OSError`, - this is used to define the :attr:`filename` attribute of the exception instance. - *filename* is decoded from the filesystem encoding + *filenameObject* is not *NULL*, it is passed to the constructor of *type* as + a third parameter. In the case of exceptions such as :exc:`IOError` and + :exc:`OSError`, this is used to define the :attr:`filename` attribute of the + exception instance. + + +.. c:function:: PyObject* PyErr_SetFromErrnoWithFilename(PyObject *type, const char *filename) + + Similar to :c:func:`PyErr_SetFromErrnoWithFilenameObject`, but the filename + is given as a C string. *filename* is decoded from the filesystem encoding (:func:`sys.getfilesystemencoding`). @@ -215,21 +256,43 @@ in various ways. There is a separate error indicator for each thread. specifying the exception type to be raised. Availability: Windows. +.. c:function:: PyObject* PyErr_SetFromWindowsErrWithFilenameObject(int ierr, PyObject *filenameObject) + + Similar to :c:func:`PyErr_SetFromWindowsErr`, with the additional behavior + that if *filenameObject* is not *NULL*, it is passed to the constructor of + :exc:`WindowsError` as a third parameter. Availability: Windows. + + .. c:function:: PyObject* PyErr_SetFromWindowsErrWithFilename(int ierr, const char *filename) - Similar to :c:func:`PyErr_SetFromWindowsErr`, with the additional behavior that - if *filename* is not *NULL*, it is passed to the constructor of - :exc:`WindowsError` as a third parameter. *filename* is decoded from the - filesystem encoding (:func:`sys.getfilesystemencoding`). Availability: - Windows. + Similar to :c:func:`PyErr_SetFromWindowsErrWithFilenameObject`, but the + filename is given as a C string. *filename* is decoded from the filesystem + encoding (:func:`sys.getfilesystemencoding`). Availability: Windows. + +.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObject(PyObject *type, int ierr, PyObject *filename) -.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilename(PyObject *type, int ierr, char *filename) + Similar to :c:func:`PyErr_SetFromWindowsErrWithFilenameObject`, with an + additional parameter specifying the exception type to be raised. + Availability: Windows. + + +.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilename(PyObject *type, int ierr, const char *filename) Similar to :c:func:`PyErr_SetFromWindowsErrWithFilename`, with an additional parameter specifying the exception type to be raised. Availability: Windows. +.. c:function:: PyObject* PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path) + + This is a convenience function to raise :exc:`ImportError`. *msg* will be + set as the exception's message string. *name* and *path*, both of which can + be ``NULL``, will be set as the :exc:`ImportError`'s respective ``name`` + and ``path`` attributes. + + .. versionadded:: 3.3 + + .. c:function:: void PyErr_SyntaxLocationEx(char *filename, int lineno, int col_offset) Set file, line, and offset information for the current exception. If the @@ -238,12 +301,12 @@ in various ways. There is a separate error indicator for each thread. is a :exc:`SyntaxError`. *filename* is decoded from the filesystem encoding (:func:`sys.getfilesystemencoding`). -.. versionadded:: 3.2 + .. versionadded:: 3.2 .. c:function:: void PyErr_SyntaxLocation(char *filename, int lineno) - Like :c:func:`PyErr_SyntaxLocationExc`, but the col_offset parameter is + Like :c:func:`PyErr_SyntaxLocationEx`, but the col_offset parameter is omitted. @@ -311,6 +374,7 @@ in various ways. There is a separate error indicator for each thread. .. versionadded:: 3.2 + .. c:function:: int PyErr_CheckSignals() .. index:: @@ -421,17 +485,18 @@ Exception Objects .. c:function:: PyObject* PyException_GetCause(PyObject *ex) - Return the cause (another exception instance set by ``raise ... from ...``) - associated with the exception as a new reference, as accessible from Python - through :attr:`__cause__`. If there is no cause associated, this returns - *NULL*. + Return the cause (either an exception instance, or :const:`None`, + set by ``raise ... from ...``) associated with the exception as a new + reference, as accessible from Python through :attr:`__cause__`. -.. c:function:: void PyException_SetCause(PyObject *ex, PyObject *ctx) +.. c:function:: void PyException_SetCause(PyObject *ex, PyObject *cause) - Set the cause associated with the exception to *ctx*. Use *NULL* to clear - it. There is no type check to make sure that *ctx* is an exception instance. - This steals a reference to *ctx*. + Set the cause associated with the exception to *cause*. Use *NULL* to clear + it. There is no type check to make sure that *cause* is either an exception + instance or :const:`None`. This steals a reference to *cause*. + + :attr:`__suppress_context__` is implicitly set to ``True`` by this function. .. _unicodeexceptions: @@ -525,7 +590,7 @@ recursion depth automatically). Marks a point where a recursive C-level call is about to be performed. - If :const:`USE_STACKCHECK` is defined, this function checks if the the OS + If :const:`USE_STACKCHECK` is defined, this function checks if the OS stack overflowed using :c:func:`PyOS_CheckStack`. In this is the case, it sets a :exc:`MemoryError` and returns a nonzero value. @@ -542,28 +607,28 @@ recursion depth automatically). Ends a :c:func:`Py_EnterRecursiveCall`. Must be called once for each *successful* invocation of :c:func:`Py_EnterRecursiveCall`. -Properly implementing :attr:`tp_repr` for container types requires +Properly implementing :c:member:`~PyTypeObject.tp_repr` for container types requires special recursion handling. In addition to protecting the stack, -:attr:`tp_repr` also needs to track objects to prevent cycles. The +:c:member:`~PyTypeObject.tp_repr` also needs to track objects to prevent cycles. The following two functions facilitate this functionality. Effectively, these are the C equivalent to :func:`reprlib.recursive_repr`. .. c:function:: int Py_ReprEnter(PyObject *object) - Called at the beginning of the :attr:`tp_repr` implementation to + Called at the beginning of the :c:member:`~PyTypeObject.tp_repr` implementation to detect cycles. If the object has already been processed, the function returns a - positive integer. In that case the :attr:`tp_repr` implementation + positive integer. In that case the :c:member:`~PyTypeObject.tp_repr` implementation should return a string object indicating a cycle. As examples, :class:`dict` objects return ``{...}`` and :class:`list` objects return ``[...]``. The function will return a negative integer if the recursion limit - is reached. In that case the :attr:`tp_repr` implementation should + is reached. In that case the :c:member:`~PyTypeObject.tp_repr` implementation should typically return ``NULL``. - Otherwise, the function returns zero and the :attr:`tp_repr` + Otherwise, the function returns zero and the :c:member:`~PyTypeObject.tp_repr` implementation can continue normally. .. c:function:: void Py_ReprLeave(PyObject *object) @@ -582,65 +647,116 @@ All standard Python exceptions are available as global variables whose names are :c:type:`PyObject\*`; they are all class objects. For completeness, here are all the variables: -+-------------------------------------+----------------------------+----------+ -| C Name | Python Name | Notes | -+=====================================+============================+==========+ -| :c:data:`PyExc_BaseException` | :exc:`BaseException` | \(1) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_Exception` | :exc:`Exception` | \(1) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_ArithmeticError` | :exc:`ArithmeticError` | \(1) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_LookupError` | :exc:`LookupError` | \(1) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_AssertionError` | :exc:`AssertionError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_AttributeError` | :exc:`AttributeError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_EOFError` | :exc:`EOFError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_EnvironmentError` | :exc:`EnvironmentError` | \(1) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_FloatingPointError` | :exc:`FloatingPointError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_IOError` | :exc:`IOError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_ImportError` | :exc:`ImportError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_IndexError` | :exc:`IndexError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_KeyError` | :exc:`KeyError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_KeyboardInterrupt` | :exc:`KeyboardInterrupt` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_MemoryError` | :exc:`MemoryError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_NameError` | :exc:`NameError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_NotImplementedError` | :exc:`NotImplementedError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_OSError` | :exc:`OSError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_OverflowError` | :exc:`OverflowError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_ReferenceError` | :exc:`ReferenceError` | \(2) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_RuntimeError` | :exc:`RuntimeError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_SyntaxError` | :exc:`SyntaxError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_SystemError` | :exc:`SystemError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_SystemExit` | :exc:`SystemExit` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_TypeError` | :exc:`TypeError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_ValueError` | :exc:`ValueError` | | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_WindowsError` | :exc:`WindowsError` | \(3) | -+-------------------------------------+----------------------------+----------+ -| :c:data:`PyExc_ZeroDivisionError` | :exc:`ZeroDivisionError` | | -+-------------------------------------+----------------------------+----------+ ++-----------------------------------------+---------------------------------+----------+ +| C Name | Python Name | Notes | ++=========================================+=================================+==========+ +| :c:data:`PyExc_BaseException` | :exc:`BaseException` | \(1) | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_Exception` | :exc:`Exception` | \(1) | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ArithmeticError` | :exc:`ArithmeticError` | \(1) | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_LookupError` | :exc:`LookupError` | \(1) | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_AssertionError` | :exc:`AssertionError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_AttributeError` | :exc:`AttributeError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_BlockingIOError` | :exc:`BlockingIOError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_BrokenPipeError` | :exc:`BrokenPipeError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ChildProcessError` | :exc:`ChildProcessError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ConnectionError` | :exc:`ConnectionError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ConnectionAbortedError` | :exc:`ConnectionAbortedError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ConnectionRefusedError` | :exc:`ConnectionRefusedError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ConnectionResetError` | :exc:`ConnectionResetError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_FileExistsError` | :exc:`FileExistsError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_FileNotFoundError` | :exc:`FileNotFoundError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_EOFError` | :exc:`EOFError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_FloatingPointError` | :exc:`FloatingPointError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ImportError` | :exc:`ImportError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_IndexError` | :exc:`IndexError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_InterruptedError` | :exc:`InterruptedError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_IsADirectoryError` | :exc:`IsADirectoryError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_KeyError` | :exc:`KeyError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_KeyboardInterrupt` | :exc:`KeyboardInterrupt` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_MemoryError` | :exc:`MemoryError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_NameError` | :exc:`NameError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_NotADirectoryError` | :exc:`NotADirectoryError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_NotImplementedError` | :exc:`NotImplementedError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_OSError` | :exc:`OSError` | \(1) | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_OverflowError` | :exc:`OverflowError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_PermissionError` | :exc:`PermissionError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ProcessLookupError` | :exc:`ProcessLookupError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ReferenceError` | :exc:`ReferenceError` | \(2) | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_RuntimeError` | :exc:`RuntimeError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_SyntaxError` | :exc:`SyntaxError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_SystemError` | :exc:`SystemError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_TimeoutError` | :exc:`TimeoutError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_SystemExit` | :exc:`SystemExit` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_TypeError` | :exc:`TypeError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ValueError` | :exc:`ValueError` | | ++-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ZeroDivisionError` | :exc:`ZeroDivisionError` | | ++-----------------------------------------+---------------------------------+----------+ + +.. versionadded:: 3.3 + :c:data:`PyExc_BlockingIOError`, :c:data:`PyExc_BrokenPipeError`, + :c:data:`PyExc_ChildProcessError`, :c:data:`PyExc_ConnectionError`, + :c:data:`PyExc_ConnectionAbortedError`, :c:data:`PyExc_ConnectionRefusedError`, + :c:data:`PyExc_ConnectionResetError`, :c:data:`PyExc_FileExistsError`, + :c:data:`PyExc_FileNotFoundError`, :c:data:`PyExc_InterruptedError`, + :c:data:`PyExc_IsADirectoryError`, :c:data:`PyExc_NotADirectoryError`, + :c:data:`PyExc_PermissionError`, :c:data:`PyExc_ProcessLookupError` + and :c:data:`PyExc_TimeoutError` were introduced following :pep:`3151`. + + +These are compatibility aliases to :c:data:`PyExc_OSError`: + ++-------------------------------------+----------+ +| C Name | Notes | ++=====================================+==========+ +| :c:data:`PyExc_EnvironmentError` | | ++-------------------------------------+----------+ +| :c:data:`PyExc_IOError` | | ++-------------------------------------+----------+ +| :c:data:`PyExc_WindowsError` | \(3) | ++-------------------------------------+----------+ + +.. versionchanged:: 3.3 + These aliases used to be separate exception types. + .. index:: single: PyExc_BaseException @@ -649,28 +765,42 @@ the variables: single: PyExc_LookupError single: PyExc_AssertionError single: PyExc_AttributeError + single: PyExc_BlockingIOError + single: PyExc_BrokenPipeError + single: PyExc_ConnectionError + single: PyExc_ConnectionAbortedError + single: PyExc_ConnectionRefusedError + single: PyExc_ConnectionResetError single: PyExc_EOFError - single: PyExc_EnvironmentError + single: PyExc_FileExistsError + single: PyExc_FileNotFoundError single: PyExc_FloatingPointError - single: PyExc_IOError single: PyExc_ImportError single: PyExc_IndexError + single: PyExc_InterruptedError + single: PyExc_IsADirectoryError single: PyExc_KeyError single: PyExc_KeyboardInterrupt single: PyExc_MemoryError single: PyExc_NameError + single: PyExc_NotADirectoryError single: PyExc_NotImplementedError single: PyExc_OSError single: PyExc_OverflowError + single: PyExc_PermissionError + single: PyExc_ProcessLookupError single: PyExc_ReferenceError single: PyExc_RuntimeError single: PyExc_SyntaxError single: PyExc_SystemError single: PyExc_SystemExit + single: PyExc_TimeoutError single: PyExc_TypeError single: PyExc_ValueError - single: PyExc_WindowsError single: PyExc_ZeroDivisionError + single: PyExc_EnvironmentError + single: PyExc_IOError + single: PyExc_WindowsError Notes: diff --git a/Doc/c-api/file.rst b/Doc/c-api/file.rst index c5a4a59..cc190c9 100644 --- a/Doc/c-api/file.rst +++ b/Doc/c-api/file.rst @@ -40,9 +40,9 @@ the :mod:`io` APIs instead. Return the file descriptor associated with *p* as an :c:type:`int`. If the object is an integer, its value is returned. If not, the - object's :meth:`fileno` method is called if it exists; the method must return - an integer, which is returned as the file descriptor value. Sets an - exception and returns ``-1`` on failure. + object's :meth:`~io.IOBase.fileno` method is called if it exists; the + method must return an integer, which is returned as the file descriptor + value. Sets an exception and returns ``-1`` on failure. .. c:function:: PyObject* PyFile_GetLine(PyObject *p, int n) @@ -50,7 +50,8 @@ the :mod:`io` APIs instead. .. index:: single: EOFError (built-in exception) Equivalent to ``p.readline([n])``, this function reads one line from the - object *p*. *p* may be a file object or any object with a :meth:`readline` + object *p*. *p* may be a file object or any object with a + :meth:`~io.IOBase.readline` method. If *n* is ``0``, exactly one line is read, regardless of the length of the line. If *n* is greater than ``0``, no more than *n* bytes will be read from the file; a partial line can be returned. In both cases, an empty string diff --git a/Doc/c-api/function.rst b/Doc/c-api/function.rst index 31805fd..ad98322 100644 --- a/Doc/c-api/function.rst +++ b/Doc/c-api/function.rst @@ -38,6 +38,16 @@ There are a few functions specific to Python functions. object, the argument defaults and closure are set to *NULL*. +.. c:function:: PyObject* PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname) + + As :c:func:`PyFunction_New`, but also allows to set the function object's + ``__qualname__`` attribute. *qualname* should be a unicode object or NULL; + if NULL, the ``__qualname__`` attribute is set to the same value as its + ``__name__`` attribute. + + .. versionadded:: 3.3 + + .. c:function:: PyObject* PyFunction_GetCode(PyObject *op) Return the code object associated with the function object *op*. diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index 3875ff2..9f6ad85 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -12,10 +12,10 @@ other objects, or which only store references to atomic types (such as numbers or strings), do not need to provide any explicit support for garbage collection. -To create a container type, the :attr:`tp_flags` field of the type object must +To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must include the :const:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the -:attr:`tp_traverse` handler. If instances of the type are mutable, a -:attr:`tp_clear` implementation must also be provided. +:c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a +:c:member:`~PyTypeObject.tp_clear` implementation must also be provided. .. data:: Py_TPFLAGS_HAVE_GC @@ -57,7 +57,7 @@ Constructors for container types must conform to two rules: Adds the object *op* to the set of container objects tracked by the collector. The collector can run at unexpected times so objects must be valid while being tracked. This should be called once all the fields - followed by the :attr:`tp_traverse` handler become valid, usually near the + followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the end of the constructor. @@ -86,8 +86,8 @@ rules: Remove the object *op* from the set of container objects tracked by the collector. Note that :c:func:`PyObject_GC_Track` can be called again on this object to add it back to the set of tracked objects. The deallocator - (:attr:`tp_dealloc` handler) should call this for the object before any of - the fields used by the :attr:`tp_traverse` handler become invalid. + (:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of + the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid. .. c:function:: void _PyObject_GC_UNTRACK(PyObject *op) @@ -95,19 +95,19 @@ rules: A macro version of :c:func:`PyObject_GC_UnTrack`. It should not be used for extension modules. -The :attr:`tp_traverse` handler accepts a function parameter of this type: +The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: .. c:type:: int (*visitproc)(PyObject *object, void *arg) - Type of the visitor function passed to the :attr:`tp_traverse` handler. + Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler. The function should be called with an object to traverse as *object* and - the third parameter to the :attr:`tp_traverse` handler as *arg*. The + the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The Python core uses several visitor functions to implement cyclic garbage detection; it's not expected that users will need to write their own visitor functions. -The :attr:`tp_traverse` handler must have the following type: +The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type: .. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg) @@ -119,15 +119,15 @@ The :attr:`tp_traverse` handler must have the following type: object argument. If *visit* returns a non-zero value that value should be returned immediately. -To simplify writing :attr:`tp_traverse` handlers, a :c:func:`Py_VISIT` macro is -provided. In order to use this macro, the :attr:`tp_traverse` implementation +To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is +provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation must name its arguments exactly *visit* and *arg*: .. c:function:: void Py_VISIT(PyObject *o) Call the *visit* callback, with arguments *o* and *arg*. If *visit* returns - a non-zero value, then return it. Using this macro, :attr:`tp_traverse` + a non-zero value, then return it. Using this macro, :c:member:`~PyTypeObject.tp_traverse` handlers look like:: static int @@ -138,7 +138,7 @@ must name its arguments exactly *visit* and *arg*: return 0; } -The :attr:`tp_clear` handler must be of the :c:type:`inquiry` type, or *NULL* +The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or *NULL* if the object is immutable. diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index cf48363..270152e 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -30,13 +30,13 @@ Importing Modules .. c:function:: PyObject* PyImport_ImportModuleNoBlock(const char *name) - This version of :c:func:`PyImport_ImportModule` does not block. It's intended - to be used in C functions that import other modules to execute a function. - The import may block if another thread holds the import lock. The function - :c:func:`PyImport_ImportModuleNoBlock` never blocks. It first tries to fetch - the module from sys.modules and falls back to :c:func:`PyImport_ImportModule` - unless the lock is held, in which case the function will raise an - :exc:`ImportError`. + This function is a deprecated alias of :c:func:`PyImport_ImportModule`. + + .. versionchanged:: 3.3 + This function used to fail immediately when the import lock was held + by another thread. In Python 3.3 though, the locking scheme switched + to per-module locks for most purposes, so this function's special + behaviour isn't needed anymore. .. c:function:: PyObject* PyImport_ImportModuleEx(char *name, PyObject *globals, PyObject *locals, PyObject *fromlist) @@ -44,8 +44,7 @@ Importing Modules .. index:: builtin: __import__ Import a module. This is best described by referring to the built-in Python - function :func:`__import__`, as the standard :func:`__import__` function calls - this function directly. + function :func:`__import__`. The return value is a new reference to the imported module or top-level package, or *NULL* with an exception set on failure. Like for @@ -57,7 +56,7 @@ Importing Modules :c:func:`PyImport_ImportModule`. -.. c:function:: PyObject* PyImport_ImportModuleLevel(char *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level) +.. c:function:: PyObject* PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level) Import a module. This is best described by referring to the built-in Python function :func:`__import__`, as the standard :func:`__import__` function calls @@ -68,6 +67,16 @@ Importing Modules the return value when a submodule of a package was requested is normally the top-level package, unless a non-empty *fromlist* was given. + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyImport_ImportModuleLevel(char *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level) + + Similar to :c:func:`PyImport_ImportModuleLevelObject`, but the name is an + UTF-8 encoded string instead of a Unicode object. + + .. versionchanged:: 3.3 + Negative values for *level* are no longer accepted. .. c:function:: PyObject* PyImport_Import(PyObject *name) @@ -86,7 +95,7 @@ Importing Modules an exception set on failure (the module still exists in this case). -.. c:function:: PyObject* PyImport_AddModule(const char *name) +.. c:function:: PyObject* PyImport_AddModuleObject(PyObject *name) Return the module object corresponding to a module name. The *name* argument may be of the form ``package.module``. First check the modules dictionary if @@ -100,6 +109,14 @@ Importing Modules or one of its variants to import a module. Package structures implied by a dotted name for *name* are not created if not already present. + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyImport_AddModule(const char *name) + + Similar to :c:func:`PyImport_AddModuleObject`, but the name is a UTF-8 + encoded string instead of a Unicode object. + .. c:function:: PyObject* PyImport_ExecCodeModule(char *name, PyObject *co) @@ -136,25 +153,43 @@ Importing Modules See also :c:func:`PyImport_ExecCodeModuleWithPathnames`. -.. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname, char *cpathname) +.. c:function:: PyObject* PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, PyObject *cpathname) Like :c:func:`PyImport_ExecCodeModuleEx`, but the :attr:`__cached__` attribute of the module object is set to *cpathname* if it is non-``NULL``. Of the three functions, this is the preferred one to use. + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname, char *cpathname) + + Like :c:func:`PyImport_ExecCodeModuleObject`, but *name*, *pathname* and + *cpathname* are UTF-8 encoded strings. Attempts are also made to figure out + what the value for *pathname* should be from *cpathname* if the former is + set to ``NULL``. + .. versionadded:: 3.2 + .. versionchanged:: 3.3 + Uses :func:`imp.source_from_cache()` in calculating the source path if + only the bytecode path is provided. + .. c:function:: long PyImport_GetMagicNumber() Return the magic number for Python bytecode files (a.k.a. :file:`.pyc` and :file:`.pyo` files). The magic number should be present in the first four bytes - of the bytecode file, in little-endian byte order. + of the bytecode file, in little-endian byte order. Returns -1 on error. + + .. versionchanged:: 3.3 + Return value of -1 upon failure. .. c:function:: const char * PyImport_GetMagicTag() Return the magic tag string for :pep:`3147` format Python bytecode file - names. + names. Keep in mind that the value at ``sys.implementation.cache_tag`` is + authoritative and should be used instead of this function. .. versionadded:: 3.2 @@ -200,7 +235,7 @@ Importing Modules For internal use only. -.. c:function:: int PyImport_ImportFrozenModule(char *name) +.. c:function:: int PyImport_ImportFrozenModuleObject(PyObject *name) Load a frozen module named *name*. Return ``1`` for success, ``0`` if the module is not found, and ``-1`` with an exception set if the initialization @@ -208,6 +243,14 @@ Importing Modules :c:func:`PyImport_ImportModule`. (Note the misnomer --- this function would reload the module if it was already imported.) + .. versionadded:: 3.3 + + +.. c:function:: int PyImport_ImportFrozenModule(char *name) + + Similar to :c:func:`PyImport_ImportFrozenModuleObject`, but the name is a + UTF-8 encoded string instead of a Unicode object. + .. c:type:: struct _frozen @@ -247,13 +290,13 @@ Importing Modules Structure describing a single entry in the list of built-in modules. Each of these structures gives the name and initialization function for a module built - into the interpreter. Programs which embed Python may use an array of these - structures in conjunction with :c:func:`PyImport_ExtendInittab` to provide - additional built-in modules. The structure is defined in - :file:`Include/import.h` as:: + into the interpreter. The name is an ASCII encoded string. Programs which + embed Python may use an array of these structures in conjunction with + :c:func:`PyImport_ExtendInittab` to provide additional built-in modules. + The structure is defined in :file:`Include/import.h` as:: struct _inittab { - char *name; + char *name; /* ASCII encoded string */ PyObject* (*initfunc)(void); }; diff --git a/Doc/c-api/index.rst b/Doc/c-api/index.rst index 2ce7b98..3bfbaf4 100644 --- a/Doc/c-api/index.rst +++ b/Doc/c-api/index.rst @@ -13,6 +13,7 @@ document the API functions in detail. :maxdepth: 2 intro.rst + stable.rst veryhigh.rst refcounting.rst exceptions.rst @@ -22,3 +23,4 @@ document the API functions in detail. init.rst memory.rst objimpl.rst + apiabiversion.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 7507e3b..705509f 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -442,6 +442,9 @@ pointer. standard :mod:`zlib` and :mod:`hashlib` modules release the GIL when compressing or hashing data. + +.. _gilstate: + Non-Python created threads -------------------------- @@ -646,7 +649,7 @@ with sub-interpreters: :c:func:`PyGILState_Release` on the same thread. -.. c:function:: PyThreadState PyGILState_GetThisThreadState() +.. c:function:: PyThreadState* PyGILState_GetThisThreadState() Get the current thread state for this thread. May return ``NULL`` if no GILState API has been used on the current thread. Note that the main thread @@ -905,41 +908,43 @@ Asynchronous Notifications A mechanism is provided to make asynchronous notifications to the main interpreter thread. These notifications take the form of a function -pointer and a void argument. +pointer and a void pointer argument. -.. index:: single: setcheckinterval() (in module sys) - -Every check interval, when the global interpreter lock is released and -reacquired, Python will also call any such provided functions. This can be used -for example by asynchronous IO handlers. The notification can be scheduled from -a worker thread and the actual call than made at the earliest convenience by the -main thread where it has possession of the global interpreter lock and can -perform any Python API calls. .. c:function:: int Py_AddPendingCall(int (*func)(void *), void *arg) .. index:: single: Py_AddPendingCall() - Post a notification to the Python main thread. If successful, *func* will be - called with the argument *arg* at the earliest convenience. *func* will be - called having the global interpreter lock held and can thus use the full - Python API and can take any action such as setting object attributes to - signal IO completion. It must return 0 on success, or -1 signalling an - exception. The notification function won't be interrupted to perform another - asynchronous notification recursively, but it can still be interrupted to - switch threads if the global interpreter lock is released, for example, if it - calls back into Python code. + Schedule a function to be called from the main interpreter thread. On + success, 0 is returned and *func* is queued for being called in the + main thread. On failure, -1 is returned without setting any exception. - This function returns 0 on success in which case the notification has been - scheduled. Otherwise, for example if the notification buffer is full, it - returns -1 without setting any exception. + When successfully queued, *func* will be *eventually* called from the + main interpreter thread with the argument *arg*. It will be called + asynchronously with respect to normally running Python code, but with + both these conditions met: - This function can be called on any thread, be it a Python thread or some - other system thread. If it is a Python thread, it doesn't matter if it holds - the global interpreter lock or not. + * on a :term:`bytecode` boundary; + * with the main thread holding the :term:`global interpreter lock` + (*func* can therefore use the full C API). - .. versionadded:: 3.1 + *func* must return 0 on success, or -1 on failure with an exception + set. *func* won't be interrupted to perform another asynchronous + notification recursively, but it can still be interrupted to switch + threads if the global interpreter lock is released. + This function doesn't need a current thread state to run, and it doesn't + need the global interpreter lock. + + .. warning:: + This is a low-level function, only useful for very special cases. + There is no guarantee that *func* will be called as quick as + possible. If the main thread is busy executing a system call, + *func* won't be called before the system call returns. This + function is generally **not** suitable for calling Python code from + arbitrary C threads. Instead, use the :ref:`PyGILState API<gilstate>`. + + .. versionadded:: 3.1 .. _profiling: diff --git a/Doc/c-api/iter.rst b/Doc/c-api/iter.rst index 3f0f554..2ba444d 100644 --- a/Doc/c-api/iter.rst +++ b/Doc/c-api/iter.rst @@ -5,7 +5,7 @@ Iterator Protocol ================= -There are only a couple of functions specifically for working with iterators. +There are two functions specifically for working with iterators. .. c:function:: int PyIter_Check(PyObject *o) @@ -14,11 +14,10 @@ There are only a couple of functions specifically for working with iterators. .. c:function:: PyObject* PyIter_Next(PyObject *o) - Return the next value from the iteration *o*. If the object is an iterator, - this retrieves the next value from the iteration, and returns *NULL* with no - exception set if there are no remaining items. If the object is not an - iterator, :exc:`TypeError` is raised, or if there is an error in retrieving the - item, returns *NULL* and passes along the exception. + Return the next value from the iteration *o*. The object must be an iterator + (it is up to the caller to check this). If there are no remaining values, + returns *NULL* with no exception set. If an error occurs while retrieving + the item, returns *NULL* and passes along the exception. To write a loop which iterates over an iterator, the C code should look something like this:: diff --git a/Doc/c-api/list.rst b/Doc/c-api/list.rst index feb9015..5b263a7 100644 --- a/Doc/c-api/list.rst +++ b/Doc/c-api/list.rst @@ -142,3 +142,10 @@ List Objects Return a new tuple object containing the contents of *list*; equivalent to ``tuple(list)``. + + +.. c:function:: int PyList_ClearFreeList() + + Clear the free list. Return the total number of freed items. + + .. versionadded:: 3.3 diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst index 72be017..d5430fd 100644 --- a/Doc/c-api/long.rst +++ b/Doc/c-api/long.rst @@ -100,6 +100,20 @@ All integers are implemented as "long" integer objects of arbitrary size. string is first encoded to a byte string using :c:func:`PyUnicode_EncodeDecimal` and then converted using :c:func:`PyLong_FromString`. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyLong_FromUnicodeObject`. + + +.. c:function:: PyObject* PyLong_FromUnicodeObject(PyObject *u, int base) + + Convert a sequence of Unicode digits in the string *u* to a Python integer + value. The Unicode string is first encoded to a byte string using + :c:func:`PyUnicode_EncodeDecimal` and then converted using + :c:func:`PyLong_FromString`. + + .. versionadded:: 3.3 + .. c:function:: PyObject* PyLong_FromVoidPtr(void *p) diff --git a/Doc/c-api/memoryview.rst b/Doc/c-api/memoryview.rst index 6b49cdf..5e50977 100644 --- a/Doc/c-api/memoryview.rst +++ b/Doc/c-api/memoryview.rst @@ -17,16 +17,21 @@ any other object. Create a memoryview object from an object that provides the buffer interface. If *obj* supports writable buffer exports, the memoryview object will be - readable and writable, otherwise it will be read-only. + read/write, otherwise it may be either read-only or read/write at the + discretion of the exporter. +.. c:function:: PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags) + + Create a memoryview object using *mem* as the underlying buffer. + *flags* can be one of :c:macro:`PyBUF_READ` or :c:macro:`PyBUF_WRITE`. + + .. versionadded:: 3.3 .. c:function:: PyObject *PyMemoryView_FromBuffer(Py_buffer *view) Create a memoryview object wrapping the given buffer structure *view*. - The memoryview object then owns the buffer represented by *view*, which - means you shouldn't try to call :c:func:`PyBuffer_Release` yourself: it - will be done on deallocation of the memoryview object. - + For simple byte buffers, :c:func:`PyMemoryView_FromMemory` is the preferred + function. .. c:function:: PyObject *PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char order) @@ -43,10 +48,16 @@ any other object. currently allowed to create subclasses of :class:`memoryview`. -.. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *obj) +.. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *mview) + + Return a pointer to the memoryview's private copy of the exporter's buffer. + *mview* **must** be a memoryview instance; this macro doesn't check its type, + you must do it yourself or you will risk crashes. + +.. c:function:: Py_buffer *PyMemoryView_GET_BASE(PyObject *mview) - Return a pointer to the buffer structure wrapped by the given - memoryview object. The object **must** be a memoryview instance; - this macro doesn't check its type, you must do it yourself or you - will risk crashes. + Return either a pointer to the exporting object that the memoryview is based + on or *NULL* if the memoryview has been created by one of the functions + :c:func:`PyMemoryView_FromMemory` or :c:func:`PyMemoryView_FromBuffer`. + *mview* **must** be a memoryview instance. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index ffd68e3..bd46170 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -29,7 +29,7 @@ There are only a few functions special to module objects. :c:data:`PyModule_Type`. -.. c:function:: PyObject* PyModule_New(const char *name) +.. c:function:: PyObject* PyModule_NewObject(PyObject *name) .. index:: single: __name__ (module attribute) @@ -40,6 +40,14 @@ There are only a few functions special to module objects. Only the module's :attr:`__doc__` and :attr:`__name__` attributes are filled in; the caller is responsible for providing a :attr:`__file__` attribute. + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyModule_New(const char *name) + + Similar to :c:func:`PyImport_NewObject`, but the name is an UTF-8 encoded + string instead of a Unicode object. + .. c:function:: PyObject* PyModule_GetDict(PyObject *module) @@ -52,7 +60,7 @@ There are only a few functions special to module objects. manipulate a module's :attr:`__dict__`. -.. c:function:: char* PyModule_GetName(PyObject *module) +.. c:function:: PyObject* PyModule_GetNameObject(PyObject *module) .. index:: single: __name__ (module attribute) @@ -61,15 +69,13 @@ There are only a few functions special to module objects. Return *module*'s :attr:`__name__` value. If the module does not provide one, or if it is not a string, :exc:`SystemError` is raised and *NULL* is returned. + .. versionadded:: 3.3 -.. c:function:: char* PyModule_GetFilename(PyObject *module) - Similar to :c:func:`PyModule_GetFilenameObject` but return the filename - encoded to 'utf-8'. +.. c:function:: char* PyModule_GetName(PyObject *module) - .. deprecated:: 3.2 - :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on - unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead. + Similar to :c:func:`PyModule_GetNameObject` but return the name encoded to + ``'utf-8'``. .. c:function:: PyObject* PyModule_GetFilenameObject(PyObject *module) @@ -81,11 +87,21 @@ There are only a few functions special to module objects. Return the name of the file from which *module* was loaded using *module*'s :attr:`__file__` attribute. If this is not defined, or if it is not a unicode string, raise :exc:`SystemError` and return *NULL*; otherwise return - a reference to a :c:type:`PyUnicodeObject`. + a reference to a Unicode object. .. versionadded:: 3.2 +.. c:function:: char* PyModule_GetFilename(PyObject *module) + + Similar to :c:func:`PyModule_GetFilenameObject` but return the filename + encoded to 'utf-8'. + + .. deprecated:: 3.2 + :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on + unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead. + + .. c:function:: void* PyModule_GetState(PyObject *module) Return the "state" of the module, that is, a pointer to the block of memory @@ -99,6 +115,26 @@ There are only a few functions special to module objects. created, or *NULL* if the module wasn't created with :c:func:`PyModule_Create`. +.. c:function:: PyObject* PyState_FindModule(PyModuleDef *def) + + Returns the module object that was created from *def* for the current interpreter. + This method requires that the module object has been attached to the interpreter state with + :c:func:`PyState_AddModule` beforehand. In case the corresponding module object is not + found or has not been attached to the interpreter state yet, it returns NULL. + +.. c:function:: int PyState_AddModule(PyObject *module, PyModuleDef *def) + + Attaches the module object passed to the function to the interpreter state. This allows + the module object to be accessible via + :c:func:`PyState_FindModule`. + + .. versionadded:: 3.3 + +.. c:function:: int PyState_RemoveModule(PyModuleDef *def) + + Removes the module object created from *def* from the interpreter state. + + .. versionadded:: 3.3 Initializing C modules ^^^^^^^^^^^^^^^^^^^^^^ @@ -146,16 +182,22 @@ These functions are usually used in the module initialization function. .. c:member:: Py_ssize_t m_size - If the module object needs additional memory, this should be set to the - number of bytes to allocate; a pointer to the block of memory can be - retrieved with :c:func:`PyModule_GetState`. If no memory is needed, set - this to ``-1``. + Some modules allow re-initialization (calling their ``PyInit_*`` function + more than once). These modules should keep their state in a per-module + memory area that can be retrieved with :c:func:`PyModule_GetState`. This memory should be used, rather than static globals, to hold per-module state, since it is then safe for use in multiple sub-interpreters. It is freed when the module object is deallocated, after the :c:member:`m_free` function has been called, if present. + Setting ``m_size`` to ``-1`` means that the module can not be + re-initialized because it has global state. Setting it to a non-negative + value means that the module can be re-initialized and specifies the + additional amount of memory it requires for its state. + + See :PEP:`3121` for more details. + .. c:member:: PyMethodDef* m_methods A pointer to a table of module-level functions, described by diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index d0d45ad..0aba360 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -6,6 +6,19 @@ Object Protocol =============== +.. c:var:: PyObject* Py_NotImplemented + + The ``NotImplemented`` singleton, used to signal that an operation is + not implemented for the given type combination. + + +.. c:macro:: Py_RETURN_NOTIMPLEMENTED + + Properly handle returning :c:data:`Py_NotImplemented` from within a C + function (that is, increment the reference count of NotImplemented and + return it). + + .. c:function:: int PyObject_Print(PyObject *o, FILE *fp, int flags) Print an object *o*, on file *fp*. Returns ``-1`` on error. The flags argument @@ -47,8 +60,8 @@ Object Protocol Generic attribute getter function that is meant to be put into a type object's ``tp_getattro`` slot. It looks for a descriptor in the dictionary of classes in the object's MRO as well as an attribute in the object's - :attr:`__dict__` (if present). As outlined in :ref:`descriptors`, data - descriptors take preference over instance attributes, while non-data + :attr:`~object.__dict__` (if present). As outlined in :ref:`descriptors`, + data descriptors take preference over instance attributes, while non-data descriptors don't. Otherwise, an :exc:`AttributeError` is raised. @@ -72,8 +85,8 @@ Object Protocol object's ``tp_setattro`` slot. It looks for a data descriptor in the dictionary of classes in the object's MRO, and if found it takes preference over setting the attribute in the instance dictionary. Otherwise, the - attribute is set in the object's :attr:`__dict__` (if present). Otherwise, - an :exc:`AttributeError` is raised and ``-1`` is returned. + attribute is set in the object's :attr:`~object.__dict__` (if present). + Otherwise, an :exc:`AttributeError` is raised and ``-1`` is returned. .. c:function:: int PyObject_DelAttr(PyObject *o, PyObject *attr_name) @@ -88,6 +101,22 @@ Object Protocol This is the equivalent of the Python statement ``del o.attr_name``. +.. c:function:: PyObject* PyType_GenericGetDict(PyObject *o, void *context) + + A generic implementation for the getter of a ``__dict__`` descriptor. It + creates the dictionary if necessary. + + .. versionadded:: 3.3 + + +.. c:function:: int PyType_GenericSetDict(PyObject *o, void *context) + + A generic implementation for the setter of a ``__dict__`` descriptor. This + implementation does not allow the dictionary to be deleted. + + .. versionadded:: 3.3 + + .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, @@ -131,10 +160,10 @@ Object Protocol a string similar to that returned by :c:func:`PyObject_Repr` in Python 2. Called by the :func:`ascii` built-in function. + .. index:: string; PyObject_Str (C function) -.. c:function:: PyObject* PyObject_Str(PyObject *o) - .. index:: builtin: str +.. c:function:: PyObject* PyObject_Str(PyObject *o) Compute a string representation of object *o*. Returns the string representation on success, *NULL* on failure. This is the equivalent of the @@ -160,9 +189,9 @@ Object Protocol be done against every entry in *cls*. The result will be ``1`` when at least one of the checks returns ``1``, otherwise it will be ``0``. If *inst* is not a class instance and *cls* is neither a type object, nor a class object, nor a - tuple, *inst* must have a :attr:`__class__` attribute --- the class relationship - of the value of that attribute with *cls* will be used to determine the result - of this function. + tuple, *inst* must have a :attr:`~instance.__class__` attribute --- the + class relationship of the value of that attribute with *cls* will be used + to determine the result of this function. Subclass determination is done in a fairly straightforward way, but includes a @@ -172,9 +201,9 @@ of. If :class:`A` and :class:`B` are class objects, :class:`B` is a subclass of either is not a class object, a more general mechanism is used to determine the class relationship of the two objects. When testing if *B* is a subclass of *A*, if *A* is *B*, :c:func:`PyObject_IsSubclass` returns true. If *A* and *B* -are different objects, *B*'s :attr:`__bases__` attribute is searched in a -depth-first fashion for *A* --- the presence of the :attr:`__bases__` attribute -is considered sufficient for this determination. +are different objects, *B*'s :attr:`~class.__bases__` attribute is searched in +a depth-first fashion for *A* --- the presence of the :attr:`~class.__bases__` +attribute is considered sufficient for this determination. .. c:function:: int PyObject_IsSubclass(PyObject *derived, PyObject *cls) diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index 66b47c4..7f4d534 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -140,7 +140,7 @@ subtypes but not for instances of :class:`frozenset` or its subtypes. Return 1 if found and removed, 0 if not found (no action taken), and -1 if an error is encountered. Does not raise :exc:`KeyError` for missing keys. Raise a - :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`discard` + :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`~set.discard` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise :exc:`PyExc_SystemError` if *set* is an not an instance of :class:`set` or its subtype. @@ -157,3 +157,10 @@ subtypes but not for instances of :class:`frozenset` or its subtypes. .. c:function:: int PySet_Clear(PyObject *set) Empty an existing set of all elements. + + +.. c:function:: int PySet_ClearFreeList() + + Clear the free list. Return the total number of freed items. + + .. versionadded:: 3.3 diff --git a/Doc/c-api/stable.rst b/Doc/c-api/stable.rst new file mode 100644 index 0000000..063f856 --- /dev/null +++ b/Doc/c-api/stable.rst @@ -0,0 +1,38 @@ +.. highlightlang:: c + +.. _stable: + +*********************************** +Stable Application Binary Interface +*********************************** + +Traditionally, the C API of Python will change with every release. Most changes +will be source-compatible, typically by only adding API, rather than changing +existing API or removing API (although some interfaces do get removed after +being deprecated first). + +Unfortunately, the API compatibility does not extend to binary compatibility +(the ABI). The reason is primarily the evolution of struct definitions, where +addition of a new field, or changing the type of a field, might not break the +API, but can break the ABI. As a consequence, extension modules need to be +recompiled for every Python release (although an exception is possible on Unix +when none of the affected interfaces are used). In addition, on Windows, +extension modules link with a specific pythonXY.dll and need to be recompiled to +link with a newer one. + +Since Python 3.2, a subset of the API has been declared to guarantee a stable +ABI. Extension modules wishing to use this API (called "limited API") need to +define ``Py_LIMITED_API``. A number of interpreter details then become hidden +from the extension module; in return, a module is built that works on any 3.x +version (x>=2) without recompilation. + +In some cases, the stable ABI needs to be extended with new functions. +Extension modules wishing to use these new APIs need to set ``Py_LIMITED_API`` +to the ``PY_VERSION_HEX`` value (see :ref:`apiabiversion`) of the minimum Python +version they want to support (e.g. ``0x03030000`` for Python 3.3). Such modules +will work on all subsequent Python releases, but fail to load (because of +missing symbols) on the older releases. + +As of Python 3.2, the set of functions available to the limited API is +documented in PEP 384. In the C API documentation, API elements that are not +part of the limited API are marked as "Not part of the limited API." diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 3cbfe5b..184affb 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -108,3 +108,103 @@ Tuple Objects .. c:function:: int PyTuple_ClearFreeList() Clear the free list. Return the total number of freed items. + + +Struct Sequence Objects +----------------------- + +Struct sequence objects are the C equivalent of :func:`~collections.namedtuple` +objects, i.e. a sequence whose items can also be accessed through attributes. +To create a struct sequence, you first have to create a specific struct sequence +type. + +.. c:function:: PyTypeObject* PyStructSequence_NewType(PyStructSequence_Desc *desc) + + Create a new struct sequence type from the data in *desc*, described below. Instances + of the resulting type can be created with :c:func:`PyStructSequence_New`. + + +.. c:function:: void PyStructSequence_InitType(PyTypeObject *type, PyStructSequence_Desc *desc) + + Initializes a struct sequence type *type* from *desc* in place. + + +.. c:type:: PyStructSequence_Desc + + Contains the meta information of a struct sequence type to create. + + +-------------------+------------------------------+------------------------------------+ + | Field | C Type | Meaning | + +===================+==============================+====================================+ + | ``name`` | ``char *`` | name of the struct sequence type | + +-------------------+------------------------------+------------------------------------+ + | ``doc`` | ``char *`` | pointer to docstring for the type | + | | | or NULL to omit | + +-------------------+------------------------------+------------------------------------+ + | ``fields`` | ``PyStructSequence_Field *`` | pointer to *NULL*-terminated array | + | | | with field names of the new type | + +-------------------+------------------------------+------------------------------------+ + | ``n_in_sequence`` | ``int`` | number of fields visible to the | + | | | Python side (if used as tuple) | + +-------------------+------------------------------+------------------------------------+ + + +.. c:type:: PyStructSequence_Field + + Describes a field of a struct sequence. As a struct sequence is modeled as a + tuple, all fields are typed as :c:type:`PyObject\*`. The index in the + :attr:`fields` array of the :c:type:`PyStructSequence_Desc` determines which + field of the struct sequence is described. + + +-----------+---------------+--------------------------------------+ + | Field | C Type | Meaning | + +===========+===============+======================================+ + | ``name`` | ``char *`` | name for the field or *NULL* to end | + | | | the list of named fields, set to | + | | | PyStructSequence_UnnamedField to | + | | | leave unnamed | + +-----------+---------------+--------------------------------------+ + | ``doc`` | ``char *`` | field docstring or *NULL* to omit | + +-----------+---------------+--------------------------------------+ + + +.. c:var:: char* PyStructSequence_UnnamedField + + Special value for a field name to leave it unnamed. + + +.. c:function:: PyObject* PyStructSequence_New(PyTypeObject *type) + + Creates an instance of *type*, which must have been created with + :c:func:`PyStructSequence_NewType`. + + +.. c:function:: PyObject* PyStructSequence_GetItem(PyObject *p, Py_ssize_t pos) + + Return the object at position *pos* in the struct sequence pointed to by *p*. + No bounds checking is performed. + + +.. c:function:: PyObject* PyStructSequence_GET_ITEM(PyObject *p, Py_ssize_t pos) + + Macro equivalent of :c:func:`PyStructSequence_GetItem`. + + +.. c:function:: void PyStructSequence_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o) + + Sets the field at index *pos* of the struct sequence *p* to value *o*. Like + :c:func:`PyTuple_SET_ITEM`, this should only be used to fill in brand new + instances. + + .. note:: + + This function "steals" a reference to *o*. + + +.. c:function:: PyObject* PyStructSequence_SET_ITEM(PyObject *p, Py_ssize_t *pos, PyObject *o) + + Macro equivalent of :c:func:`PyStructSequence_SetItem`. + + .. note:: + + This function "steals" a reference to *o*. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index b3386ea..5d83254 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -37,10 +37,10 @@ Type Objects .. c:function:: long PyType_GetFlags(PyTypeObject* type) - Return the :attr:`tp_flags` member of *type*. This function is primarily + Return the :c:member:`~PyTypeObject.tp_flags` member of *type*. This function is primarily meant for use with `Py_LIMITED_API`; the individual flag bits are guaranteed to be stable across Python releases, but access to - :attr:`tp_flags` itself is not part of the limited API. + :c:member:`~PyTypeObject.tp_flags` itself is not part of the limited API. .. versionadded:: 3.2 @@ -51,13 +51,13 @@ Type Objects modification of the attributes or base classes of the type. -.. c:function:: int PyType_HasFeature(PyObject *o, int feature) +.. c:function:: int PyType_HasFeature(PyTypeObject *o, int feature) Return true if the type object *o* sets the feature *feature*. Type features are denoted by single bit flags. -.. c:function:: int PyType_IS_GC(PyObject *o) +.. c:function:: int PyType_IS_GC(PyTypeObject *o) Return true if the type object includes support for the cycle detector; this tests the type flag :const:`Py_TPFLAGS_HAVE_GC`. @@ -70,13 +70,14 @@ Type Objects .. c:function:: PyObject* PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems) - XXX: Document. - + Generic handler for the :c:member:`~PyTypeObject.tp_alloc` slot of a type object. Use + Python's default memory allocation mechanism to allocate a new instance and + initialize all its contents to *NULL*. .. c:function:: PyObject* PyType_GenericNew(PyTypeObject *type, PyObject *args, PyObject *kwds) - XXX: Document. - + Generic handler for the :c:member:`~PyTypeObject.tp_new` slot of a type object. Create a + new instance using the type's :c:member:`~PyTypeObject.tp_alloc` slot. .. c:function:: int PyType_Ready(PyTypeObject *type) @@ -84,3 +85,15 @@ Type Objects their initialization. This function is responsible for adding inherited slots from a type's base class. Return ``0`` on success, or return ``-1`` and sets an exception on error. + +.. c:function:: PyObject* PyType_FromSpec(PyType_Spec *spec) + + Creates and returns a heap type object from the *spec* passed to the function. + +.. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases) + + Creates and returns a heap type object from the *spec*. In addition to that, + the created heap type contains all types contained by the *bases* tuple as base + types. This allows the caller to reference other heap types as base types. + + .. versionadded:: 3.3 diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 68ca9ad..f3089d0 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -35,7 +35,7 @@ definition found there: The type object structure extends the :c:type:`PyVarObject` structure. The :attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the -metatype) initializes :attr:`tp_itemsize`, which means that its instances (i.e. +metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :attr:`ob_size` field. @@ -102,7 +102,7 @@ type objects) *must* have the :attr:`ob_size` field. should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named :class:`T` defined in module :mod:`M` in subpackage :mod:`Q` in package :mod:`P` - should have the :attr:`tp_name` initializer ``"P.Q.M.T"``. + should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For dynamically allocated type objects, this should just be the type name, and the module name explicitly stored in the type dict as the value for key @@ -113,7 +113,7 @@ type objects) *must* have the :attr:`ob_size` field. attribute, and everything after the last dot is made accessible as the :attr:`__name__` attribute. - If no dot is present, the entire :attr:`tp_name` field is made accessible as the + If no dot is present, the entire :c:member:`~PyTypeObject.tp_name` field is made accessible as the :attr:`__name__` attribute, and the :attr:`__module__` attribute is undefined (unless explicitly set in the dictionary, as explained above). This means your type will be impossible to pickle. @@ -127,13 +127,13 @@ type objects) *must* have the :attr:`ob_size` field. These fields allow calculating the size in bytes of instances of the type. There are two kinds of types: types with fixed-length instances have a zero - :attr:`tp_itemsize` field, types with variable-length instances have a non-zero - :attr:`tp_itemsize` field. For a type with fixed-length instances, all - instances have the same size, given in :attr:`tp_basicsize`. + :c:member:`~PyTypeObject.tp_itemsize` field, types with variable-length instances have a non-zero + :c:member:`~PyTypeObject.tp_itemsize` field. For a type with fixed-length instances, all + instances have the same size, given in :c:member:`~PyTypeObject.tp_basicsize`. For a type with variable-length instances, the instances must have an - :attr:`ob_size` field, and the instance size is :attr:`tp_basicsize` plus N - times :attr:`tp_itemsize`, where N is the "length" of the object. The value of + :attr:`ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N + times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the object. The value of N is typically stored in the instance's :attr:`ob_size` field. There are exceptions: for example, ints use a negative :attr:`ob_size` to indicate a negative number, and N is ``abs(ob_size)`` there. Also, the presence of an @@ -146,20 +146,20 @@ type objects) *must* have the :attr:`ob_size` field. :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used to declare the instance struct) and this in turn includes the :attr:`_ob_prev` and :attr:`_ob_next` fields if they are present. This means that the only correct - way to get an initializer for the :attr:`tp_basicsize` is to use the + way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is to use the ``sizeof`` operator on the struct used to declare the instance layout. The basic size does not include the GC header size. These fields are inherited separately by subtypes. If the base type has a - non-zero :attr:`tp_itemsize`, it is generally not safe to set - :attr:`tp_itemsize` to a different non-zero value in a subtype (though this + non-zero :c:member:`~PyTypeObject.tp_itemsize`, it is generally not safe to set + :c:member:`~PyTypeObject.tp_itemsize` to a different non-zero value in a subtype (though this depends on the implementation of the base type). A note about alignment: if the variable items require a particular alignment, - this should be taken care of by the value of :attr:`tp_basicsize`. Example: - suppose a type implements an array of ``double``. :attr:`tp_itemsize` is + this should be taken care of by the value of :c:member:`~PyTypeObject.tp_basicsize`. Example: + suppose a type implements an array of ``double``. :c:member:`~PyTypeObject.tp_itemsize` is ``sizeof(double)``. It is the programmer's responsibility that - :attr:`tp_basicsize` is a multiple of ``sizeof(double)`` (assuming this is the + :c:member:`~PyTypeObject.tp_basicsize` is a multiple of ``sizeof(double)`` (assuming this is the alignment requirement for ``double``). @@ -175,10 +175,10 @@ type objects) *must* have the :attr:`ob_size` field. destructor function should free all references which the instance owns, free all memory buffers owned by the instance (using the freeing function corresponding to the allocation function used to allocate the buffer), and finally (as its - last action) call the type's :attr:`tp_free` function. If the type is not + last action) call the type's :c:member:`~PyTypeObject.tp_free` function. If the type is not subtypable (doesn't have the :const:`Py_TPFLAGS_BASETYPE` flag bit set), it is permissible to call the object deallocator directly instead of via - :attr:`tp_free`. The object deallocator should be the one used to allocate the + :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated using :c:func:`PyObject_New` or :c:func:`PyObject_VarNew`, or :c:func:`PyObject_GC_Del` if the instance was allocated using @@ -192,26 +192,25 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to the instance print function. The print function is only called when the instance is printed to a *real* file; - when it is printed to a pseudo-file (like a :class:`StringIO` instance), the - instance's :attr:`tp_repr` or :attr:`tp_str` function is called to convert it to - a string. These are also called when the type's :attr:`tp_print` field is - *NULL*. A type should never implement :attr:`tp_print` in a way that produces - different output than :attr:`tp_repr` or :attr:`tp_str` would. + when it is printed to a pseudo-file (like a :class:`io.StringIO` instance), the + instance's :c:member:`~PyTypeObject.tp_repr` or :c:member:`~PyTypeObject.tp_str` function is called to convert it to + a string. These are also called when the type's :c:member:`~PyTypeObject.tp_print` field is + *NULL*. A type should never implement :c:member:`~PyTypeObject.tp_print` in a way that produces + different output than :c:member:`~PyTypeObject.tp_repr` or :c:member:`~PyTypeObject.tp_str` would. The print function is called with the same signature as :c:func:`PyObject_Print`: ``int tp_print(PyObject *self, FILE *file, int flags)``. The *self* argument is the instance to be printed. The *file* argument is the stdio file to which it is to be printed. The *flags* argument is composed of flag bits. The only flag bit currently defined is :const:`Py_PRINT_RAW`. When the :const:`Py_PRINT_RAW` - flag bit is set, the instance should be printed the same way as :attr:`tp_str` + flag bit is set, the instance should be printed the same way as :c:member:`~PyTypeObject.tp_str` would format it; when the :const:`Py_PRINT_RAW` flag bit is clear, the instance - should be printed the same was as :attr:`tp_repr` would format it. It should - return ``-1`` and set an exception condition when an error occurred during the - comparison. + should be printed the same way as :c:member:`~PyTypeObject.tp_repr` would format it. It should + return ``-1`` and set an exception condition when an error occurs. - It is possible that the :attr:`tp_print` field will be deprecated. In any case, - it is recommended not to define :attr:`tp_print`, but instead to rely on - :attr:`tp_repr` and :attr:`tp_str` for printing. + It is possible that the :c:member:`~PyTypeObject.tp_print` field will be deprecated. In any case, + it is recommended not to define :c:member:`~PyTypeObject.tp_print`, but instead to rely on + :c:member:`~PyTypeObject.tp_repr` and :c:member:`~PyTypeObject.tp_str` for printing. This field is inherited by subtypes. @@ -221,13 +220,13 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to the get-attribute-string function. This field is deprecated. When it is defined, it should point to a function - that acts the same as the :attr:`tp_getattro` function, but taking a C string + that acts the same as the :c:member:`~PyTypeObject.tp_getattro` function, but taking a C string instead of a Python string object to give the attribute name. The signature is the same as for :c:func:`PyObject_GetAttrString`. - This field is inherited by subtypes together with :attr:`tp_getattro`: a subtype - inherits both :attr:`tp_getattr` and :attr:`tp_getattro` from its base type when - the subtype's :attr:`tp_getattr` and :attr:`tp_getattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype + inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` are both *NULL*. .. c:member:: setattrfunc PyTypeObject.tp_setattr @@ -235,13 +234,13 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to the set-attribute-string function. This field is deprecated. When it is defined, it should point to a function - that acts the same as the :attr:`tp_setattro` function, but taking a C string + that acts the same as the :c:member:`~PyTypeObject.tp_setattro` function, but taking a C string instead of a Python string object to give the attribute name. The signature is the same as for :c:func:`PyObject_SetAttrString`. - This field is inherited by subtypes together with :attr:`tp_setattro`: a subtype - inherits both :attr:`tp_setattr` and :attr:`tp_setattro` from its base type when - the subtype's :attr:`tp_setattr` and :attr:`tp_setattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype + inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*. .. c:member:: void* PyTypeObject.tp_reserved @@ -275,7 +274,7 @@ type objects) *must* have the :attr:`ob_size` field. objects which implement the number protocol. These fields are documented in :ref:`number-structs`. - The :attr:`tp_as_number` field is not inherited, but the contained fields are + The :c:member:`~PyTypeObject.tp_as_number` field is not inherited, but the contained fields are inherited individually. @@ -285,7 +284,7 @@ type objects) *must* have the :attr:`ob_size` field. objects which implement the sequence protocol. These fields are documented in :ref:`sequence-structs`. - The :attr:`tp_as_sequence` field is not inherited, but the contained fields + The :c:member:`~PyTypeObject.tp_as_sequence` field is not inherited, but the contained fields are inherited individually. @@ -295,7 +294,7 @@ type objects) *must* have the :attr:`ob_size` field. objects which implement the mapping protocol. These fields are documented in :ref:`mapping-structs`. - The :attr:`tp_as_mapping` field is not inherited, but the contained fields + The :c:member:`~PyTypeObject.tp_as_mapping` field is not inherited, but the contained fields are inherited individually. @@ -323,9 +322,9 @@ type objects) *must* have the :attr:`ob_size` field. object raises :exc:`TypeError`. This field is inherited by subtypes together with - :attr:`tp_richcompare`: a subtype inherits both of - :attr:`tp_richcompare` and :attr:`tp_hash`, when the subtype's - :attr:`tp_richcompare` and :attr:`tp_hash` are both *NULL*. + :c:member:`~PyTypeObject.tp_richcompare`: a subtype inherits both of + :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash`, when the subtype's + :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` are both *NULL*. .. c:member:: ternaryfunc PyTypeObject.tp_call @@ -363,9 +362,9 @@ type objects) *must* have the :attr:`ob_size` field. convenient to set this field to :c:func:`PyObject_GenericGetAttr`, which implements the normal way of looking for object attributes. - This field is inherited by subtypes together with :attr:`tp_getattr`: a subtype - inherits both :attr:`tp_getattr` and :attr:`tp_getattro` from its base type when - the subtype's :attr:`tp_getattr` and :attr:`tp_getattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattr`: a subtype + inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` are both *NULL*. .. c:member:: setattrofunc PyTypeObject.tp_setattro @@ -376,9 +375,9 @@ type objects) *must* have the :attr:`ob_size` field. convenient to set this field to :c:func:`PyObject_GenericSetAttr`, which implements the normal way of setting object attributes. - This field is inherited by subtypes together with :attr:`tp_setattr`: a subtype - inherits both :attr:`tp_setattr` and :attr:`tp_setattro` from its base type when - the subtype's :attr:`tp_setattr` and :attr:`tp_setattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattr`: a subtype + inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*. .. c:member:: PyBufferProcs* PyTypeObject.tp_as_buffer @@ -387,7 +386,7 @@ type objects) *must* have the :attr:`ob_size` field. which implement the buffer interface. These fields are documented in :ref:`buffer-structs`. - The :attr:`tp_as_buffer` field is not inherited, but the contained fields are + The :c:member:`~PyTypeObject.tp_as_buffer` field is not inherited, but the contained fields are inherited individually. @@ -396,8 +395,8 @@ type objects) *must* have the :attr:`ob_size` field. This field is a bit mask of various flags. Some flags indicate variant semantics for certain situations; others are used to indicate that certain fields in the type object (or in the extension structures referenced via - :attr:`tp_as_number`, :attr:`tp_as_sequence`, :attr:`tp_as_mapping`, and - :attr:`tp_as_buffer`) that were historically not always present are valid; if + :c:member:`~PyTypeObject.tp_as_number`, :c:member:`~PyTypeObject.tp_as_sequence`, :c:member:`~PyTypeObject.tp_as_mapping`, and + :c:member:`~PyTypeObject.tp_as_buffer`) that were historically not always present are valid; if such a flag bit is clear, the type fields it guards must not be accessed and must be considered to have a zero or *NULL* value instead. @@ -407,13 +406,13 @@ type objects) *must* have the :attr:`ob_size` field. inherited if the extension structure is inherited, i.e. the base type's value of the flag bit is copied into the subtype together with a pointer to the extension structure. The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with - the :attr:`tp_traverse` and :attr:`tp_clear` fields, i.e. if the + the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the - :attr:`tp_traverse` and :attr:`tp_clear` fields in the subtype exist and have + :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have *NULL* values. The following bit masks are currently defined; these can be ORed together using - the ``|`` operator to form the value of the :attr:`tp_flags` field. The macro + the ``|`` operator to form the value of the :c:member:`~PyTypeObject.tp_flags` field. The macro :c:func:`PyType_HasFeature` takes a type and a flags value, *tp* and *f*, and checks whether ``tp->tp_flags & f`` is non-zero. @@ -453,7 +452,7 @@ type objects) *must* have the :attr:`ob_size` field. is set, instances must be created using :c:func:`PyObject_GC_New` and destroyed using :c:func:`PyObject_GC_Del`. More information in section :ref:`supporting-cycle-detection`. This bit also implies that the - GC-related fields :attr:`tp_traverse` and :attr:`tp_clear` are present in + GC-related fields :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` are present in the type object. @@ -481,8 +480,8 @@ type objects) *must* have the :attr:`ob_size` field. about Python's garbage collection scheme can be found in section :ref:`supporting-cycle-detection`. - The :attr:`tp_traverse` pointer is used by the garbage collector to detect - reference cycles. A typical implementation of a :attr:`tp_traverse` function + The :c:member:`~PyTypeObject.tp_traverse` pointer is used by the garbage collector to detect + reference cycles. A typical implementation of a :c:member:`~PyTypeObject.tp_traverse` function simply calls :c:func:`Py_VISIT` on each of the instance's members that are Python objects. For example, this is function :c:func:`local_traverse` from the :mod:`_thread` extension module:: @@ -502,15 +501,15 @@ type objects) *must* have the :attr:`ob_size` field. On the other hand, even if you know a member can never be part of a cycle, as a debugging aid you may want to visit it anyway just so the :mod:`gc` module's - :func:`get_referents` function will include it. + :func:`~gc.get_referents` function will include it. Note that :c:func:`Py_VISIT` requires the *visit* and *arg* parameters to :c:func:`local_traverse` to have these specific names; don't name them just anything. - This field is inherited by subtypes together with :attr:`tp_clear` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :attr:`tp_traverse`, and - :attr:`tp_clear` are all inherited from the base type if they are all zero in + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the + :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -519,17 +518,17 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to a clear function for the garbage collector. This is only used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. - The :attr:`tp_clear` member function is used to break reference cycles in cyclic - garbage detected by the garbage collector. Taken together, all :attr:`tp_clear` + The :c:member:`~PyTypeObject.tp_clear` member function is used to break reference cycles in cyclic + garbage detected by the garbage collector. Taken together, all :c:member:`~PyTypeObject.tp_clear` functions in the system must combine to break all reference cycles. This is - subtle, and if in any doubt supply a :attr:`tp_clear` function. For example, - the tuple type does not implement a :attr:`tp_clear` function, because it's + subtle, and if in any doubt supply a :c:member:`~PyTypeObject.tp_clear` function. For example, + the tuple type does not implement a :c:member:`~PyTypeObject.tp_clear` function, because it's possible to prove that no reference cycle can be composed entirely of tuples. - Therefore the :attr:`tp_clear` functions of other types must be sufficient to + Therefore the :c:member:`~PyTypeObject.tp_clear` functions of other types must be sufficient to break any cycle containing a tuple. This isn't immediately obvious, and there's - rarely a good reason to avoid implementing :attr:`tp_clear`. + rarely a good reason to avoid implementing :c:member:`~PyTypeObject.tp_clear`. - Implementations of :attr:`tp_clear` should drop the instance's references to + Implementations of :c:member:`~PyTypeObject.tp_clear` should drop the instance's references to those of its members that may be Python objects, and set its pointers to those members to *NULL*, as in the following example:: @@ -554,18 +553,18 @@ type objects) *must* have the :attr:`ob_size` field. so that *self* knows the contained object can no longer be used. The :c:func:`Py_CLEAR` macro performs the operations in a safe order. - Because the goal of :attr:`tp_clear` functions is to break reference cycles, + Because the goal of :c:member:`~PyTypeObject.tp_clear` functions is to break reference cycles, it's not necessary to clear contained objects like Python strings or Python integers, which can't participate in reference cycles. On the other hand, it may be convenient to clear all contained Python objects, and write the type's - :attr:`tp_dealloc` function to invoke :attr:`tp_clear`. + :c:member:`~PyTypeObject.tp_dealloc` function to invoke :c:member:`~PyTypeObject.tp_clear`. More information about Python's garbage collection scheme can be found in section :ref:`supporting-cycle-detection`. - This field is inherited by subtypes together with :attr:`tp_traverse` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :attr:`tp_traverse`, and - :attr:`tp_clear` are all inherited from the base type if they are all zero in + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the + :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -585,13 +584,13 @@ type objects) *must* have the :attr:`ob_size` field. comparisons makes sense (e.g. ``==`` and ``!=``, but not ``<`` and friends), directly raise :exc:`TypeError` in the rich comparison function. - This field is inherited by subtypes together with :attr:`tp_hash`: - a subtype inherits :attr:`tp_richcompare` and :attr:`tp_hash` when - the subtype's :attr:`tp_richcompare` and :attr:`tp_hash` are both + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_hash`: + a subtype inherits :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` when + the subtype's :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` are both *NULL*. The following constants are defined to be used as the third argument for - :attr:`tp_richcompare` and for :c:func:`PyObject_RichCompare`: + :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: +----------------+------------+ | Constant | Comparison | @@ -619,26 +618,26 @@ type objects) *must* have the :attr:`ob_size` field. instance structure needs to include a field of type :c:type:`PyObject\*` which is initialized to *NULL*. - Do not confuse this field with :attr:`tp_weaklist`; that is the list head for + Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for weak references to the type object itself. This field is inherited by subtypes, but see the rules listed below. A subtype may override this offset; this means that the subtype uses a different weak reference list head than the base type. Since the list head is always found via - :attr:`tp_weaklistoffset`, this should not be a problem. + :c:member:`~PyTypeObject.tp_weaklistoffset`, this should not be a problem. - When a type defined by a class statement has no :attr:`__slots__` declaration, + When a type defined by a class statement has no :attr:`~object.__slots__` declaration, and none of its base types are weakly referenceable, the type is made weakly referenceable by adding a weak reference list head slot to the instance layout - and setting the :attr:`tp_weaklistoffset` of that slot's offset. + and setting the :c:member:`~PyTypeObject.tp_weaklistoffset` of that slot's offset. When a type's :attr:`__slots__` declaration contains a slot named :attr:`__weakref__`, that slot becomes the weak reference list head for instances of the type, and the slot's offset is stored in the type's - :attr:`tp_weaklistoffset`. + :c:member:`~PyTypeObject.tp_weaklistoffset`. When a type's :attr:`__slots__` declaration does not contain a slot named - :attr:`__weakref__`, the type inherits its :attr:`tp_weaklistoffset` from its + :attr:`__weakref__`, the type inherits its :c:member:`~PyTypeObject.tp_weaklistoffset` from its base type. .. c:member:: getiterfunc PyTypeObject.tp_iter @@ -660,7 +659,7 @@ type objects) *must* have the :attr:`ob_size` field. *NULL* too. Its presence signals that the instances of this type are iterators. - Iterator types should also define the :attr:`tp_iter` function, and that + Iterator types should also define the :c:member:`~PyTypeObject.tp_iter` function, and that function should return the iterator instance itself (not a new iterator instance). @@ -675,7 +674,7 @@ type objects) *must* have the :attr:`ob_size` field. structures, declaring regular methods of this type. For each entry in the array, an entry is added to the type's dictionary (see - :attr:`tp_dict` below) containing a method descriptor. + :c:member:`~PyTypeObject.tp_dict` below) containing a method descriptor. This field is not inherited by subtypes (methods are inherited through a different mechanism). @@ -688,7 +687,7 @@ type objects) *must* have the :attr:`ob_size` field. this type. For each entry in the array, an entry is added to the type's dictionary (see - :attr:`tp_dict` below) containing a member descriptor. + :c:member:`~PyTypeObject.tp_dict` below) containing a member descriptor. This field is not inherited by subtypes (members are inherited through a different mechanism). @@ -700,7 +699,7 @@ type objects) *must* have the :attr:`ob_size` field. structures, declaring computed attributes of instances of this type. For each entry in the array, an entry is added to the type's dictionary (see - :attr:`tp_dict` below) containing a getset descriptor. + :c:member:`~PyTypeObject.tp_dict` below) containing a getset descriptor. This field is not inherited by subtypes (computed attributes are inherited through a different mechanism). @@ -748,7 +747,7 @@ type objects) *must* have the :attr:`ob_size` field. .. warning:: It is not safe to use :c:func:`PyDict_SetItem` on or otherwise modify - :attr:`tp_dict` with the dictionary C-API. + :c:member:`~PyTypeObject.tp_dict` with the dictionary C-API. .. c:member:: descrgetfunc PyTypeObject.tp_descr_get @@ -784,7 +783,7 @@ type objects) *must* have the :attr:`ob_size` field. the instance variable dictionary; this offset is used by :c:func:`PyObject_GenericGetAttr`. - Do not confuse this field with :attr:`tp_dict`; that is the dictionary for + Do not confuse this field with :c:member:`~PyTypeObject.tp_dict`; that is the dictionary for attributes of the type object itself. If the value of this field is greater than zero, it specifies the offset from @@ -793,20 +792,20 @@ type objects) *must* have the :attr:`ob_size` field. offset is more expensive to use, and should only be used when the instance structure contains a variable-length part. This is used for example to add an instance variable dictionary to subtypes of :class:`str` or :class:`tuple`. Note - that the :attr:`tp_basicsize` field should account for the dictionary added to + that the :c:member:`~PyTypeObject.tp_basicsize` field should account for the dictionary added to the end in that case, even though the dictionary is not included in the basic object layout. On a system with a pointer size of 4 bytes, - :attr:`tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is + :c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is at the very end of the structure. The real dictionary offset in an instance can be computed from a negative - :attr:`tp_dictoffset` as follows:: + :c:member:`~PyTypeObject.tp_dictoffset` as follows:: dictoffset = tp_basicsize + abs(ob_size)*tp_itemsize + tp_dictoffset if dictoffset is not aligned on sizeof(void*): round up to sizeof(void*) - where :attr:`tp_basicsize`, :attr:`tp_itemsize` and :attr:`tp_dictoffset` are + where :c:member:`~PyTypeObject.tp_basicsize`, :c:member:`~PyTypeObject.tp_itemsize` and :c:member:`~PyTypeObject.tp_dictoffset` are taken from the type object, and :attr:`ob_size` is taken from the instance. The absolute value is taken because ints use the sign of :attr:`ob_size` to store the sign of the number. (There's never a need to do this calculation @@ -815,17 +814,17 @@ type objects) *must* have the :attr:`ob_size` field. This field is inherited by subtypes, but see the rules listed below. A subtype may override this offset; this means that the subtype instances store the dictionary at a difference offset than the base type. Since the dictionary is - always found via :attr:`tp_dictoffset`, this should not be a problem. + always found via :c:member:`~PyTypeObject.tp_dictoffset`, this should not be a problem. - When a type defined by a class statement has no :attr:`__slots__` declaration, + When a type defined by a class statement has no :attr:`~object.__slots__` declaration, and none of its base types has an instance variable dictionary, a dictionary - slot is added to the instance layout and the :attr:`tp_dictoffset` is set to + slot is added to the instance layout and the :c:member:`~PyTypeObject.tp_dictoffset` is set to that slot's offset. When a type defined by a class statement has a :attr:`__slots__` declaration, - the type inherits its :attr:`tp_dictoffset` from its base type. + the type inherits its :c:member:`~PyTypeObject.tp_dictoffset` from its base type. - (Adding a slot named :attr:`__dict__` to the :attr:`__slots__` declaration does + (Adding a slot named :attr:`~object.__dict__` to the :attr:`__slots__` declaration does not have the expected effect, it just causes confusion. Maybe this should be added as a feature just like :attr:`__weakref__` though.) @@ -847,12 +846,12 @@ type objects) *must* have the :attr:`ob_size` field. arguments represent positional and keyword arguments of the call to :meth:`__init__`. - The :attr:`tp_init` function, if not *NULL*, is called when an instance is - created normally by calling its type, after the type's :attr:`tp_new` function - has returned an instance of the type. If the :attr:`tp_new` function returns an + The :c:member:`~PyTypeObject.tp_init` function, if not *NULL*, is called when an instance is + created normally by calling its type, after the type's :c:member:`~PyTypeObject.tp_new` function + has returned an instance of the type. If the :c:member:`~PyTypeObject.tp_new` function returns an instance of some other type that is not a subtype of the original type, no - :attr:`tp_init` function is called; if :attr:`tp_new` returns an instance of a - subtype of the original type, the subtype's :attr:`tp_init` is called. + :c:member:`~PyTypeObject.tp_init` function is called; if :c:member:`~PyTypeObject.tp_new` returns an instance of a + subtype of the original type, the subtype's :c:member:`~PyTypeObject.tp_init` is called. This field is inherited by subtypes. @@ -869,14 +868,14 @@ type objects) *must* have the :attr:`ob_size` field. initialization. It should return a pointer to a block of memory of adequate length for the instance, suitably aligned, and initialized to zeros, but with :attr:`ob_refcnt` set to ``1`` and :attr:`ob_type` set to the type argument. If - the type's :attr:`tp_itemsize` is non-zero, the object's :attr:`ob_size` field + the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :attr:`ob_size` field should be initialized to *nitems* and the length of the allocated memory block should be ``tp_basicsize + nitems*tp_itemsize``, rounded up to a multiple of ``sizeof(void*)``; otherwise, *nitems* is not used and the length of the block - should be :attr:`tp_basicsize`. + should be :c:member:`~PyTypeObject.tp_basicsize`. Do not use this function to do any other instance initialization, not even to - allocate additional memory; that should be done by :attr:`tp_new`. + allocate additional memory; that should be done by :c:member:`~PyTypeObject.tp_new`. This field is inherited by static subtypes, but not by dynamic subtypes (subtypes created by a class statement); in the latter, this field is always set @@ -898,20 +897,20 @@ type objects) *must* have the :attr:`ob_size` field. The subtype argument is the type of the object being created; the *args* and *kwds* arguments represent positional and keyword arguments of the call to the - type. Note that subtype doesn't have to equal the type whose :attr:`tp_new` + type. Note that subtype doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` function is called; it may be a subtype of that type (but not an unrelated type). - The :attr:`tp_new` function should call ``subtype->tp_alloc(subtype, nitems)`` + The :c:member:`~PyTypeObject.tp_new` function should call ``subtype->tp_alloc(subtype, nitems)`` to allocate space for the object, and then do only as much further initialization as is absolutely necessary. Initialization that can safely be - ignored or repeated should be placed in the :attr:`tp_init` handler. A good + ignored or repeated should be placed in the :c:member:`~PyTypeObject.tp_init` handler. A good rule of thumb is that for immutable types, all initialization should take place - in :attr:`tp_new`, while for mutable types, most initialization should be - deferred to :attr:`tp_init`. + in :c:member:`~PyTypeObject.tp_new`, while for mutable types, most initialization should be + deferred to :c:member:`~PyTypeObject.tp_init`. This field is inherited by subtypes, except it is not inherited by static types - whose :attr:`tp_base` is *NULL* or ``&PyBaseObject_Type``. + whose :c:member:`~PyTypeObject.tp_base` is *NULL* or ``&PyBaseObject_Type``. .. c:member:: destructor PyTypeObject.tp_free @@ -935,7 +934,7 @@ type objects) *must* have the :attr:`ob_size` field. The garbage collector needs to know whether a particular object is collectible or not. Normally, it is sufficient to look at the object's type's - :attr:`tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But + :c:member:`~PyTypeObject.tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But some types have a mixture of statically and dynamically allocated instances, and the statically allocated instances are not collectible. Such types should define this function; it should return ``1`` for a collectible instance, and @@ -1006,7 +1005,7 @@ subtypes. .. c:member:: PyTypeObject* PyTypeObject.tp_next - Pointer to the next type object with a non-zero :attr:`tp_allocs` field. + Pointer to the next type object with a non-zero :c:member:`~PyTypeObject.tp_allocs` field. Also, note that, in a garbage collected Python, tp_dealloc may be called from any Python thread, not just the thread which created the object (if the object @@ -1145,13 +1144,13 @@ Sequence Object Structures This function is used by :c:func:`PySequence_Concat` and has the same signature. It is also used by the ``+`` operator, after trying the numeric - addition via the :attr:`tp_as_number.nb_add` slot. + addition via the :c:member:`~PyTypeObject.tp_as_number.nb_add` slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_repeat This function is used by :c:func:`PySequence_Repeat` and has the same signature. It is also used by the ``*`` operator, after trying numeric - multiplication via the :attr:`tp_as_number.nb_mul` slot. + multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_mul` slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_item @@ -1198,46 +1197,88 @@ Buffer Object Structures .. sectionauthor:: Greg J. Stein <greg@lyra.org> .. sectionauthor:: Benjamin Peterson +.. sectionauthor:: Stefan Krah +.. c:type:: PyBufferProcs -The :ref:`buffer interface <bufferobjects>` exports a model where an object can expose its internal -data. + This structure holds pointers to the functions required by the + :ref:`Buffer protocol <bufferobjects>`. The protocol defines how + an exporter object can expose its internal data to consumer objects. -If an object does not export the buffer interface, then its :attr:`tp_as_buffer` -member in the :c:type:`PyTypeObject` structure should be *NULL*. Otherwise, the -:attr:`tp_as_buffer` will point to a :c:type:`PyBufferProcs` structure. +.. c:member:: getbufferproc PyBufferProcs.bf_getbuffer + The signature of this function is:: -.. c:type:: PyBufferProcs + int (PyObject *exporter, Py_buffer *view, int flags); + + Handle a request to *exporter* to fill in *view* as specified by *flags*. + Except for point (3), an implementation of this function MUST take these + steps: + + (1) Check if the request can be met. If not, raise :c:data:`PyExc_BufferError`, + set :c:data:`view->obj` to *NULL* and return -1. + + (2) Fill in the requested fields. + + (3) Increment an internal counter for the number of exports. + + (4) Set :c:data:`view->obj` to *exporter* and increment :c:data:`view->obj`. + + (5) Return 0. + + If *exporter* is part of a chain or tree of buffer providers, two main + schemes can be used: + + * Re-export: Each member of the tree acts as the exporting object and + sets :c:data:`view->obj` to a new reference to itself. + + * Redirect: The buffer request is redirected to the root object of the + tree. Here, :c:data:`view->obj` will be a new reference to the root + object. + + The individual fields of *view* are described in section + :ref:`Buffer structure <buffer-structure>`, the rules how an exporter + must react to specific requests are in section + :ref:`Buffer request types <buffer-request-types>`. + + All memory pointed to in the :c:type:`Py_buffer` structure belongs to + the exporter and must remain valid until there are no consumers left. + :c:member:`~Py_buffer.format`, :c:member:`~Py_buffer.shape`, + :c:member:`~Py_buffer.strides`, :c:member:`~Py_buffer.suboffsets` + and :c:member:`~Py_buffer.internal` + are read-only for the consumer. + + :c:func:`PyBuffer_FillInfo` provides an easy way of exposing a simple + bytes buffer while dealing correctly with all request types. + + :c:func:`PyObject_GetBuffer` is the interface for the consumer that + wraps this function. + +.. c:member:: releasebufferproc PyBufferProcs.bf_releasebuffer + + The signature of this function is:: + + void (PyObject *exporter, Py_buffer *view); - Structure used to hold the function pointers which define an implementation of - the buffer protocol. + Handle a request to release the resources of the buffer. If no resources + need to be released, :c:member:`PyBufferProcs.bf_releasebuffer` may be + *NULL*. Otherwise, a standard implementation of this function will take + these optional steps: - .. c:member:: getbufferproc bf_getbuffer + (1) Decrement an internal counter for the number of exports. - This should fill a :c:type:`Py_buffer` with the necessary data for - exporting the type. The signature of :data:`getbufferproc` is ``int - (PyObject *obj, Py_buffer *view, int flags)``. *obj* is the object to - export, *view* is the :c:type:`Py_buffer` struct to fill, and *flags* gives - the conditions the caller wants the memory under. (See - :c:func:`PyObject_GetBuffer` for all flags.) :c:member:`bf_getbuffer` is - responsible for filling *view* with the appropriate information. - (:c:func:`PyBuffer_FillView` can be used in simple cases.) See - :c:type:`Py_buffer`\s docs for what needs to be filled in. + (2) If the counter is 0, free all memory associated with *view*. + The exporter MUST use the :c:member:`~Py_buffer.internal` field to keep + track of buffer-specific resources. This field is guaranteed to remain + constant, while a consumer MAY pass a copy of the original buffer as the + *view* argument. - .. c:member:: releasebufferproc bf_releasebuffer - This should release the resources of the buffer. The signature of - :c:data:`releasebufferproc` is ``void (PyObject *obj, Py_buffer *view)``. - If the :c:data:`bf_releasebuffer` function is not provided (i.e. it is - *NULL*), then it does not ever need to be called. + This function MUST NOT decrement :c:data:`view->obj`, since that is + done automatically in :c:func:`PyBuffer_Release` (this scheme is + useful for breaking reference cycles). - The exporter of the buffer interface must make sure that any memory - pointed to in the :c:type:`Py_buffer` structure remains valid until - releasebuffer is called. Exporters will need to define a - :c:data:`bf_releasebuffer` function if they can re-allocate their memory, - strides, shape, suboffsets, or format variables which they might share - through the struct bufferinfo. - See :c:func:`PyBuffer_Release`. + :c:func:`PyBuffer_Release` is the interface for the consumer that + wraps this function. diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index d1b57d9..3649cfb 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -6,38 +6,72 @@ Unicode Objects and Codecs -------------------------- .. sectionauthor:: Marc-André Lemburg <mal@lemburg.com> +.. sectionauthor:: Georg Brandl <georg@python.org> Unicode Objects ^^^^^^^^^^^^^^^ +Since the implementation of :pep:`393` in Python 3.3, Unicode objects internally +use a variety of representations, in order to allow handling the complete range +of Unicode characters while staying memory efficient. There are special cases +for strings where all code points are below 128, 256, or 65536; otherwise, code +points must be below 1114112 (which is the full Unicode range). + +:c:type:`Py_UNICODE*` and UTF-8 representations are created on demand and cached +in the Unicode object. The :c:type:`Py_UNICODE*` representation is deprecated +and inefficient; it should be avoided in performance- or memory-sensitive +situations. + +Due to the transition between the old APIs and the new APIs, unicode objects +can internally be in two states depending on how they were created: + +* "canonical" unicode objects are all objects created by a non-deprecated + unicode API. They use the most efficient representation allowed by the + implementation. + +* "legacy" unicode objects have been created through one of the deprecated + APIs (typically :c:func:`PyUnicode_FromUnicode`) and only bear the + :c:type:`Py_UNICODE*` representation; you will have to call + :c:func:`PyUnicode_READY` on them before calling any other API. + + Unicode Type """""""""""" These are the basic Unicode object types used for the Unicode implementation in Python: +.. c:type:: Py_UCS4 + Py_UCS2 + Py_UCS1 + + These types are typedefs for unsigned integer types wide enough to contain + characters of 32 bits, 16 bits and 8 bits, respectively. When dealing with + single Unicode characters, use :c:type:`Py_UCS4`. + + .. versionadded:: 3.3 + .. c:type:: Py_UNICODE - This type represents the storage type which is used by Python internally as - basis for holding Unicode ordinals. Python's default builds use a 16-bit type - for :c:type:`Py_UNICODE` and store Unicode values internally as UCS2. It is also - possible to build a UCS4 version of Python (most recent Linux distributions come - with UCS4 builds of Python). These builds then use a 32-bit type for - :c:type:`Py_UNICODE` and store Unicode data internally as UCS4. On platforms - where :c:type:`wchar_t` is available and compatible with the chosen Python - Unicode build variant, :c:type:`Py_UNICODE` is a typedef alias for - :c:type:`wchar_t` to enhance native platform compatibility. On all other - platforms, :c:type:`Py_UNICODE` is a typedef alias for either :c:type:`unsigned - short` (UCS2) or :c:type:`unsigned long` (UCS4). + This is a typedef of :c:type:`wchar_t`, which is a 16-bit type or 32-bit type + depending on the platform. -Note that UCS2 and UCS4 Python builds are not binary compatible. Please keep -this in mind when writing extensions or interfaces. + .. versionchanged:: 3.3 + In previous versions, this was a 16-bit type or a 32-bit type depending on + whether you selected a "narrow" or "wide" Unicode version of Python at + build time. -.. c:type:: PyUnicodeObject +.. c:type:: PyASCIIObject + PyCompactUnicodeObject + PyUnicodeObject - This subtype of :c:type:`PyObject` represents a Python Unicode object. + These subtypes of :c:type:`PyObject` represent a Python Unicode object. In + almost all cases, they shouldn't be used directly, since all API functions + that deal with Unicode objects take and return :c:type:`PyObject` pointers. + + .. versionadded:: 3.3 .. c:var:: PyTypeObject PyUnicode_Type @@ -45,10 +79,10 @@ this in mind when writing extensions or interfaces. This instance of :c:type:`PyTypeObject` represents the Python Unicode type. It is exposed to Python code as ``str``. + The following APIs are really C macros and can be used to do fast checks and to access internal read-only data of Unicode objects: - .. c:function:: int PyUnicode_Check(PyObject *o) Return true if the object *o* is a Unicode object or an instance of a Unicode @@ -61,28 +95,106 @@ access internal read-only data of Unicode objects: subtype. -.. c:function:: Py_ssize_t PyUnicode_GET_SIZE(PyObject *o) +.. c:function:: int PyUnicode_READY(PyObject *o) - Return the size of the object. *o* has to be a :c:type:`PyUnicodeObject` (not - checked). + Ensure the string object *o* is in the "canonical" representation. This is + required before using any of the access macros described below. + .. XXX expand on when it is not required -.. c:function:: Py_ssize_t PyUnicode_GET_DATA_SIZE(PyObject *o) + Returns 0 on success and -1 with an exception set on failure, which in + particular happens if memory allocation fails. - Return the size of the object's internal buffer in bytes. *o* has to be a - :c:type:`PyUnicodeObject` (not checked). + .. versionadded:: 3.3 -.. c:function:: Py_UNICODE* PyUnicode_AS_UNICODE(PyObject *o) +.. c:function:: Py_ssize_t PyUnicode_GET_LENGTH(PyObject *o) + + Return the length of the Unicode string, in code points. *o* has to be a + Unicode object in the "canonical" representation (not checked). + + .. versionadded:: 3.3 + + +.. c:function:: Py_UCS1* PyUnicode_1BYTE_DATA(PyObject *o) + Py_UCS2* PyUnicode_2BYTE_DATA(PyObject *o) + Py_UCS4* PyUnicode_4BYTE_DATA(PyObject *o) + + Return a pointer to the canonical representation cast to UCS1, UCS2 or UCS4 + integer types for direct character access. No checks are performed if the + canonical representation has the correct character size; use + :c:func:`PyUnicode_KIND` to select the right macro. Make sure + :c:func:`PyUnicode_READY` has been called before accessing this. + + .. versionadded:: 3.3 + + +.. c:macro:: PyUnicode_WCHAR_KIND + PyUnicode_1BYTE_KIND + PyUnicode_2BYTE_KIND + PyUnicode_4BYTE_KIND + + Return values of the :c:func:`PyUnicode_KIND` macro. + + .. versionadded:: 3.3 + + +.. c:function:: int PyUnicode_KIND(PyObject *o) + + Return one of the PyUnicode kind constants (see above) that indicate how many + bytes per character this Unicode object uses to store its data. *o* has to + be a Unicode object in the "canonical" representation (not checked). + + .. XXX document "0" return value? + + .. versionadded:: 3.3 + + +.. c:function:: void* PyUnicode_DATA(PyObject *o) + + Return a void pointer to the raw unicode buffer. *o* has to be a Unicode + object in the "canonical" representation (not checked). + + .. versionadded:: 3.3 + + +.. c:function:: void PyUnicode_WRITE(int kind, void *data, Py_ssize_t index, \ + Py_UCS4 value) + + Write into a canonical representation *data* (as obtained with + :c:func:`PyUnicode_DATA`). This macro does not do any sanity checks and is + intended for usage in loops. The caller should cache the *kind* value and + *data* pointer as obtained from other macro calls. *index* is the index in + the string (starts at 0) and *value* is the new code point value which should + be written to that location. + + .. versionadded:: 3.3 + + +.. c:function:: Py_UCS4 PyUnicode_READ(int kind, void *data, Py_ssize_t index) + + Read a code point from a canonical representation *data* (as obtained with + :c:func:`PyUnicode_DATA`). No checks or ready calls are performed. + + .. versionadded:: 3.3 + + +.. c:function:: Py_UCS4 PyUnicode_READ_CHAR(PyObject *o, Py_ssize_t index) + + Read a character from a Unicode object *o*, which must be in the "canonical" + representation. This is less efficient than :c:func:`PyUnicode_READ` if you + do multiple consecutive reads. + + .. versionadded:: 3.3 - Return a pointer to the internal :c:type:`Py_UNICODE` buffer of the object. *o* - has to be a :c:type:`PyUnicodeObject` (not checked). +.. c:function:: PyUnicode_MAX_CHAR_VALUE(PyObject *o) -.. c:function:: const char* PyUnicode_AS_DATA(PyObject *o) + Return the maximum code point that is suitable for creating another string + based on *o*, which must be in the "canonical" representation. This is + always an approximation but more efficient than iterating over the string. - Return a pointer to the internal buffer of the object. *o* has to be a - :c:type:`PyUnicodeObject` (not checked). + .. versionadded:: 3.3 .. c:function:: int PyUnicode_ClearFreeList() @@ -90,6 +202,46 @@ access internal read-only data of Unicode objects: Clear the free list. Return the total number of freed items. +.. c:function:: Py_ssize_t PyUnicode_GET_SIZE(PyObject *o) + + Return the size of the deprecated :c:type:`Py_UNICODE` representation, in + code units (this includes surrogate pairs as 2 units). *o* has to be a + Unicode object (not checked). + + .. deprecated-removed:: 3.3 4.0 + Part of the old-style Unicode API, please migrate to using + :c:func:`PyUnicode_GET_LENGTH`. + + +.. c:function:: Py_ssize_t PyUnicode_GET_DATA_SIZE(PyObject *o) + + Return the size of the deprecated :c:type:`Py_UNICODE` representation in + bytes. *o* has to be a Unicode object (not checked). + + .. deprecated-removed:: 3.3 4.0 + Part of the old-style Unicode API, please migrate to using + :c:func:`PyUnicode_GET_LENGTH`. + + +.. c:function:: Py_UNICODE* PyUnicode_AS_UNICODE(PyObject *o) + const char* PyUnicode_AS_DATA(PyObject *o) + + Return a pointer to a :c:type:`Py_UNICODE` representation of the object. The + ``AS_DATA`` form casts the pointer to :c:type:`const char *`. *o* has to be + a Unicode object (not checked). + + .. versionchanged:: 3.3 + This macro is now inefficient -- because in many cases the + :c:type:`Py_UNICODE` representation does not exist and needs to be created + -- and can fail (return *NULL* with an exception set). Try to port the + code to use the new :c:func:`PyUnicode_nBYTE_DATA` macros or use + :c:func:`PyUnicode_WRITE` or :c:func:`PyUnicode_READ`. + + .. deprecated-removed:: 3.3 4.0 + Part of the old-style Unicode API, please migrate to using the + :c:func:`PyUnicode_nBYTE_DATA` family of macros. + + Unicode Character Properties """""""""""""""""""""""""""" @@ -166,16 +318,25 @@ These APIs can be used for fast direct character conversions: Return the character *ch* converted to lower case. + .. deprecated:: 3.3 + This function uses simple case mappings. + .. c:function:: Py_UNICODE Py_UNICODE_TOUPPER(Py_UNICODE ch) Return the character *ch* converted to upper case. + .. deprecated:: 3.3 + This function uses simple case mappings. + .. c:function:: Py_UNICODE Py_UNICODE_TOTITLE(Py_UNICODE ch) Return the character *ch* converted to title case. + .. deprecated:: 3.3 + This function uses simple case mappings. + .. c:function:: int Py_UNICODE_TODECIMAL(Py_UNICODE ch) @@ -195,31 +356,66 @@ These APIs can be used for fast direct character conversions: possible. This macro does not raise exceptions. -Plain Py_UNICODE -"""""""""""""""" +These APIs can be used to work with surrogates: + +.. c:macro:: Py_UNICODE_IS_SURROGATE(ch) + + Check if *ch* is a surrogate (``0xD800 <= ch <= 0xDFFF``). + +.. c:macro:: Py_UNICODE_IS_HIGH_SURROGATE(ch) + + Check if *ch* is an high surrogate (``0xD800 <= ch <= 0xDBFF``). + +.. c:macro:: Py_UNICODE_IS_LOW_SURROGATE(ch) + + Check if *ch* is a low surrogate (``0xDC00 <= ch <= 0xDFFF``). + +.. c:macro:: Py_UNICODE_JOIN_SURROGATES(high, low) + + Join two surrogate characters and return a single Py_UCS4 value. + *high* and *low* are respectively the leading and trailing surrogates in a + surrogate pair. + + +Creating and accessing Unicode strings +"""""""""""""""""""""""""""""""""""""" To create Unicode objects and access their basic sequence properties, use these APIs: +.. c:function:: PyObject* PyUnicode_New(Py_ssize_t size, Py_UCS4 maxchar) -.. c:function:: PyObject* PyUnicode_FromUnicode(const Py_UNICODE *u, Py_ssize_t size) + Create a new Unicode object. *maxchar* should be the true maximum code point + to be placed in the string. As an approximation, it can be rounded up to the + nearest value in the sequence 127, 255, 65535, 1114111. - Create a Unicode object from the Py_UNICODE buffer *u* of the given size. *u* - may be *NULL* which causes the contents to be undefined. It is the user's - responsibility to fill in the needed data. The buffer is copied into the new - object. If the buffer is not *NULL*, the return value might be a shared object. - Therefore, modification of the resulting Unicode object is only allowed when *u* - is *NULL*. + This is the recommended way to allocate a new Unicode object. Objects + created using this function are not resizable. + + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyUnicode_FromKindAndData(int kind, const void *buffer, \ + Py_ssize_t size) + + Create a new Unicode object with the given *kind* (possible values are + :c:macro:`PyUnicode_1BYTE_KIND` etc., as returned by + :c:func:`PyUnicode_KIND`). The *buffer* must point to an array of *size* + units of 1, 2 or 4 bytes per character, as given by the kind. + + .. versionadded:: 3.3 .. c:function:: PyObject* PyUnicode_FromStringAndSize(const char *u, Py_ssize_t size) - Create a Unicode object from the char buffer *u*. The bytes will be interpreted - as being UTF-8 encoded. *u* may also be *NULL* which - causes the contents to be undefined. It is the user's responsibility to fill in - the needed data. The buffer is copied into the new object. If the buffer is not - *NULL*, the return value might be a shared object. Therefore, modification of - the resulting Unicode object is only allowed when *u* is *NULL*. + Create a Unicode object from the char buffer *u*. The bytes will be + interpreted as being UTF-8 encoded. The buffer is copied into the new + object. If the buffer is not *NULL*, the return value might be a shared + object, i.e. modification of the data is not allowed. + + If *u* is *NULL*, this function behaves like :c:func:`PyUnicode_FromUnicode` + with the buffer set to *NULL*. This usage is deprecated in favor of + :c:func:`PyUnicode_New`. .. c:function:: PyObject *PyUnicode_FromString(const char *u) @@ -243,6 +439,8 @@ APIs: .. % Similar comments apply to the %ll width modifier and .. % PY_FORMAT_LONG_LONG. + .. tabularcolumns:: |l|l|L| + +-------------------+---------------------+--------------------------------+ | Format Characters | Type | Comment | +===================+=====================+================================+ @@ -260,18 +458,27 @@ APIs: | :attr:`%ld` | long | Exactly equivalent to | | | | ``printf("%ld")``. | +-------------------+---------------------+--------------------------------+ + | :attr:`%li` | long | Exactly equivalent to | + | | | ``printf("%li")``. | + +-------------------+---------------------+--------------------------------+ | :attr:`%lu` | unsigned long | Exactly equivalent to | | | | ``printf("%lu")``. | +-------------------+---------------------+--------------------------------+ | :attr:`%lld` | long long | Exactly equivalent to | | | | ``printf("%lld")``. | +-------------------+---------------------+--------------------------------+ + | :attr:`%lli` | long long | Exactly equivalent to | + | | | ``printf("%lli")``. | + +-------------------+---------------------+--------------------------------+ | :attr:`%llu` | unsigned long long | Exactly equivalent to | | | | ``printf("%llu")``. | +-------------------+---------------------+--------------------------------+ | :attr:`%zd` | Py_ssize_t | Exactly equivalent to | | | | ``printf("%zd")``. | +-------------------+---------------------+--------------------------------+ + | :attr:`%zi` | Py_ssize_t | Exactly equivalent to | + | | | ``printf("%zi")``. | + +-------------------+---------------------+--------------------------------+ | :attr:`%zu` | size_t | Exactly equivalent to | | | | ``printf("%zu")``. | +-------------------+---------------------+--------------------------------+ @@ -322,27 +529,178 @@ APIs: .. versionchanged:: 3.2 Support for ``"%lld"`` and ``"%llu"`` added. + .. versionchanged:: 3.3 + Support for ``"%li"``, ``"%lli"`` and ``"%zi"`` added. + .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs) Identical to :c:func:`PyUnicode_FromFormat` except that it takes exactly two arguments. + +.. c:function:: PyObject* PyUnicode_FromEncodedObject(PyObject *obj, \ + const char *encoding, const char *errors) + + Coerce an encoded object *obj* to an Unicode object and return a reference with + incremented refcount. + + :class:`bytes`, :class:`bytearray` and other char buffer compatible objects + are decoded according to the given *encoding* and using the error handling + defined by *errors*. Both can be *NULL* to have the interface use the default + values (see the next section for details). + + All other objects, including Unicode objects, cause a :exc:`TypeError` to be + set. + + The API returns *NULL* if there was an error. The caller is responsible for + decref'ing the returned objects. + + +.. c:function:: Py_ssize_t PyUnicode_GetLength(PyObject *unicode) + + Return the length of the Unicode object, in code points. + + .. versionadded:: 3.3 + + +.. c:function:: int PyUnicode_CopyCharacters(PyObject *to, Py_ssize_t to_start, \ + PyObject *from, Py_ssize_t from_start, Py_ssize_t how_many) + + Copy characters from one Unicode object into another. This function performs + character conversion when necessary and falls back to :c:func:`memcpy` if + possible. Returns ``-1`` and sets an exception on error, otherwise returns + ``0``. + + .. versionadded:: 3.3 + + +.. c:function:: Py_ssize_t PyUnicode_Fill(PyObject *unicode, Py_ssize_t start, \ + Py_ssize_t length, Py_UCS4 fill_char) + + Fill a string with a character: write *fill_char* into + ``unicode[start:start+length]``. + + Fail if *fill_char* is bigger than the string maximum character, or if the + string has more than 1 reference. + + Return the number of written character, or return ``-1`` and raise an + exception on error. + + .. versionadded:: 3.3 + + +.. c:function:: int PyUnicode_WriteChar(PyObject *unicode, Py_ssize_t index, \ + Py_UCS4 character) + + Write a character to a string. The string must have been created through + :c:func:`PyUnicode_New`. Since Unicode strings are supposed to be immutable, + the string must not be shared, or have been hashed yet. + + This function checks that *unicode* is a Unicode object, that the index is + not out of bounds, and that the object can be modified safely (i.e. that it + its reference count is one), in contrast to the macro version + :c:func:`PyUnicode_WRITE_CHAR`. + + .. versionadded:: 3.3 + + +.. c:function:: Py_UCS4 PyUnicode_ReadChar(PyObject *unicode, Py_ssize_t index) + + Read a character from a string. This function checks that *unicode* is a + Unicode object and the index is not out of bounds, in contrast to the macro + version :c:func:`PyUnicode_READ_CHAR`. + + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyUnicode_Substring(PyObject *str, Py_ssize_t start, \ + Py_ssize_t end) + + Return a substring of *str*, from character index *start* (included) to + character index *end* (excluded). Negative indices are not supported. + + .. versionadded:: 3.3 + + +.. c:function:: Py_UCS4* PyUnicode_AsUCS4(PyObject *u, Py_UCS4 *buffer, \ + Py_ssize_t buflen, int copy_null) + + Copy the string *u* into a UCS4 buffer, including a null character, if + *copy_null* is set. Returns *NULL* and sets an exception on error (in + particular, a :exc:`ValueError` if *buflen* is smaller than the length of + *u*). *buffer* is returned on success. + + .. versionadded:: 3.3 + + +.. c:function:: Py_UCS4* PyUnicode_AsUCS4Copy(PyObject *u) + + Copy the string *u* into a new UCS4 buffer that is allocated using + :c:func:`PyMem_Malloc`. If this fails, *NULL* is returned with a + :exc:`MemoryError` set. + + .. versionadded:: 3.3 + + +Deprecated Py_UNICODE APIs +"""""""""""""""""""""""""" + +.. deprecated-removed:: 3.3 4.0 + +These API functions are deprecated with the implementation of :pep:`393`. +Extension modules can continue using them, as they will not be removed in Python +3.x, but need to be aware that their use can now cause performance and memory hits. + + +.. c:function:: PyObject* PyUnicode_FromUnicode(const Py_UNICODE *u, Py_ssize_t size) + + Create a Unicode object from the Py_UNICODE buffer *u* of the given size. *u* + may be *NULL* which causes the contents to be undefined. It is the user's + responsibility to fill in the needed data. The buffer is copied into the new + object. + + If the buffer is not *NULL*, the return value might be a shared object. + Therefore, modification of the resulting Unicode object is only allowed when + *u* is *NULL*. + + If the buffer is *NULL*, :c:func:`PyUnicode_READY` must be called once the + string content has been filled before using any of the access macros such as + :c:func:`PyUnicode_KIND`. + + Please migrate to using :c:func:`PyUnicode_FromKindAndData` or + :c:func:`PyUnicode_New`. + + +.. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode) + + Return a read-only pointer to the Unicode object's internal + :c:type:`Py_UNICODE` buffer, or *NULL* on error. This will create the + :c:type:`Py_UNICODE*` representation of the object if it is not yet + available. Note that the resulting :c:type:`Py_UNICODE` string may contain + embedded null characters, which would cause the string to be truncated when + used in most C functions. + + Please migrate to using :c:func:`PyUnicode_AsUCS4`, + :c:func:`PyUnicode_Substring`, :c:func:`PyUnicode_ReadChar` or similar new + APIs. + + .. c:function:: PyObject* PyUnicode_TransformDecimalToASCII(Py_UNICODE *s, Py_ssize_t size) Create a Unicode object by replacing all decimal digits in :c:type:`Py_UNICODE` buffer of the given *size* by ASCII digits 0--9 - according to their decimal value. Return *NULL* if an exception - occurs. + according to their decimal value. Return *NULL* if an exception occurs. -.. c:function:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode) +.. c:function:: Py_UNICODE* PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size) - Return a read-only pointer to the Unicode object's internal - :c:type:`Py_UNICODE` buffer, *NULL* if *unicode* is not a Unicode object. - Note that the resulting :c:type:`Py_UNICODE*` string may contain embedded - null characters, which would cause the string to be truncated when used in - most C functions. + Like :c:func:`PyUnicode_AsUnicode`, but also saves the :c:func:`Py_UNICODE` + array length in *size*. Note that the resulting :c:type:`Py_UNICODE*` string + may contain embedded null characters, which would cause the string to be + truncated when used in most C functions. + + .. versionadded:: 3.3 .. c:function:: Py_UNICODE* PyUnicode_AsUnicodeCopy(PyObject *unicode) @@ -350,44 +708,77 @@ APIs: Create a copy of a Unicode string ending with a nul character. Return *NULL* and raise a :exc:`MemoryError` exception on memory allocation failure, otherwise return a new allocated buffer (use :c:func:`PyMem_Free` to free - the buffer). Note that the resulting :c:type:`Py_UNICODE*` string may contain - embedded null characters, which would cause the string to be truncated when - used in most C functions. + the buffer). Note that the resulting :c:type:`Py_UNICODE*` string may + contain embedded null characters, which would cause the string to be + truncated when used in most C functions. .. versionadded:: 3.2 + Please migrate to using :c:func:`PyUnicode_AsUCS4Copy` or similar new APIs. + .. c:function:: Py_ssize_t PyUnicode_GetSize(PyObject *unicode) - Return the length of the Unicode object. + Return the size of the deprecated :c:type:`Py_UNICODE` representation, in + code units (this includes surrogate pairs as 2 units). + Please migrate to using :c:func:`PyUnicode_GetLength`. -.. c:function:: PyObject* PyUnicode_FromEncodedObject(PyObject *obj, const char *encoding, const char *errors) - Coerce an encoded object *obj* to an Unicode object and return a reference with - incremented refcount. +.. c:function:: PyObject* PyUnicode_FromObject(PyObject *obj) - :class:`bytes`, :class:`bytearray` and other char buffer compatible objects - are decoded according to the given *encoding* and using the error handling - defined by *errors*. Both can be *NULL* to have the interface use the default - values (see the next section for details). + Shortcut for ``PyUnicode_FromEncodedObject(obj, NULL, "strict")`` which is used + throughout the interpreter whenever coercion to Unicode is needed. - All other objects, including Unicode objects, cause a :exc:`TypeError` to be - set. - The API returns *NULL* if there was an error. The caller is responsible for - decref'ing the returned objects. +Locale Encoding +""""""""""""""" +The current locale encoding can be used to decode text from the operating +system. -.. c:function:: PyObject* PyUnicode_FromObject(PyObject *obj) +.. c:function:: PyObject* PyUnicode_DecodeLocaleAndSize(const char *str, \ + Py_ssize_t len, \ + const char *errors) - Shortcut for ``PyUnicode_FromEncodedObject(obj, NULL, "strict")`` which is used - throughout the interpreter whenever coercion to Unicode is needed. + Decode a string from the current locale encoding. The supported + error handlers are ``"strict"`` and ``"surrogateescape"`` + (:pep:`383`). The decoder uses ``"strict"`` error handler if + *errors* is ``NULL``. *str* must end with a null character but + cannot contain embedded null characters. + + .. seealso:: + + Use :c:func:`PyUnicode_DecodeFSDefaultAndSize` to decode a string from + :c:data:`Py_FileSystemDefaultEncoding` (the locale encoding read at + Python startup). + + .. versionadded:: 3.3 + + +.. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors) + + Similar to :c:func:`PyUnicode_DecodeLocaleAndSize`, but compute the string + length using :c:func:`strlen`. + + .. versionadded:: 3.3 -If the platform supports :c:type:`wchar_t` and provides a header file wchar.h, -Python can interface directly to this type using the following functions. -Support is optimized if Python's own :c:type:`Py_UNICODE` type is identical to -the system's :c:type:`wchar_t`. + +.. c:function:: PyObject* PyUnicode_EncodeLocale(PyObject *unicode, const char *errors) + + Encode a Unicode object to the current locale encoding. The + supported error handlers are ``"strict"`` and ``"surrogateescape"`` + (:pep:`383`). The encoder uses ``"strict"`` error handler if + *errors* is ``NULL``. Return a :class:`bytes` object. *str* cannot + contain embedded null characters. + + .. seealso:: + + Use :c:func:`PyUnicode_EncodeFSDefault` to encode a string to + :c:data:`Py_FileSystemDefaultEncoding` (the locale encoding read at + Python startup). + + .. versionadded:: 3.3 File System Encoding @@ -425,19 +816,26 @@ used, passing :c:func:`PyUnicode_FSDecoder` as the conversion function: .. c:function:: PyObject* PyUnicode_DecodeFSDefaultAndSize(const char *s, Py_ssize_t size) Decode a string using :c:data:`Py_FileSystemDefaultEncoding` and the - ``'surrogateescape'`` error handler, or ``'strict'`` on Windows. + ``"surrogateescape"`` error handler, or ``"strict"`` on Windows. If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the locale encoding. + .. seealso:: + + :c:data:`Py_FileSystemDefaultEncoding` is initialized at startup from the + locale encoding and cannot be modified later. If you need to decode a + string from the current locale encoding, use + :c:func:`PyUnicode_DecodeLocaleAndSize`. + .. versionchanged:: 3.2 - Use ``'strict'`` error handler on Windows. + Use ``"strict"`` error handler on Windows. .. c:function:: PyObject* PyUnicode_DecodeFSDefault(const char *s) Decode a null-terminated string using :c:data:`Py_FileSystemDefaultEncoding` - and the ``'surrogateescape'`` error handler, or ``'strict'`` on Windows. + and the ``"surrogateescape"`` error handler, or ``"strict"`` on Windows. If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the locale encoding. @@ -445,19 +843,26 @@ used, passing :c:func:`PyUnicode_FSDecoder` as the conversion function: Use :c:func:`PyUnicode_DecodeFSDefaultAndSize` if you know the string length. .. versionchanged:: 3.2 - Use ``'strict'`` error handler on Windows. + Use ``"strict"`` error handler on Windows. .. c:function:: PyObject* PyUnicode_EncodeFSDefault(PyObject *unicode) Encode a Unicode object to :c:data:`Py_FileSystemDefaultEncoding` with the - ``'surrogateescape'`` error handler, or ``'strict'`` on Windows, and return + ``"surrogateescape"`` error handler, or ``"strict"`` on Windows, and return :class:`bytes`. Note that the resulting :class:`bytes` object may contain null bytes. If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the locale encoding. + .. seealso:: + + :c:data:`Py_FileSystemDefaultEncoding` is initialized at startup from the + locale encoding and cannot be modified later. If you need to encode a + string to the current locale encoding, use + :c:func:`PyUnicode_EncodeLocale`. + .. versionadded:: 3.2 @@ -479,9 +884,9 @@ wchar_t Support Copy the Unicode object contents into the :c:type:`wchar_t` buffer *w*. At most *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing 0-termination character). Return the number of :c:type:`wchar_t` characters - copied or -1 in case of an error. Note that the resulting :c:type:`wchar_t` + copied or -1 in case of an error. Note that the resulting :c:type:`wchar_t*` string may or may not be 0-terminated. It is the responsibility of the caller - to make sure that the :c:type:`wchar_t` string is 0-terminated in case this is + to make sure that the :c:type:`wchar_t*` string is 0-terminated in case this is required by the application. Also, note that the :c:type:`wchar_t*` string might contain null characters, which would cause the string to be truncated when used with most C functions. @@ -497,12 +902,32 @@ wchar_t Support Returns a buffer allocated by :c:func:`PyMem_Alloc` (use :c:func:`PyMem_Free` to free it) on success. On error, returns *NULL*, *\*size* is undefined and raises a :exc:`MemoryError`. Note that the - resulting :c:type:`wchar_t*` string might contain null characters, which + resulting :c:type:`wchar_t` string might contain null characters, which would cause the string to be truncated when used with most C functions. .. versionadded:: 3.2 +UCS4 Support +"""""""""""" + +.. versionadded:: 3.3 + +.. XXX are these meant to be public? + +.. c:function:: size_t Py_UCS4_strlen(const Py_UCS4 *u) + Py_UCS4* Py_UCS4_strcpy(Py_UCS4 *s1, const Py_UCS4 *s2) + Py_UCS4* Py_UCS4_strncpy(Py_UCS4 *s1, const Py_UCS4 *s2, size_t n) + Py_UCS4* Py_UCS4_strcat(Py_UCS4 *s1, const Py_UCS4 *s2) + int Py_UCS4_strcmp(const Py_UCS4 *s1, const Py_UCS4 *s2) + int Py_UCS4_strncmp(const Py_UCS4 *s1, const Py_UCS4 *s2, size_t n) + Py_UCS4* Py_UCS4_strchr(const Py_UCS4 *s, Py_UCS4 c) + Py_UCS4* Py_UCS4_strrchr(const Py_UCS4 *s, Py_UCS4 c) + + These utility functions work on strings of :c:type:`Py_UCS4` characters and + otherwise behave like the C standard library functions with the same name. + + .. _builtincodecs: Built-in Codecs @@ -537,31 +962,38 @@ Generic Codecs These are the generic codec APIs: -.. c:function:: PyObject* PyUnicode_Decode(const char *s, Py_ssize_t size, const char *encoding, const char *errors) +.. c:function:: PyObject* PyUnicode_Decode(const char *s, Py_ssize_t size, \ + const char *encoding, const char *errors) Create a Unicode object by decoding *size* bytes of the encoded string *s*. *encoding* and *errors* have the same meaning as the parameters of the same name - in the :func:`unicode` built-in function. The codec to be used is looked up + in the :func:`str` built-in function. The codec to be used is looked up using the Python codec registry. Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_Encode(const Py_UNICODE *s, Py_ssize_t size, const char *encoding, const char *errors) +.. c:function:: PyObject* PyUnicode_AsEncodedString(PyObject *unicode, \ + const char *encoding, const char *errors) + + Encode a Unicode object and return the result as Python bytes object. + *encoding* and *errors* have the same meaning as the parameters of the same + name in the Unicode :meth:`~str.encode` method. The codec to be used is looked up + using the Python codec registry. Return *NULL* if an exception was raised by + the codec. + + +.. c:function:: PyObject* PyUnicode_Encode(const Py_UNICODE *s, Py_ssize_t size, \ + const char *encoding, const char *errors) Encode the :c:type:`Py_UNICODE` buffer *s* of the given *size* and return a Python bytes object. *encoding* and *errors* have the same meaning as the - parameters of the same name in the Unicode :meth:`encode` method. The codec + parameters of the same name in the Unicode :meth:`~str.encode` method. The codec to be used is looked up using the Python codec registry. Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsEncodedString(PyObject *unicode, const char *encoding, const char *errors) - - Encode a Unicode object and return the result as Python bytes object. - *encoding* and *errors* have the same meaning as the parameters of the same - name in the Unicode :meth:`encode` method. The codec to be used is looked up - using the Python codec registry. Return *NULL* if an exception was raised by - the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsEncodedString`. UTF-8 Codecs @@ -576,7 +1008,8 @@ These are the UTF-8 codec APIs: *s*. Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_DecodeUTF8Stateful(const char *s, Py_ssize_t size, const char *errors, Py_ssize_t *consumed) +.. c:function:: PyObject* PyUnicode_DecodeUTF8Stateful(const char *s, Py_ssize_t size, \ + const char *errors, Py_ssize_t *consumed) If *consumed* is *NULL*, behave like :c:func:`PyUnicode_DecodeUTF8`. If *consumed* is not *NULL*, trailing incomplete UTF-8 byte sequences will not be @@ -584,18 +1017,45 @@ These are the UTF-8 codec APIs: that have been decoded will be stored in *consumed*. +.. c:function:: PyObject* PyUnicode_AsUTF8String(PyObject *unicode) + + Encode a Unicode object using UTF-8 and return the result as Python bytes + object. Error handling is "strict". Return *NULL* if an exception was + raised by the codec. + + +.. c:function:: char* PyUnicode_AsUTF8AndSize(PyObject *unicode, Py_ssize_t *size) + + Return a pointer to the default encoding (UTF-8) of the Unicode object, and + store the size of the encoded representation (in bytes) in *size*. *size* + can be *NULL*, in this case no size will be stored. + + In the case of an error, *NULL* is returned with an exception set and no + *size* is stored. + + This caches the UTF-8 representation of the string in the Unicode object, and + subsequent calls will return a pointer to the same buffer. The caller is not + responsible for deallocating the buffer. + + .. versionadded:: 3.3 + + +.. c:function:: char* PyUnicode_AsUTF8(PyObject *unicode) + + As :c:func:`PyUnicode_AsUTF8AndSize`, but does not store the size. + + .. versionadded:: 3.3 + + .. c:function:: PyObject* PyUnicode_EncodeUTF8(const Py_UNICODE *s, Py_ssize_t size, const char *errors) Encode the :c:type:`Py_UNICODE` buffer *s* of the given *size* using UTF-8 and return a Python bytes object. Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsUTF8String(PyObject *unicode) - - Encode a Unicode object using UTF-8 and return the result as Python bytes - object. Error handling is "strict". Return *NULL* if an exception was - raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsUTF8String` or :c:func:`PyUnicode_AsUTF8AndSize`. UTF-32 Codecs @@ -604,7 +1064,8 @@ UTF-32 Codecs These are the UTF-32 codec APIs: -.. c:function:: PyObject* PyUnicode_DecodeUTF32(const char *s, Py_ssize_t size, const char *errors, int *byteorder) +.. c:function:: PyObject* PyUnicode_DecodeUTF32(const char *s, Py_ssize_t size, \ + const char *errors, int *byteorder) Decode *size* bytes from a UTF-32 encoded buffer string and return the corresponding Unicode object. *errors* (if non-*NULL*) defines the error @@ -625,14 +1086,13 @@ These are the UTF-32 codec APIs: After completion, *\*byteorder* is set to the current byte order at the end of input data. - In a narrow build codepoints outside the BMP will be decoded as surrogate pairs. - If *byteorder* is *NULL*, the codec starts in native order mode. Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_DecodeUTF32Stateful(const char *s, Py_ssize_t size, const char *errors, int *byteorder, Py_ssize_t *consumed) +.. c:function:: PyObject* PyUnicode_DecodeUTF32Stateful(const char *s, Py_ssize_t size, \ + const char *errors, int *byteorder, Py_ssize_t *consumed) If *consumed* is *NULL*, behave like :c:func:`PyUnicode_DecodeUTF32`. If *consumed* is not *NULL*, :c:func:`PyUnicode_DecodeUTF32Stateful` will not treat @@ -641,7 +1101,15 @@ These are the UTF-32 codec APIs: that have been decoded will be stored in *consumed*. -.. c:function:: PyObject* PyUnicode_EncodeUTF32(const Py_UNICODE *s, Py_ssize_t size, const char *errors, int byteorder) +.. c:function:: PyObject* PyUnicode_AsUTF32String(PyObject *unicode) + + Return a Python byte string using the UTF-32 encoding in native byte + order. The string always starts with a BOM mark. Error handling is "strict". + Return *NULL* if an exception was raised by the codec. + + +.. c:function:: PyObject* PyUnicode_EncodeUTF32(const Py_UNICODE *s, Py_ssize_t size, \ + const char *errors, int byteorder) Return a Python bytes object holding the UTF-32 encoded value of the Unicode data in *s*. Output is written according to the following byte order:: @@ -658,12 +1126,9 @@ These are the UTF-32 codec APIs: Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsUTF32String(PyObject *unicode) - - Return a Python byte string using the UTF-32 encoding in native byte - order. The string always starts with a BOM mark. Error handling is "strict". - Return *NULL* if an exception was raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsUTF32String`. UTF-16 Codecs @@ -672,7 +1137,8 @@ UTF-16 Codecs These are the UTF-16 codec APIs: -.. c:function:: PyObject* PyUnicode_DecodeUTF16(const char *s, Py_ssize_t size, const char *errors, int *byteorder) +.. c:function:: PyObject* PyUnicode_DecodeUTF16(const char *s, Py_ssize_t size, \ + const char *errors, int *byteorder) Decode *size* bytes from a UTF-16 encoded buffer string and return the corresponding Unicode object. *errors* (if non-*NULL*) defines the error @@ -699,7 +1165,8 @@ These are the UTF-16 codec APIs: Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_DecodeUTF16Stateful(const char *s, Py_ssize_t size, const char *errors, int *byteorder, Py_ssize_t *consumed) +.. c:function:: PyObject* PyUnicode_DecodeUTF16Stateful(const char *s, Py_ssize_t size, \ + const char *errors, int *byteorder, Py_ssize_t *consumed) If *consumed* is *NULL*, behave like :c:func:`PyUnicode_DecodeUTF16`. If *consumed* is not *NULL*, :c:func:`PyUnicode_DecodeUTF16Stateful` will not treat @@ -708,7 +1175,15 @@ These are the UTF-16 codec APIs: number of bytes that have been decoded will be stored in *consumed*. -.. c:function:: PyObject* PyUnicode_EncodeUTF16(const Py_UNICODE *s, Py_ssize_t size, const char *errors, int byteorder) +.. c:function:: PyObject* PyUnicode_AsUTF16String(PyObject *unicode) + + Return a Python byte string using the UTF-16 encoding in native byte + order. The string always starts with a BOM mark. Error handling is "strict". + Return *NULL* if an exception was raised by the codec. + + +.. c:function:: PyObject* PyUnicode_EncodeUTF16(const Py_UNICODE *s, Py_ssize_t size, \ + const char *errors, int byteorder) Return a Python bytes object holding the UTF-16 encoded value of the Unicode data in *s*. Output is written according to the following byte order:: @@ -726,12 +1201,9 @@ These are the UTF-16 codec APIs: Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsUTF16String(PyObject *unicode) - - Return a Python byte string using the UTF-16 encoding in native byte - order. The string always starts with a BOM mark. Error handling is "strict". - Return *NULL* if an exception was raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsUTF16String`. UTF-7 Codecs @@ -746,7 +1218,8 @@ These are the UTF-7 codec APIs: *s*. Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_DecodeUTF7Stateful(const char *s, Py_ssize_t size, const char *errors, Py_ssize_t *consumed) +.. c:function:: PyObject* PyUnicode_DecodeUTF7Stateful(const char *s, Py_ssize_t size, \ + const char *errors, Py_ssize_t *consumed) If *consumed* is *NULL*, behave like :c:func:`PyUnicode_DecodeUTF7`. If *consumed* is not *NULL*, trailing incomplete UTF-7 base-64 sections will not @@ -754,7 +1227,8 @@ These are the UTF-7 codec APIs: bytes that have been decoded will be stored in *consumed*. -.. c:function:: PyObject* PyUnicode_EncodeUTF7(const Py_UNICODE *s, Py_ssize_t size, int base64SetO, int base64WhiteSpace, const char *errors) +.. c:function:: PyObject* PyUnicode_EncodeUTF7(const Py_UNICODE *s, Py_ssize_t size, \ + int base64SetO, int base64WhiteSpace, const char *errors) Encode the :c:type:`Py_UNICODE` buffer of the given size using UTF-7 and return a Python bytes object. Return *NULL* if an exception was raised by @@ -765,6 +1239,11 @@ These are the UTF-7 codec APIs: nonzero, whitespace will be encoded in base-64. Both are set to zero for the Python "utf-7" codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API. + + .. XXX replace with what? + Unicode-Escape Codecs """"""""""""""""""""" @@ -772,24 +1251,29 @@ Unicode-Escape Codecs These are the "Unicode Escape" codec APIs: -.. c:function:: PyObject* PyUnicode_DecodeUnicodeEscape(const char *s, Py_ssize_t size, const char *errors) +.. c:function:: PyObject* PyUnicode_DecodeUnicodeEscape(const char *s, \ + Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the Unicode-Escape encoded string *s*. Return *NULL* if an exception was raised by the codec. +.. c:function:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode) + + Encode a Unicode object using Unicode-Escape and return the result as Python + string object. Error handling is "strict". Return *NULL* if an exception was + raised by the codec. + + .. c:function:: PyObject* PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size) Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Unicode-Escape and return a Python string object. Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode) - - Encode a Unicode object using Unicode-Escape and return the result as Python - string object. Error handling is "strict". Return *NULL* if an exception was - raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsUnicodeEscapeString`. Raw-Unicode-Escape Codecs @@ -798,19 +1282,13 @@ Raw-Unicode-Escape Codecs These are the "Raw Unicode Escape" codec APIs: -.. c:function:: PyObject* PyUnicode_DecodeRawUnicodeEscape(const char *s, Py_ssize_t size, const char *errors) +.. c:function:: PyObject* PyUnicode_DecodeRawUnicodeEscape(const char *s, \ + Py_ssize_t size, const char *errors) Create a Unicode object by decoding *size* bytes of the Raw-Unicode-Escape encoded string *s*. Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size, const char *errors) - - Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Raw-Unicode-Escape - and return a Python string object. Return *NULL* if an exception was raised by - the codec. - - .. c:function:: PyObject* PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode) Encode a Unicode object using Raw-Unicode-Escape and return the result as @@ -818,6 +1296,18 @@ These are the "Raw Unicode Escape" codec APIs: was raised by the codec. +.. c:function:: PyObject* PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s, \ + Py_ssize_t size, const char *errors) + + Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Raw-Unicode-Escape + and return a Python string object. Return *NULL* if an exception was raised by + the codec. + + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsRawUnicodeEscapeString`. + + Latin-1 Codecs """""""""""""" @@ -831,18 +1321,22 @@ ordinals and only these are accepted by the codecs during encoding. *s*. Return *NULL* if an exception was raised by the codec. +.. c:function:: PyObject* PyUnicode_AsLatin1String(PyObject *unicode) + + Encode a Unicode object using Latin-1 and return the result as Python bytes + object. Error handling is "strict". Return *NULL* if an exception was + raised by the codec. + + .. c:function:: PyObject* PyUnicode_EncodeLatin1(const Py_UNICODE *s, Py_ssize_t size, const char *errors) Encode the :c:type:`Py_UNICODE` buffer of the given *size* using Latin-1 and return a Python bytes object. Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsLatin1String(PyObject *unicode) - - Encode a Unicode object using Latin-1 and return the result as Python bytes - object. Error handling is "strict". Return *NULL* if an exception was - raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsLatin1String`. ASCII Codecs @@ -858,18 +1352,22 @@ codes generate errors. *s*. Return *NULL* if an exception was raised by the codec. +.. c:function:: PyObject* PyUnicode_AsASCIIString(PyObject *unicode) + + Encode a Unicode object using ASCII and return the result as Python bytes + object. Error handling is "strict". Return *NULL* if an exception was + raised by the codec. + + .. c:function:: PyObject* PyUnicode_EncodeASCII(const Py_UNICODE *s, Py_ssize_t size, const char *errors) Encode the :c:type:`Py_UNICODE` buffer of the given *size* using ASCII and return a Python bytes object. Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsASCIIString(PyObject *unicode) - - Encode a Unicode object using ASCII and return the result as Python bytes - object. Error handling is "strict". Return *NULL* if an exception was - raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsASCIIString`. Character Map Codecs @@ -898,7 +1396,8 @@ characters to different code points. These are the mapping codec APIs: -.. c:function:: PyObject* PyUnicode_DecodeCharmap(const char *s, Py_ssize_t size, PyObject *mapping, const char *errors) +.. c:function:: PyObject* PyUnicode_DecodeCharmap(const char *s, Py_ssize_t size, \ + PyObject *mapping, const char *errors) Create a Unicode object by decoding *size* bytes of the encoded string *s* using the given *mapping* object. Return *NULL* if an exception was raised by the @@ -908,13 +1407,6 @@ These are the mapping codec APIs: treated as "undefined mapping". -.. c:function:: PyObject* PyUnicode_EncodeCharmap(const Py_UNICODE *s, Py_ssize_t size, PyObject *mapping, const char *errors) - - Encode the :c:type:`Py_UNICODE` buffer of the given *size* using the given - *mapping* object and return a Python string object. Return *NULL* if an - exception was raised by the codec. - - .. c:function:: PyObject* PyUnicode_AsCharmapString(PyObject *unicode, PyObject *mapping) Encode a Unicode object using the given *mapping* object and return the result @@ -924,7 +1416,8 @@ These are the mapping codec APIs: The following codec API is special in that maps Unicode to Unicode. -.. c:function:: PyObject* PyUnicode_TranslateCharmap(const Py_UNICODE *s, Py_ssize_t size, PyObject *table, const char *errors) +.. c:function:: PyObject* PyUnicode_TranslateCharmap(const Py_UNICODE *s, Py_ssize_t size, \ + PyObject *table, const char *errors) Translate a :c:type:`Py_UNICODE` buffer of the given *size* by applying a character mapping *table* to it and return the resulting Unicode object. Return @@ -937,6 +1430,22 @@ The following codec API is special in that maps Unicode to Unicode. and sequences work well. Unmapped character ordinals (ones which cause a :exc:`LookupError`) are left untouched and are copied as-is. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API. + + .. XXX replace with what? + + +.. c:function:: PyObject* PyUnicode_EncodeCharmap(const Py_UNICODE *s, Py_ssize_t size, \ + PyObject *mapping, const char *errors) + + Encode the :c:type:`Py_UNICODE` buffer of the given *size* using the given + *mapping* object and return a Python string object. Return *NULL* if an + exception was raised by the codec. + + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsCharmapString`. MBCS codecs for Windows @@ -953,7 +1462,8 @@ the user settings on the machine running the codec. Return *NULL* if an exception was raised by the codec. -.. c:function:: PyObject* PyUnicode_DecodeMBCSStateful(const char *s, int size, const char *errors, int *consumed) +.. c:function:: PyObject* PyUnicode_DecodeMBCSStateful(const char *s, int size, \ + const char *errors, int *consumed) If *consumed* is *NULL*, behave like :c:func:`PyUnicode_DecodeMBCS`. If *consumed* is not *NULL*, :c:func:`PyUnicode_DecodeMBCSStateful` will not decode @@ -961,18 +1471,31 @@ the user settings on the machine running the codec. in *consumed*. +.. c:function:: PyObject* PyUnicode_AsMBCSString(PyObject *unicode) + + Encode a Unicode object using MBCS and return the result as Python bytes + object. Error handling is "strict". Return *NULL* if an exception was + raised by the codec. + + +.. c:function:: PyObject* PyUnicode_EncodeCodePage(int code_page, PyObject *unicode, const char *errors) + + Encode the Unicode object using the specified code page and return a Python + bytes object. Return *NULL* if an exception was raised by the codec. Use + :c:data:`CP_ACP` code page to get the MBCS encoder. + + .. versionadded:: 3.3 + + .. c:function:: PyObject* PyUnicode_EncodeMBCS(const Py_UNICODE *s, Py_ssize_t size, const char *errors) Encode the :c:type:`Py_UNICODE` buffer of the given *size* using MBCS and return a Python bytes object. Return *NULL* if an exception was raised by the codec. - -.. c:function:: PyObject* PyUnicode_AsMBCSString(PyObject *unicode) - - Encode a Unicode object using MBCS and return the result as Python bytes - object. Error handling is "strict". Return *NULL* if an exception was - raised by the codec. + .. deprecated-removed:: 3.3 4.0 + Part of the old-style :c:type:`Py_UNICODE` API; please migrate to using + :c:func:`PyUnicode_AsMBCSString` or :c:func:`PyUnicode_EncodeCodePage`. Methods & Slots @@ -1011,7 +1534,8 @@ They all return *NULL* or ``-1`` if an exception occurs. characters are not included in the resulting strings. -.. c:function:: PyObject* PyUnicode_Translate(PyObject *str, PyObject *table, const char *errors) +.. c:function:: PyObject* PyUnicode_Translate(PyObject *str, PyObject *table, \ + const char *errors) Translate a string by applying a character mapping table to it and return the resulting Unicode object. @@ -1033,14 +1557,16 @@ They all return *NULL* or ``-1`` if an exception occurs. Unicode string. -.. c:function:: int PyUnicode_Tailmatch(PyObject *str, PyObject *substr, Py_ssize_t start, Py_ssize_t end, int direction) +.. c:function:: int PyUnicode_Tailmatch(PyObject *str, PyObject *substr, \ + Py_ssize_t start, Py_ssize_t end, int direction) Return 1 if *substr* matches ``str[start:end]`` at the given tail end (*direction* == -1 means to do a prefix match, *direction* == 1 a suffix match), 0 otherwise. Return ``-1`` if an error occurred. -.. c:function:: Py_ssize_t PyUnicode_Find(PyObject *str, PyObject *substr, Py_ssize_t start, Py_ssize_t end, int direction) +.. c:function:: Py_ssize_t PyUnicode_Find(PyObject *str, PyObject *substr, \ + Py_ssize_t start, Py_ssize_t end, int direction) Return the first position of *substr* in ``str[start:end]`` using the given *direction* (*direction* == 1 means to do a forward search, *direction* == -1 a @@ -1049,13 +1575,27 @@ They all return *NULL* or ``-1`` if an exception occurs. occurred and an exception has been set. -.. c:function:: Py_ssize_t PyUnicode_Count(PyObject *str, PyObject *substr, Py_ssize_t start, Py_ssize_t end) +.. c:function:: Py_ssize_t PyUnicode_FindChar(PyObject *str, Py_UCS4 ch, \ + Py_ssize_t start, Py_ssize_t end, int direction) + + Return the first position of the character *ch* in ``str[start:end]`` using + the given *direction* (*direction* == 1 means to do a forward search, + *direction* == -1 a backward search). The return value is the index of the + first match; a value of ``-1`` indicates that no match was found, and ``-2`` + indicates that an error occurred and an exception has been set. + + .. versionadded:: 3.3 + + +.. c:function:: Py_ssize_t PyUnicode_Count(PyObject *str, PyObject *substr, \ + Py_ssize_t start, Py_ssize_t end) Return the number of non-overlapping occurrences of *substr* in ``str[start:end]``. Return ``-1`` if an error occurred. -.. c:function:: PyObject* PyUnicode_Replace(PyObject *str, PyObject *substr, PyObject *replstr, Py_ssize_t maxcount) +.. c:function:: PyObject* PyUnicode_Replace(PyObject *str, PyObject *substr, \ + PyObject *replstr, Py_ssize_t maxcount) Replace at most *maxcount* occurrences of *substr* in *str* with *replstr* and return the resulting Unicode object. *maxcount* == -1 means replace all @@ -1076,7 +1616,7 @@ They all return *NULL* or ``-1`` if an exception occurs. ISO-8859-1 if it contains non-ASCII characters". -.. c:function:: int PyUnicode_RichCompare(PyObject *left, PyObject *right, int op) +.. c:function:: PyObject* PyUnicode_RichCompare(PyObject *left, PyObject *right, int op) Rich compare two unicode strings and return one of the following: @@ -1103,8 +1643,8 @@ They all return *NULL* or ``-1`` if an exception occurs. Check whether *element* is contained in *container* and return true or false accordingly. - *element* has to coerce to a one element Unicode string. ``-1`` is returned if - there was an error. + *element* has to coerce to a one element Unicode string. ``-1`` is returned + if there was an error. .. c:function:: void PyUnicode_InternInPlace(PyObject **string) @@ -1123,7 +1663,6 @@ They all return *NULL* or ``-1`` if an exception occurs. .. c:function:: PyObject* PyUnicode_InternFromString(const char *v) A combination of :c:func:`PyUnicode_FromString` and - :c:func:`PyUnicode_InternInPlace`, returning either a new unicode string object - that has been interned, or a new ("owned") reference to an earlier interned - string object with the same value. - + :c:func:`PyUnicode_InternInPlace`, returning either a new unicode string + object that has been interned, or a new ("owned") reference to an earlier + interned string object with the same value. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 41cdd6b..14ef8df 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -95,12 +95,6 @@ the same library that the Python runtime is using. leaving *closeit* set to ``0`` and *flags* set to *NULL*. -.. c:function:: int PyRun_SimpleFileFlags(FILE *fp, const char *filename, PyCompilerFlags *flags) - - This is a simplified interface to :c:func:`PyRun_SimpleFileExFlags` below, - leaving *closeit* set to ``0``. - - .. c:function:: int PyRun_SimpleFileEx(FILE *fp, const char *filename, int closeit) This is a simplified interface to :c:func:`PyRun_SimpleFileExFlags` below, @@ -289,7 +283,7 @@ the same library that the Python runtime is using. frame *f* is executed, interpreting bytecode and executing calls as needed. The additional *throwflag* parameter can mostly be ignored - if true, then it causes an exception to immediately be thrown; this is used for the - :meth:`throw` methods of generator objects. + :meth:`~generator.throw` methods of generator objects. .. c:function:: int PyEval_MergeCompilerFlags(PyCompilerFlags *cf) |