From 9a13a388f202268dd7b771638adbec132449b98b Mon Sep 17 00:00:00 2001 From: Jeroen Demeyer Date: Tue, 12 Nov 2019 14:08:00 +0100 Subject: bpo-36974: expand call protocol documentation (GH-13844) CC @encukou I'm also adding Petr Viktorin as contributor for vectorcall in the "what's new" section. https://bugs.python.org/issue36974 Automerge-Triggered-By: @encukou Automerge-Triggered-By: @encukou --- Doc/c-api/abstract.rst | 1 + Doc/c-api/call.rst | 411 +++++++++++++++++++++++++++++++++++++++++++++++ Doc/c-api/exceptions.rst | 4 + Doc/c-api/object.rst | 240 --------------------------- Doc/c-api/structures.rst | 3 +- Doc/c-api/type.rst | 1 + Doc/c-api/typeobj.rst | 90 +++++------ Doc/whatsnew/3.8.rst | 10 +- 8 files changed, 467 insertions(+), 293 deletions(-) create mode 100644 Doc/c-api/call.rst diff --git a/Doc/c-api/abstract.rst b/Doc/c-api/abstract.rst index 0edd1d5..1823f9d 100644 --- a/Doc/c-api/abstract.rst +++ b/Doc/c-api/abstract.rst @@ -18,6 +18,7 @@ but whose items have not been set to some non-\ ``NULL`` value yet. .. toctree:: object.rst + call.rst number.rst sequence.rst mapping.rst diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst new file mode 100644 index 0000000..0833531 --- /dev/null +++ b/Doc/c-api/call.rst @@ -0,0 +1,411 @@ +.. highlight:: c + +.. _call: + +Call Protocol +============= + +CPython supports two different calling protocols: +*tp_call* and vectorcall. + +The *tp_call* Protocol +---------------------- + +Instances of classes that set :c:member:`~PyTypeObject.tp_call` are callable. +The signature of the slot is:: + + PyObject *tp_call(PyObject *callable, PyObject *args, PyObject *kwargs); + +A call is made using a tuple for the positional arguments +and a dict for the keyword arguments, similarly to +``callable(*args, **kwargs)`` in Python code. +*args* must be non-NULL (use an empty tuple if there are no arguments) +but *kwargs* may be *NULL* if there are no keyword arguments. + +This convention is not only used by *tp_call*: +:c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init` +also pass arguments this way. + +To call an object, use :c:func:`PyObject_Call` or other +:ref:`call API `. + + +.. _vectorcall: + +The Vectorcall Protocol +----------------------- + +.. versionadded:: 3.8 + +The vectorcall protocol was introduced in :pep:`590` as an additional protocol +for making calls more efficient. + +.. warning:: + + The vectorcall API is provisional and expected to become public in + Python 3.9, with a different names and, possibly, changed semantics. + If you use the it, plan for updating your code for Python 3.9. + +As rule of thumb, CPython will prefer the vectorcall for internal calls +if the callable supports it. However, this is not a hard rule. +Additionally, some third-party extensions use *tp_call* directly +(rather than using :c:func:`PyObject_Call`). +Therefore, a class supporting vectorcall must also implement +:c:member:`~PyTypeObject.tp_call`. +Moreover, the callable must behave the same +regardless of which protocol is used. +The recommended way to achieve this is by setting +:c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`. +This bears repeating: + +.. warning:: + + A class supporting vectorcall **must** also implement + :c:member:`~PyTypeObject.tp_call` with the same semantics. + +A class should not implement vectorcall if that would be slower +than *tp_call*. For example, if the callee needs to convert +the arguments to an args tuple and kwargs dict anyway, then there is no point +in implementing vectorcall. + +Classes can implement the vectorcall protocol by enabling the +:const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag and setting +:c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the +object structure where a *vectorcallfunc* appears. +This is a pointer to a function with the following signature: + +.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) + +- *callable* is the object being called. +- *args* is a C array consisting of the positional arguments followed by the + values of the keyword arguments. + This can be *NULL* if there are no arguments. +- *nargsf* is the number of positional arguments plus possibly the + :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. + To get the actual number of positional arguments from *nargsf*, + use :c:func:`PyVectorcall_NARGS`. +- *kwnames* is a tuple containing the names of the keyword arguments; + in other words, the keys of the kwargs dict. + These names must be strings (instances of ``str`` or a subclass) + and they must be unique. + If there are no keyword arguments, then *kwnames* can instead be *NULL*. + +.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET + + If this flag is set in a vectorcall *nargsf* argument, the callee is allowed + to temporarily change ``args[-1]``. In other words, *args* points to + argument 1 (not 0) in the allocated vector. + The callee must restore the value of ``args[-1]`` before returning. + + For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that + ``args[0]`` may be changed. + + Whenever they can do so cheaply (without additional allocation), callers + are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. + Doing so will allow callables such as bound methods to make their onward + calls (which include a prepended *self* argument) very efficiently. + +To call an object that implements vectorcall, use a :ref:`call API ` +function as with any other callable. +:c:func:`_PyObject_Vectorcall` will usually be most efficient. + + +Recursion Control +................. + +When using *tp_call*, callees do not need to worry about +:ref:`recursion `: CPython uses +:c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall` +for calls made using *tp_call*. + +For efficiency, this is not the case for calls done using vectorcall: +the callee should use *Py_EnterRecursiveCall* and *Py_LeaveRecursiveCall* +if needed. + + +Vectorcall Support API +...................... + +.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) + + Given a vectorcall *nargsf* argument, return the actual number of + arguments. + Currently equivalent to:: + + (Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET) + + However, the function ``PyVectorcall_NARGS`` should be used to allow + for future extensions. + + .. versionadded:: 3.8 + +.. c:function:: vectorcallfunc _PyVectorcall_Function(PyObject *op) + + If *op* does not support the vectorcall protocol (either because the type + does not or because the specific instance does not), return *NULL*. + Otherwise, return the vectorcall function pointer stored in *op*. + This function never raises an exception. + + This is mostly useful to check whether or not *op* supports vectorcall, + which can be done by checking ``_PyVectorcall_Function(op) != NULL``. + + .. versionadded:: 3.8 + +.. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) + + Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword + arguments given in a tuple and dict, respectively. + + This is a specialized function, intended to be put in the + :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. + It does not check the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag + and it does not fall back to ``tp_call``. + + .. versionadded:: 3.8 + + +.. _capi-call: + +Object Calling API +------------------ + +Various functions are available for calling a Python object. +Each converts its arguments to a convention supported by the called object – +either *tp_call* or vectorcall. +In order to do as litle conversion as possible, pick one that best fits +the format of data you have available. + +The following table summarizes the available functions; +please see individual documentation for details. + ++------------------------------------------+------------------+--------------------+---------------+ +| Function | callable | args | kwargs | ++==========================================+==================+====================+===============+ +| :c:func:`PyObject_Call` | ``PyObject *`` | tuple | dict/``NULL`` | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallNoArgs` | ``PyObject *`` | --- | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_CallOneArg` | ``PyObject *`` | 1 object | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallObject` | ``PyObject *`` | tuple/``NULL`` | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallFunction` | ``PyObject *`` | format | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallMethod` | obj + ``char*`` | format | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallFunctionObjArgs` | ``PyObject *`` | variadic | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallMethodObjArgs` | obj + name | variadic | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_CallMethodNoArgs` | obj + name | --- | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_CallMethodOneArg` | obj + name | 1 object | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_Vectorcall` | ``PyObject *`` | vectorcall | vectorcall | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_FastCallDict` | ``PyObject *`` | vectorcall | dict/``NULL`` | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_VectorcallMethod` | arg + name | vectorcall | vectorcall | ++------------------------------------------+------------------+--------------------+---------------+ + + +.. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs) + + Call a callable Python object *callable*, with arguments given by the + tuple *args*, and named arguments given by the dictionary *kwargs*. + + *args* must not be *NULL*; use an empty tuple if no arguments are needed. + If no named arguments are needed, *kwargs* can be *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: + ``callable(*args, **kwargs)``. + + +.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) + + Call a callable Python object *callable* without any arguments. It is the + most efficient way to call a callable Python object without any argument. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg) + + Call a callable Python object *callable* with exactly 1 positional argument + *arg* and no keyword arguments. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args) + + Call a callable Python object *callable*, with arguments given by the + tuple *args*. If no arguments are needed, then *args* can be *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: ``callable(*args)``. + + +.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...) + + Call a callable Python object *callable*, with a variable number of C arguments. + The C arguments are described using a :c:func:`Py_BuildValue` style format + string. The format can be *NULL*, indicating that no arguments are provided. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: ``callable(*args)``. + + Note that if you only pass :c:type:`PyObject \*` args, + :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative. + + .. versionchanged:: 3.4 + The type of *format* was changed from ``char *``. + + +.. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...) + + Call the method named *name* of object *obj* with a variable number of C + arguments. The C arguments are described by a :c:func:`Py_BuildValue` format + string that should produce a tuple. + + The format can be *NULL*, indicating that no arguments are provided. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: + ``obj.name(arg1, arg2, ...)``. + + Note that if you only pass :c:type:`PyObject \*` args, + :c:func:`PyObject_CallMethodObjArgs` is a faster alternative. + + .. versionchanged:: 3.4 + The types of *name* and *format* were changed from ``char *``. + + +.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL) + + Call a callable Python object *callable*, with a variable number of + :c:type:`PyObject \*` arguments. The arguments are provided as a variable number + of parameters followed by *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: + ``callable(arg1, arg2, ...)``. + + +.. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL) + + Call a method of the Python object *obj*, where the name of the method is given as a + Python string object in *name*. It is called with a variable number of + :c:type:`PyObject \*` arguments. The arguments are provided as a variable number + of parameters followed by *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + +.. c:function:: PyObject* _PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name) + + Call a method of the Python object *obj* without arguments, + where the name of the method is given as a Python string object in *name*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg) + + Call a method of the Python object *obj* with a single positional argument + *arg*, where the name of the method is given as a Python string object in + *name*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) + + Call a callable Python object *callable*. + The arguments are the same as for :c:type:`vectorcallfunc`. + If *callable* supports vectorcall_, this directly calls + the vectorcall function stored in *callable*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. note:: + + This function is provisional and expected to become public in Python 3.9, + with a different name and, possibly, changed semantics. + If you use the function, plan for updating your code for Python 3.9. + + .. versionadded:: 3.8 + +.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) + + Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol, + but with keyword arguments passed as a dictionary *kwdict*. + The *args* array contains only the positional arguments. + + Regardless of which protocol is used internally, + a conversion of arguments needs to be done. + Therefore, this function should only be used if the caller + already has a dictionary ready to use for the keyword arguments, + but not a tuple for the positional arguments. + + .. note:: + + This function is provisional and expected to become public in Python 3.9, + with a different name and, possibly, changed semantics. + If you use the function, plan for updating your code for Python 3.9. + + .. versionadded:: 3.8 + +.. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) + + Call a method using the vectorcall calling convention. The name of the method + is given as a Python string *name*. The object whose method is called is + *args[0]*, and the *args* array starting at *args[1]* represents the arguments + of the call. There must be at least one positional argument. + *nargsf* is the number of positional arguments including *args[0]*, + plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may + temporarily be changed. Keyword arguments can be passed just like in + :c:func:`_PyObject_Vectorcall`. + + If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, + this will call the unbound method object with the full + *args* vector as arguments. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +Call Support API +---------------- + +.. c:function:: int PyCallable_Check(PyObject *o) + + Determine if the object *o* is callable. Return ``1`` if the object is callable + and ``0`` otherwise. This function always succeeds. diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index cd6df00..2edcbf7 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -697,6 +697,8 @@ The following functions are used to create and modify Unicode exceptions from C. ``0`` on success, ``-1`` on failure. +.. _recursion: + Recursion Control ================= @@ -704,6 +706,8 @@ These two functions provide a way to perform safe recursive calls at the C level, both in the core and in extension modules. They are needed if the recursive code does not necessarily invoke Python code (which tracks its recursion depth automatically). +They are also not needed for *tp_call* implementations +because the :ref:`call protocol ` takes care of recursion handling. .. c:function:: int Py_EnterRecursiveCall(const char *where) diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 7d7a3be..ca9db1a 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -248,246 +248,6 @@ Object Protocol of base classes). -.. c:function:: int PyCallable_Check(PyObject *o) - - Determine if the object *o* is callable. Return ``1`` if the object is callable - and ``0`` otherwise. This function always succeeds. - - -.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) - - Call a callable Python object *callable* without any arguments. It is the - most efficient way to call a callable Python object without any argument. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg) - - Call a callable Python object *callable* with exactly 1 positional argument - *arg* and no keyword arguments. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs) - - Call a callable Python object *callable*, with arguments given by the - tuple *args*, and named arguments given by the dictionary *kwargs*. - - *args* must not be ``NULL``, use an empty tuple if no arguments are needed. - If no named arguments are needed, *kwargs* can be ``NULL``. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - This is the equivalent of the Python expression: - ``callable(*args, **kwargs)``. - - -.. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args) - - Call a callable Python object *callable*, with arguments given by the - tuple *args*. If no arguments are needed, then *args* can be ``NULL``. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - This is the equivalent of the Python expression: ``callable(*args)``. - - -.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...) - - Call a callable Python object *callable*, with a variable number of C arguments. - The C arguments are described using a :c:func:`Py_BuildValue` style format - string. The format can be ``NULL``, indicating that no arguments are provided. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - This is the equivalent of the Python expression: ``callable(*args)``. - - Note that if you only pass :c:type:`PyObject \*` args, - :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative. - - .. versionchanged:: 3.4 - The type of *format* was changed from ``char *``. - - -.. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...) - - Call the method named *name* of object *obj* with a variable number of C - arguments. The C arguments are described by a :c:func:`Py_BuildValue` format - string that should produce a tuple. - - The format can be ``NULL``, indicating that no arguments are provided. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - This is the equivalent of the Python expression: - ``obj.name(arg1, arg2, ...)``. - - Note that if you only pass :c:type:`PyObject \*` args, - :c:func:`PyObject_CallMethodObjArgs` is a faster alternative. - - .. versionchanged:: 3.4 - The types of *name* and *format* were changed from ``char *``. - - -.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL) - - Call a callable Python object *callable*, with a variable number of - :c:type:`PyObject\*` arguments. The arguments are provided as a variable number - of parameters followed by ``NULL``. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - This is the equivalent of the Python expression: - ``callable(arg1, arg2, ...)``. - - -.. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL) - - Calls a method of the Python object *obj*, where the name of the method is given as a - Python string object in *name*. It is called with a variable number of - :c:type:`PyObject\*` arguments. The arguments are provided as a variable number - of parameters followed by ``NULL``. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - -.. c:function:: PyObject* _PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name) - - Call a method of the Python object *obj* without arguments, - where the name of the method is given as a Python string object in *name*. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* _PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg) - - Call a method of the Python object *obj* with a single positional argument - *arg*, where the name of the method is given as a Python string object in - *name*. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - Call a callable Python object *callable*, using - :c:data:`vectorcall ` if possible. - - *args* is a C array with the positional arguments. - - *nargsf* is the number of positional arguments plus optionally the flag - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` (see below). - To get actual number of arguments, use - :c:func:`PyVectorcall_NARGS(nargsf) `. - - *kwnames* can be either ``NULL`` (no keyword arguments) or a tuple of keyword - names, which must be strings. In the latter case, the values of the keyword - arguments are stored in *args* after the positional arguments. - The number of keyword arguments does not influence *nargsf*. - - *kwnames* must contain only objects of type ``str`` (not a subclass), - and all keys must be unique. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - This uses the vectorcall protocol if the callable supports it; - otherwise, the arguments are converted to use - :c:member:`~PyTypeObject.tp_call`. - - .. note:: - - This function is provisional and expected to become public in Python 3.9, - with a different name and, possibly, changed semantics. - If you use the function, plan for updating your code for Python 3.9. - - .. versionadded:: 3.8 - -.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET - - If set in a vectorcall *nargsf* argument, the callee is allowed to - temporarily change ``args[-1]``. In other words, *args* points to - argument 1 (not 0) in the allocated vector. - The callee must restore the value of ``args[-1]`` before returning. - - For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that - ``args[0]`` may be changed. - - Whenever they can do so cheaply (without additional allocation), callers - are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. - Doing so will allow callables such as bound methods to make their onward - calls (which include a prepended *self* argument) cheaply. - - .. versionadded:: 3.8 - -.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) - - Given a vectorcall *nargsf* argument, return the actual number of - arguments. - Currently equivalent to ``nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET``. - - .. versionadded:: 3.8 - -.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) - - Same as :c:func:`_PyObject_Vectorcall` except that the keyword arguments - are passed as a dictionary in *kwdict*. This may be ``NULL`` if there - are no keyword arguments. - - For callables supporting :c:data:`vectorcall `, - the arguments are internally converted to the vectorcall convention. - Therefore, this function adds some overhead compared to - :c:func:`_PyObject_Vectorcall`. - It should only be used if the caller already has a dictionary ready to use. - - .. note:: - - This function is provisional and expected to become public in Python 3.9, - with a different name and, possibly, changed semantics. - If you use the function, plan for updating your code for Python 3.9. - - .. versionadded:: 3.8 - -.. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - Call a method using the vectorcall calling convention. The name of the method - is given as Python string *name*. The object whose method is called is - *args[0]* and the *args* array starting at *args[1]* represents the arguments - of the call. There must be at least one positional argument. - *nargsf* is the number of positional arguments including *args[0]*, - plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may - temporarily be changed. Keyword arguments can be passed just like in - :c:func:`_PyObject_Vectorcall`. - - If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, - this will actually call the unbound method object with the full - *args* vector as arguments. - - Return the result of the call on success, or raise an exception and return - ``NULL`` on failure. - - .. versionadded:: 3.9 - .. c:function:: Py_hash_t PyObject_Hash(PyObject *o) .. index:: builtin: hash diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 1035277..1bd769f 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -208,7 +208,8 @@ also keyword arguments. So there are a total of 6 calling conventions: Extension of :const:`METH_FASTCALL` supporting also keyword arguments, with methods of type :c:type:`_PyCFunctionFastWithKeywords`. - Keyword arguments are passed the same way as in the vectorcall protocol: + Keyword arguments are passed the same way as in the + :ref:`vectorcall protocol `: there is an additional fourth :c:type:`PyObject\*` parameter which is a tuple representing the names of the keyword arguments (which are guaranteed to be strings) diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index b1b2df9..41956b7 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -193,6 +193,7 @@ The following functions and structs are used to create (see :ref:`PyMemberDef `) * :c:member:`~PyTypeObject.tp_dictoffset` (see :ref:`PyMemberDef `) + * :c:member:`~PyTypeObject.tp_vectorcall_offset` * :c:member:`~PyBufferProcs.bf_getbuffer` * :c:member:`~PyBufferProcs.bf_releasebuffer` diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index bff5abf..b4ffd6b 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -49,7 +49,7 @@ Quick Reference +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_dealloc` | :c:type:`destructor` | | X | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ - | :c:member:`~PyTypeObject.tp_vectorcall_offset` | Py_ssize_t | | | | | ? | + | :c:member:`~PyTypeObject.tp_vectorcall_offset` | Py_ssize_t | | | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_getattr`) | :c:type:`getattrfunc` | __getattribute__, | | | | G | | | | __getattr__ | | | | | @@ -145,6 +145,8 @@ Quick Reference +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_finalize` | :c:type:`destructor` | __del__ | | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ + | :c:member:`~PyTypeObject.tp_vectorcall` | :c:type:`vectorcallfunc` | | | | | | + +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ If :const:`COUNT_ALLOCS` is defined then the following (internal-only) fields exist as well: @@ -180,7 +182,7 @@ fields exist as well: .. code-block:: none - X - type slot is inherited via PyType_Ready if defined with a NULL value + X - type slot is inherited via *PyType_Ready* if defined with a *NULL* value % - the slots of the sub-struct are inherited individually G - inherited, but only in combination with other slots; see the slot's description ? - it's complicated; see the slot's description @@ -687,42 +689,29 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: Py_ssize_t PyTypeObject.tp_vectorcall_offset An optional offset to a per-instance function that implements calling - the object using the *vectorcall* protocol, a more efficient alternative + the object using the :ref:`vectorcall protocol `, + a more efficient alternative of the simpler :c:member:`~PyTypeObject.tp_call`. This field is only used if the flag :const:`_Py_TPFLAGS_HAVE_VECTORCALL` is set. If so, this must be a positive integer containing the offset in the instance of a :c:type:`vectorcallfunc` pointer. - The signature is the same as for :c:func:`_PyObject_Vectorcall`:: - - PyObject *vectorcallfunc(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - The *vectorcallfunc* pointer may be zero, in which case the instance behaves + The *vectorcallfunc* pointer may be ``NULL``, in which case the instance behaves as if :const:`_Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance falls back to :c:member:`~PyTypeObject.tp_call`. Any class that sets ``_Py_TPFLAGS_HAVE_VECTORCALL`` must also set :c:member:`~PyTypeObject.tp_call` and make sure its behaviour is consistent with the *vectorcallfunc* function. - This can be done by setting *tp_call* to ``PyVectorcall_Call``: - - .. c:function:: PyObject *PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) + This can be done by setting *tp_call* to :c:func:`PyVectorcall_Call`. - Call *callable*'s *vectorcallfunc* with positional and keyword - arguments given in a tuple and dict, respectively. - - This function is intended to be used in the ``tp_call`` slot. - It does not fall back to ``tp_call`` and it currently does not check the - ``_Py_TPFLAGS_HAVE_VECTORCALL`` flag. - To call an object, use one of the :c:func:`PyObject_Call ` - functions instead. - - .. note:: + .. warning:: It is not recommended for :ref:`heap types ` to implement the vectorcall protocol. - When a user sets ``__call__`` in Python code, only ``tp_call`` is updated, - possibly making it inconsistent with the vectorcall function. + When a user sets :attr:`__call__` in Python code, only *tp_call* is updated, + likely making it inconsistent with the vectorcall function. .. note:: @@ -732,18 +721,19 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionchanged:: 3.8 - This slot was used for print formatting in Python 2.x. - In Python 3.0 to 3.7, it was reserved and named ``tp_print``. + Before version 3.8, this slot was named ``tp_print``. + In Python 2.x, it was used for printing to a file. + In Python 3.0 to 3.7, it was unused. **Inheritance:** - This field is inherited by subtypes together with - :c:member:`~PyTypeObject.tp_call`: a subtype inherits - :c:member:`~PyTypeObject.tp_vectorcall_offset` from its base type when - the subtype’s :c:member:`~PyTypeObject.tp_call` is ``NULL``. - - Note that `heap types`_ (including subclasses defined in Python) do not - inherit the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag. + This field is always inherited. + However, the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag is not + always inherited. If it's not, then the subclass won't use + :ref:`vectorcall `, except when + :c:func:`PyVectorcall_Call` is explicitly called. + This is in particular the case for `heap types`_ + (including subclasses defined in Python). .. c:member:: getattrfunc PyTypeObject.tp_getattr @@ -1171,18 +1161,17 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_finalize` slot is always present in the type structure. + .. data:: _Py_TPFLAGS_HAVE_VECTORCALL - This bit is set when the class implements the vectorcall protocol. + This bit is set when the class implements + the :ref:`vectorcall protocol `. See :c:member:`~PyTypeObject.tp_vectorcall_offset` for details. **Inheritance:** - This bit is set on *static* subtypes if ``tp_flags`` is not overridden: - a subtype inherits ``_Py_TPFLAGS_HAVE_VECTORCALL`` from its base type - when the subtype’s :c:member:`~PyTypeObject.tp_call` is ``NULL`` - and the subtype's ``Py_TPFLAGS_HEAPTYPE`` is not set. - + This bit is inherited for *static* subtypes if + :c:member:`~PyTypeObject.tp_call` is also inherited. `Heap types`_ do not inherit ``_Py_TPFLAGS_HAVE_VECTORCALL``. .. note:: @@ -1715,9 +1704,9 @@ and :c:type:`PyType_Type` effectively act as defaults.) PyObject *tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds); - The subtype argument is the type of the object being created; the *args* and + The *subtype* argument is the type of the object being created; the *args* and *kwds* arguments represent positional and keyword arguments of the call to the - type. Note that subtype doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` + type. Note that *subtype* doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` function is called; it may be a subtype of that type (but not an unrelated type). @@ -1900,6 +1889,21 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. seealso:: "Safe object finalization" (:pep:`442`) +.. c:member:: vectorcallfunc PyTypeObject.tp_vectorcall + + Vectorcall function to use for calls of this type object. + In other words, it is used to implement + :ref:`vectorcall ` for ``type.__call__``. + If ``tp_vectorcall`` is ``NULL``, the default call implementation + using :attr:`__new__` and :attr:`__init__` is used. + + **Inheritance:** + + This field is never inherited. + + .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9) + + The remaining fields are only defined if the feature test macro :const:`COUNT_ALLOCS` is defined, and are for internal use only. They are documented here for completeness. None of these fields are inherited by @@ -2369,14 +2373,6 @@ Slot Type typedefs .. c:type:: void (*destructor)(PyObject *) -.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - See :c:member:`~PyTypeObject.tp_vectorcall_offset`. - - Arguments to ``vectorcallfunc`` are the same as for :c:func:`_PyObject_Vectorcall`. - - .. versionadded:: 3.8 - .. c:type:: void (*freefunc)(void *) See :c:member:`~PyTypeObject.tp_free`. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 8338cb6..0890955 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -347,20 +347,20 @@ See :pep:`587` for a full description. (Contributed by Victor Stinner in :issue:`36763`.) -Vectorcall: a fast calling protocol for CPython ------------------------------------------------ +PEP 590: Vectorcall: a fast calling protocol for CPython +-------------------------------------------------------- -The "vectorcall" protocol is added to the Python/C API. +:ref:`vectorcall` is added to the Python/C API. It is meant to formalize existing optimizations which were already done for various classes. -Any extension type implementing a callable can use this protocol. +Any static type implementing a callable can use this protocol. This is currently provisional. The aim is to make it fully public in Python 3.9. See :pep:`590` for a full description. -(Contributed by Jeroen Demeyer and Mark Shannon in :issue:`36974`.) +(Contributed by Jeroen Demeyer, Mark Shannon and Petr Viktorin in :issue:`36974`.) Pickle protocol 5 with out-of-band data buffers -- cgit v0.12