summaryrefslogtreecommitdiffstats
path: root/Modules/_testinternalcapi.c
Commit message (Collapse)AuthorAgeFilesLines
* [3.13] gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121013)Miss Islington (bot)2024-06-261-2/+2
| | | | | | | | | We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence(). This change only affects internal API. (cherry picked from commit a905721b9c5c15279e67c2f7785034b7356b2d46, AKA gh-121010) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.13] gh-120642: Move private PyCode APIs to the internal C API (#120643) ↵Victor Stinner2024-06-261-4/+4
| | | | | | | | | | | | | | | | | | | | | (#121043) gh-120642: Move private PyCode APIs to the internal C API (#120643) * Move _Py_CODEUNIT and related functions to pycore_code.h. * Move _Py_BackoffCounter to pycore_backoff.h. * Move Include/cpython/optimizer.h content to pycore_optimizer.h. * Remove Include/cpython/optimizer.h. * Remove PyUnstable_Replace_Executor(). Rename functions: * PyUnstable_GetExecutor() => _Py_GetExecutor() * PyUnstable_GetOptimizer() => _Py_GetOptimizer() * PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer() * PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter() * PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer() (cherry picked from commit 9e4a81f00fef689c6e18a64245aa064eaadc7ac7)
* [3.13] gh-117657: Fix race involving immortalizing objects (GH-119927) (#120005)Sam Gross2024-06-031-15/+9
| | | | | | | | | | | The free-threaded build currently immortalizes objects that use deferred reference counting (see gh-117783). This typically happens once the first non-main thread is created, but the behavior can be suppressed for tests, in subinterpreters, or during a compile() call. This fixes a race condition involving the tracking of whether the behavior is suppressed. (cherry picked from commit 47fb4327b5c405da6df066dcaa01b7c1aefab313)
* [3.13] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across ↵Miss Islington (bot)2024-05-221-0/+20
| | | | | | | | | | | | | Interpreters (gh-119331) (gh-119410) _PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed. This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple. This change also fixes a bug where only the most recent parser was added to the global linked list. (cherry picked from commit 81865002aee8eaaeb3c7e402f86183afa6de77bf) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* gh-118527: Intern code consts in free-threaded build (#118667)Sam Gross2024-05-071-0/+12
| | | | | | We already intern and immortalize most string constants. In the free-threaded build, other constants can be a source of reference count contention because they are shared by all threads running the same code objects.
* gh-116322: Add Py_mod_gil module slot (#116882)Brett Simmers2024-05-031-0/+1
| | | | | | | | | | | | | | This PR adds the ability to enable the GIL if it was disabled at interpreter startup, and modifies the multi-phase module initialization path to enable the GIL when loading a module, unless that module's spec includes a slot indicating it can run safely without the GIL. PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148. A warning will be issued up to once per interpreter for the first GIL-using module that is loaded. If `-v` is given, a shorter message will be printed to stderr every time a GIL-using module is loaded (including the first one that issues a warning).
* gh-118335: Configure Tier 2 interpreter at build time (#118339)Guido van Rossum2024-05-011-1/+12
| | | | | | | | | | | | | | | | | | | | | | The code for Tier 2 is now only compiled when configured with `--enable-experimental-jit[=yes|interpreter]`. We drop support for `PYTHON_UOPS` and -`Xuops`, but you can disable the interpreter or JIT at runtime by setting `PYTHON_JIT=0`. You can also build it without enabling it by default using `--enable-experimental-jit=yes-off`; enable with `PYTHON_JIT=1`. On Windows, the `build.bat` script supports `--experimental-jit`, `--experimental-jit-off`, `--experimental-interpreter`. In the C code, `_Py_JIT` is defined as before when the JIT is enabled; the new variable `_Py_TIER2` is defined when the JIT *or* the interpreter is enabled. It is actually a bitmask: 1: JIT; 2: default-off; 4: interpreter.
* gh-117783: Immortalize objects that use deferred reference counting (#118112)Sam Gross2024-04-291-0/+22
| | | | | | | | | Deferred reference counting is not fully implemented yet. As a temporary measure, we immortalize objects that would use deferred reference counting to avoid multi-threaded scaling bottlenecks. This is only performed in the free-threaded build once the first non-main thread is started. Additionally, some tests, including refleak tests, suppress this behavior.
* gh-110693: Pending Calls Machinery Cleanups (gh-118296)Eric Snow2024-04-261-21/+42
| | | This does some cleanup in preparation for later changes.
* gh-117680: make _PyInstructionSequence a PyObject and use it in tests (#117629)Irit Katriel2024-04-171-0/+15
|
* gh-76785: Handle Legacy Interpreters Properly (gh-117490)Eric Snow2024-04-111-2/+2
| | | This is similar to the situation with threading._DummyThread. The methods (incl. __del__()) of interpreters.Interpreter objects must be careful with interpreters not created by interpreters.create(). The simplest thing to start with is to disable any method that modifies or runs in the interpreter. As part of this, the runtime keeps track of where an interpreter was created. We also handle interpreter "refcounts" properly.
* gh-76785: Add More Tests to test_interpreters.test_api (gh-117662)Eric Snow2024-04-111-30/+272
| | | In addition to the increase test coverage, this is a precursor to sorting out how we handle interpreters created directly via the C-API.
* gh-116968: Reimplement Tier 2 counters (#117144)Guido van Rossum2024-04-041-0/+6
| | | | | | | | | | | | Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``), shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The API used for adaptive specialization counters is changed but the behavior is (supposed to be) identical. The behavior of the Tier 2 counters is changed: - There are no longer dynamic thresholds (we never varied these). - All counters now use the same exponential backoff. - The counter for ``JUMP_BACKWARD`` starts counting down from 16. - The ``temperature`` in side exits starts counting down from 64.
* gh-116168: Remove extra `_CHECK_STACK_SPACE` uops (#117242)Peter Lazorchak2024-04-031-0/+12
| | | This merges all `_CHECK_STACK_SPACE` uops in a trace into a single `_CHECK_STACK_SPACE_OPERAND` uop that checks whether there is enough stack space for all calls included in the entire trace.
* gh-76785: Consolidate Some Interpreter-related Testing Helpers (gh-117485)Eric Snow2024-04-021-206/+9
| | | This eliminates the duplication of functionally identical helpers in the _testinternalcapi and _xxsubinterpreters modules.
* gh-76785: Add PyInterpreterConfig Helpers (gh-117170)Eric Snow2024-04-021-60/+146
| | | These helpers make it easier to customize and inspect the config used to initialize interpreters. This is especially valuable in our tests. I found inspiration from the PyConfig API for the PyInterpreterConfig dict conversion stuff. As part of this PR I've also added a bunch of tests.
* GH-115776: Embed the values array into the object, for "normal" Python ↵Mark Shannon2024-04-021-5/+15
| | | | objects. (GH-116115)
* gh-112383: teach dis how to interpret ENTER_EXECUTOR (#117171)Irit Katriel2024-03-231-21/+0
|
* gh-76785: Drop PyInterpreterID_Type (gh-117101)Eric Snow2024-03-211-0/+119
| | | I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation. Relatively recently I refactored that implementation to no longer rely on InterpreterID objects. Thus now I'm removing it.
* gh-76785: Clean Up Interpreter ID Conversions (gh-117048)Eric Snow2024-03-211-4/+28
| | | Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t. We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
* GH-112536: Add more TSan tests (#116911)Antoine Pitrou2024-03-171-2/+2
| | | | These may all exercise some non-trivial aspects of thread synchronization.
* gh-111696, PEP 737: Add PyType_GetModuleName() function (#116824)Victor Stinner2024-03-141-9/+0
| | | Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* GH-115816: Assorted naming and formatting changes to improve ↵Mark Shannon2024-02-271-1/+1
| | | | | | | maintainability. (GH-115987) * Rename _Py_UOpsAbstractInterpContext to _Py_UOpsContext and _Py_UOpsSymType to _Py_UopsSymbol. * #define shortened form of _Py_uop_... names for improved readability.
* GH-115816: Make tier2 optimizer symbols testable, and add a few tests. ↵Mark Shannon2024-02-271-1/+2
| | | | (GH-115953)
* gh-115168: Add pystats counter for invalidated executors (GH-115169)Michael Droettboom2024-02-261-1/+1
|
* Tier 2 cleanups and tweaks (#115534)Guido van Rossum2024-02-201-4/+4
| | | | | | | | | * Rename `_testinternalcapi.get_{uop,counter}_optimizer` to `new_*_optimizer` * Use `_PyUOpName()` instead of` _PyOpcode_uop_name[]` * Add `target` to executor iterator items -- `list(ex)` now returns `(opcode, oparg, target, operand)` quadruples * Add executor methods `get_opcode()` and `get_oparg()` to get `vmdata.opcode`, `vmdata.oparg` * Define a helper for printing uops, and unify various places where they are printed * Add a hack to summarize_stats.py to fix legacy uop names (e.g. `POP_TOP` -> `_POP_TOP`) * Define helpers in `test_opt.py` for accessing the set or list of opnames of an executor
* GH-112354: Initial implementation of warm up on exits and trace-stitching ↵Mark Shannon2024-02-201-1/+3
| | | | (GH-114142)
* gh-115058: Add ``reset_rare_event_counters`` function in `_testinternalcapi` ↵Kirill Podoprigora2024-02-121-0/+15
| | | | (GH-115128)
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build ↵Sam Gross2024-02-011-1/+11
| | | | | | | | | | | | | | | | | (#114564) * gh-112529: Remove PyGC_Head from object pre-header in free-threaded build This avoids allocating space for PyGC_Head in the free-threaded build. The GC implementation for free-threaded CPython does not use the PyGC_Head structure. * The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev` in the free-threaded build. * The GDB libpython.py file now determines the offset of the managed dict field based on whether the running process is a free-threaded build. Those are identified by the `ob_ref_local` field in PyObject. * Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the size of `PyGC_Head` in the size of static `PyTypeObject`.
* gh-114312: Fix rare event counter tests on aarch64 (GH-114554)Michael Droettboom2024-01-261-5/+5
|
* gh-114312: Collect stats for unlikely events (GH-114493)Michael Droettboom2024-01-251-0/+16
|
* gh-76785: Fixes for test.support.interpreters (gh-112982)Eric Snow2023-12-121-0/+12
| | | This involves a number of changes for PEP 734.
* gh-112535: Add test on _Py_ThreadId() (#112709)Victor Stinner2023-12-041-0/+14
| | | Add also test.support.Py_GIL_DISABLED constant.
* gh-76785: Add _PyType_GetModuleName() to the Internal C-API (gh-112323)Eric Snow2023-11-221-0/+10
| | | The new function corresponds to the existing (public) PyType_GetName() and PyType_GetQualName().
* gh-111798: Use lower Py_C_RECURSION_LIMIT in debug mode (#112124)Victor Stinner2023-11-161-0/+9
| | | | | | | * Run again test_ast_recursion_limit() on WASI platform. * Add _testinternalcapi.get_c_recursion_remaining(). * Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and test_trace_unpack_long_sequence() now adjust the maximum recursion depth depending on the the remaining C recursion.
* gh-111569: Implement Python critical section API (gh-111571)Sam Gross2023-11-081-0/+3
| | | | | | | | Critical sections are helpers to replace the global interpreter lock with finer grained locking. They provide similar guarantees to the GIL and avoid the deadlock risk that plain locking involves. Critical sections are implicitly ended whenever the GIL would be released. They are resumed when the GIL would be acquired. Nested critical sections behave as if the sections were interleaved.
* gh-108082: Remove _PyErr_WriteUnraisableMsg() (GH-111643)Serhiy Storchaka2023-11-031-32/+0
| | | | Replace the remaining calls with PyErr_FormatUnraisable().
* gh-76785: Move the Cross-Interpreter Code to Its Own File (gh-111502)Eric Snow2023-10-301-0/+54
| | | This is partly to clear this stuff out of pystate.c, but also in preparation for moving some code out of _xxsubinterpretersmodule.c. This change also moves this stuff to the internal API (new: Include/internal/pycore_crossinterp.h). @vstinner did this previously and I undid it. Now I'm re-doing it. :/
* gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost ↵gsallam2023-10-271-0/+32
| | | | of enabling perf-trampolines (#109666)
* GH-109369: Add machinery for deoptimizing tier2 executors, both individually ↵Mark Shannon2023-10-231-0/+28
| | | | and globally. (GH-110384)
* gh-110525: Add tests for internal `set` CAPI (GH-110630)Nikita Sobolev2023-10-101-0/+3
|
* gh-110241: Add missing error check to `record_eval` in `_testinternalcapi` ↵Nikita Sobolev2023-10-021-1/+5
| | | | (#110242)
* gh-109723: Disable Py_BUILD_CORE in _testcapi (#109727)Victor Stinner2023-09-221-0/+62
| | | | | | | | | | | | | | | | | | Make sure that the internal C API is not tested by mistake by _testcapi. Undefine Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE macros in Modules/_testcapi/parts.h: move code from _testcapimodule.c. heaptype_relative.c and vectorcall_limited.c are using the limited C API which is incompatible with the internal C API. Move test_long_numbits() from _testcapi to _testinternalcapi since it uses the internal C API "pycore_long.h". Fix Modules/_testcapi/pyatomic.c: don't include Python.h directly, just include _testcapi/parts.h. Ajust "make check-c-globals" for these changes.
* gh-109693: Remove pycore_atomic_funcs.h (#109694)Sam Gross2023-09-211-13/+0
| | | _PyUnicode_FromId() now uses pyatomic.h functions instead.
* gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344)Sam Gross2023-09-191-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case. The design is based on WebKit's WTF::Lock. PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot). This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events. This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock. Uncontended acquisition + release: * Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns * macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns * Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns PR Overview: The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below). * PyMutex: A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock. The API is internal only for now. * _PyParking_Lot: A futex-like API based on the API of the same name in WebKit. Used to implement PyMutex. * _PyRawMutex: A word sized lock used to implement _PyParking_Lot. * PyEvent: A one time event. This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR. * pycore_llist.h: Defines common operations on doubly-linked list. Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue) --------- Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)Victor Stinner2023-09-081-0/+5
| | | | | | | Symbols of the C API should be prefixed by "Py_" to avoid conflict with existing names in 3rd party C extensions on "#include <Python.h>". test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other _testcapi and _testinternalcapi constants.
* gh-106320: Remove private _PyErr_WriteUnraisableMsg() (#108863)Victor Stinner2023-09-041-1/+32
| | | | | | Move the private _PyErr_WriteUnraisableMsg() functions to the internal C API (pycore_pyerrors.h). Move write_unraisable_exc() from _testcapi to _testinternalcapi.
* gh-108777: Split _PyTime tests from _testinternalcapi.c (gh-108787)Sam Gross2023-09-021-261/+6
|
* gh-106320: Remove private _PyErr_ChainExceptions() (#108713)Victor Stinner2023-08-311-0/+103
| | | | | | | | | | | | | Remove _PyErr_ChainExceptions(), _PyErr_ChainExceptions1() and _PyErr_SetFromPyStatus() functions from the public C API. * Move the private _PyErr_ChainExceptions() and _PyErr_ChainExceptions1() function to the internal C API (pycore_pyerrors.h). * Move the private _PyErr_SetFromPyStatus() to the internal C API (pycore_initconfig.h). * No longer export the _PyErr_ChainExceptions() function. * Move run_in_subinterp_with_config() from _testcapi to _testinternalcapi.
* gh-106320: Remove private _PyDict functions (#108449)Victor Stinner2023-08-241-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | Move private functions to the internal C API (pycore_dict.h): * _PyDictView_Intersect() * _PyDictView_New() * _PyDict_ContainsId() * _PyDict_DelItemId() * _PyDict_DelItem_KnownHash() * _PyDict_GetItemIdWithError() * _PyDict_GetItem_KnownHash() * _PyDict_HasSplitTable() * _PyDict_NewPresized() * _PyDict_Next() * _PyDict_Pop() * _PyDict_SetItemId() * _PyDict_SetItem_KnownHash() * _PyDict_SizeOf() No longer export most of these functions. Move also the _PyDictViewObject structure to the internal C API. Move dict_getitem_knownhash() function from _testcapi to the _testinternalcapi extension. Update test_capi.test_dict for this change.
generated by cgit v0.12 at 2024-11-22 16:33:12 (GMT)