summaryrefslogtreecommitdiffstats
path: root/Include/cpython/pystate.h
Commit message (Collapse)AuthorAgeFilesLines
* [3.14] gh-139653: Add PyUnstable_ThreadState_SetStackProtection() ↵Petr Viktorin2025-11-251-0/+12
| | | | | | | (GH-139668) (#141661) Co-authored-by: Rok Mandeljc <rok.mandeljc@gmail.com> Co-authored-by: Mark Shannon <mark@hotpy.org> Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.14] gh-135755: Make Py_MAX_SCRIPT_PATH_SIZE private (GH-138350) (#138356)Miss Islington (bot)2025-09-161-2/+2
| | | | Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.14] gh-131591: Add Py_ prefix to MAX_SCRIPT_PATH_SIZE; remove unprefixed ↵Miss Islington (bot)2025-06-261-3/+3
| | | | | | | | | struct tag (GH-135924) (GH-135969) Names/macros defined in public headers should have `Py`/`_Py` prefixes. (cherry picked from commit a1da208eec3028b1ecae804d4c0dc6b43cdddae9) Co-authored-by: Petr Viktorin <encukou@gmail.com>
* [3.14] gh-91048: Refactor and optimize remote debugging module (#134652) ↵Pablo Galindo Salgado2025-05-251-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#134673) gh-91048: Refactor and optimize remote debugging module (#134652) Completely refactor Modules/_remote_debugging_module.c with improved code organization, replacing scattered reference counting and error handling with centralized goto error paths. This cleanup improves maintainability and reduces code duplication throughout the module while preserving the same external API. Implement memory page caching optimization in Python/remote_debug.h to avoid repeated reads of the same memory regions during debugging operations. The cache stores previously read memory pages and reuses them for subsequent reads, significantly reducing system calls and improving performance. Add code object caching mechanism with a new code_object_generation field in the interpreter state that tracks when code object caches need invalidation. This allows efficient reuse of parsed code object metadata and eliminates redundant processing of the same code objects across debugging sessions. Optimize memory operations by replacing multiple individual structure copies with single bulk reads for the same data structures. This reduces the number of memory operations and system calls required to gather debugging information from the target process. Update Makefile.pre.in to include Python/remote_debug.h in the headers list, ensuring that changes to the remote debugging header force proper recompilation of dependent modules and maintain build consistency across the codebase. Also, make the module compatible with the free threading build as an extra :) Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit 42b25ad4d3d6bcdc28ddfe07d2bf8831378bb0d1)
* GH-133231: Changes to executor management to support proposed `sys._jit` ↵Mark Shannon2025-05-041-1/+1
| | | | | | | | module (GH-133287) * Track the current executor, not the previous one, on the thread-state. * Batch executors for deallocation to avoid having to constantly incref executors; this is an ad-hoc form of deferred reference counting.
* gh-133079: Remove Py_C_RECURSION_LIMIT & PyThreadState.c_recursion_remaining ↵Petr Viktorin2025-04-291-4/+0
| | | | | | | | (GH-133080) Both were added in 3.13, are undocumented, and don't make sense in 3.14 due to changes in the stack overflow detection machinery (gh-112282). PEP 387 exception for skipping a deprecation period: https://github.com/python/steering-council/issues/288
* gh-132775: Drop PyUnstable_InterpreterState_GetMainModule() (gh-132978)Eric Snow2025-04-281-2/+0
| | | | | We replace it with _Py_GetMainModule(), and add _Py_CheckMainModule(), but both in the internal-only C-API. We also add _PyImport_GetModulesRef(), which is the equivalent of _PyImport_GetModules(), but which increfs before the lock is released. This is used by a later change related to pickle and handling __main__.
* gh-131591: Implement PEP 768 (#131937)Pablo Galindo Salgado2025-04-031-0/+8
| | | | Co-authored-by: Ivona Stojanovic <stojanovic.i@hotmail.com> Co-authored-by: Matt Wozniski <godlygeek@gmail.com>
* gh-124878: Fix race conditions during interpreter finalization (#130649)Sam Gross2025-03-061-3/+5
| | | | | | | | | | | | | | | | | | | | The PyThreadState field gains a reference count field to avoid issues with PyThreadState being a dangling pointer to freed memory. The refcount starts with a value of two: one reference is owned by the interpreter's linked list of thread states and one reference is owned by the OS thread. The reference count is decremented when the thread state is removed from the interpreter's linked list and before the OS thread calls `PyThread_hang_thread()`. The thread that decrements it to zero frees the `PyThreadState` memory. The `holds_gil` field is moved out of the `_status` bit field, to avoid a data race where on thread calls `PyThreadState_Clear()`, modifying the `_status` bit field while the OS thread reads `holds_gil` when attempting to acquire the GIL. The `PyThreadState.state` field now has `_Py_THREAD_SHUTTING_DOWN` as a possible value. This corresponds to the `_PyThreadState_MustExit()` check. This avoids race conditions in the free threading build when checking `_PyThreadState_MustExit()`.
* GH-130396: Use computed stack limits on linux (GH-130398)Mark Shannon2025-02-251-32/+2
| | | | | | | | | | | * Implement C recursion protection with limit pointers for Linux, MacOS and Windows * Remove calls to PyOS_CheckStack * Add stack protection to parser * Make tests more robust to low stacks * Improve error messages for stack overflow
* GH-91079: Revert "GH-91079: Implement C stack limits using addresses, not ↵Petr Viktorin2025-02-241-2/+32
| | | | | | | | | counters. (GH-130007)" for now (GH130413) Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now Unfortunatlely, the change broke some buildbots. This reverts commit 2498c22fa0a2b560491bc503fa676585c1a603d0.
* GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)Mark Shannon2025-02-191-32/+2
| | | | | | | | | | | | * Implement C recursion protection with limit pointers * Remove calls to PyOS_CheckStack * Add stack protection to parser * Make tests more robust to low stacks * Improve error messages for stack overflow
* gh-128863: Deprecate private C API functions (#128864)Victor Stinner2025-01-221-2/+6
| | | | | | | | | | | | | | | Deprecate private C API functions: * _PyBytes_Join() * _PyDict_GetItemStringWithError() * _PyDict_Pop() * _PyThreadState_UncheckedGet() * _PyUnicode_AsString() * _Py_HashPointer() * _Py_fopen_obj() Replace _Py_HashPointer() with Py_HashPointer(). Remove references to deprecated functions.
* GH-113655: Lower the C recursion limit for HPPA, PPC64 and SPARC (#124264)Sam James2024-09-231-0/+6
| | | | | | | | | Lower the C recursion limit for HPPA, PPC64 and SPARC, as they use relatively large stack frames that cause e.g. `test_descr` to hit a stack overflow. According to quick testing, it seems that values around 8000 are max for HPPA and PPC64 (ELFv1 ABI) and 7000 for SPARC64. To keep things safe, let's use 5000 for PPC64 and 4000 for SPARC. Co-authored-by: Michał Górny <mgorny@gentoo.org>
* gh-120973: Fix thread-safety issues with `threading.local` (#121655)mpage2024-07-191-0/+8
| | | | | | This is a small refactoring to the current design that allows us to avoid manually iterating over threads. This should also fix gh-118490.
* gh-121621: Move asyncio_running_loop to private struct (#121939)Sam Gross2024-07-171-2/+0
| | | | This avoids changing the ABI and keeps the field in the private struct.
* gh-121621: Move asyncio running loop to thread state (GH-121695)Ken Jin2024-07-161-0/+2
|
* gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121010)Eric Snow2024-06-251-4/+5
| | | | | We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence(). This change only affects internal API.
* gh-118727: Don't drop the GIL in `drop_gil()` unless the current thread ↵Brett Simmers2024-05-231-1/+3
| | | | | | | | | | | | | | | | | holds it (#118745) `drop_gil()` assumes that its caller is attached, which means that the current thread holds the GIL if and only if the GIL is enabled, and the enabled-state of the GIL won't change. This isn't true, though, because `detach_thread()` calls `_PyEval_ReleaseLock()` after detaching and `_PyThreadState_DeleteCurrent()` calls it after removing the current thread from consideration for stop-the-world requests (effectively detaching it). Fix this by remembering whether or not a thread acquired the GIL when it last attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()` instead of `gil->enabled`. This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've reenabled it.
* gh-112075: use per-thread dict version pool (#118676)Dino Viehland2024-05-071-0/+1
| | | use thread state set of dict versions
* GH-117760: Streamline the trashcan mechanism (GH-117763)Mark Shannon2024-04-171-6/+1
|
* gh-117645: Increase WASI stack size from 512 KiB to 8 MiB (#117674)Victor Stinner2024-04-161-8/+3
| | | | | Increase also the initial memory from 10 MiB to 20 MiB. Reenable test_dynamic on WASI build.
* gh-117008: Fix functools test_recursive_pickle() (#117009)Victor Stinner2024-03-231-0/+2
| | | | | | | Use support.infinite_recursion() in test_recursive_pickle() of test_functools to prevent a stack overflow on "ARM64 Windows Non-Debug" buildbot. Lower Py_C_RECURSION_LIMIT to 1,000 frames on Windows ARM64.
* gh-71052: Add Android build script and instructions (#116426)Malcolm Smith2024-03-211-0/+4
|
* gh-114271: Fix race in `Thread.join()` (#114839)mpage2024-03-161-26/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()` and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution involving threads A, B, and C: 1. A starts. 2. B joins A, blocking on its `_tstate_lock`. 3. C joins A, blocking on its `_tstate_lock`. 4. A finishes and releases its `_tstate_lock`. 5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped out before calling `_stop()`. 6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped out before releasing it. 7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held. However, C holds it, so the assertion fails. The race can be reproduced[^3] by inserting sleeps at the appropriate points in the threading code. To do so, run the `repro_join_race.py` from the linked repo. There are two main parts to this PR: 1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`. The event is set by the runtime prior to the thread being cleared (in the same place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the event to be set. 2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all non-daemon threads to exit. To do so, an `is_daemon` predicate was added to `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()` now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on `_tstate_lock`s. [^1]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1201 [^2]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1115 [^3]: https://github.com/mpage/cpython/commit/81946532792f938cd6f6ab4c4ff92a4edf61314f --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Antoine Pitrou <antoine@python.org>
* gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)Brett Simmers2024-02-201-0/+5
| | | | | | | | | | | This change adds an `eval_breaker` field to `PyThreadState`. The primary motivation is for performance in free-threaded builds: with thread-local eval breakers, we can stop a specific thread (e.g., for an async exception) without interrupting other threads. The source of truth for the global instrumentation version is stored in the `instrumentation_version` field in PyInterpreterState. Threads usually read the version from their local `eval_breaker`, where it continues to be colocated with the eval breaker bits.
* GH-112354: Initial implementation of warm up on exits and trace-stitching ↵Mark Shannon2024-02-201-0/+2
| | | | (GH-114142)
* GH-113655 Lower C recursion limit from 4000 to 3000 on Windows. (GH-114896)Mark Shannon2024-02-021-1/+1
|
* GH-114456: lower the recursion limit under WASI for debug builds (GH-114457)Brett Cannon2024-01-231-4/+7
| | | Testing under wasmtime 16.0.0 w/ code from https://github.com/python/cpython/issues/114413 is how the value was found.
* gh-111964: Implement stop-the-world pauses (gh-112471)Sam Gross2024-01-231-1/+1
| | | | | | | | | | | | | | | | | The `--disable-gil` builds occasionally need to pause all but one thread. Some examples include: * Cyclic garbage collection, where this is often called a "stop the world event" * Before calling `fork()`, to ensure a consistent state for internal data structures * During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects This adds the following functions to implement global and per-interpreter pauses: * `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime) * `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter) (The function names may change.) These functions are no-ops outside of the `--disable-gil` build.
* GH-113655: Lower the C recursion limit on various platforms (GH-113944)Mark Shannon2024-01-161-2/+6
|
* GH-112215: Increase C recursion limit for non debug builds (GH-113397)Mark Shannon2023-12-221-1/+3
|
* gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864)Hugo van Kemenade2023-11-201-1/+1
| | | Rename Py_NOGIL to Py_GIL_DISABLED
* gh-111798: Use lower Py_C_RECURSION_LIMIT in debug mode (#112124)Victor Stinner2023-11-161-1/+5
| | | | | | | * Run again test_ast_recursion_limit() on WASI platform. * Add _testinternalcapi.get_c_recursion_remaining(). * Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and test_trace_unpack_long_sequence() now adjust the maximum recursion depth depending on the the remaining C recursion.
* gh-112026: Add again _PyThreadState_UncheckedGet() function (#112121)Victor Stinner2023-11-151-0/+3
| | | | Add again the private _PyThreadState_UncheckedGet() function as an alias to the new public PyThreadState_GetUnchecked() function.
* gh-112026: Restore removed private C API (#112115)Victor Stinner2023-11-151-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Restore removed private C API functions, macros and structures which have no simple replacement for now: * _PyDict_GetItem_KnownHash() * _PyDict_NewPresized() * _PyHASH_BITS * _PyHASH_IMAG * _PyHASH_INF * _PyHASH_MODULUS * _PyHASH_MULTIPLIER * _PyLong_Copy() * _PyLong_FromDigits() * _PyLong_New() * _PyLong_Sign() * _PyObject_CallMethodId() * _PyObject_CallMethodNoArgs() * _PyObject_CallMethodOneArg() * _PyObject_CallOneArg() * _PyObject_EXTRA_INIT * _PyObject_FastCallDict() * _PyObject_GetAttrId() * _PyObject_Vectorcall() * _PyObject_VectorcallMethod() * _PyStack_AsDict() * _PyThread_CurrentFrames() * _PyUnicodeWriter structure * _PyUnicodeWriter_Dealloc() * _PyUnicodeWriter_Finish() * _PyUnicodeWriter_Init() * _PyUnicodeWriter_Prepare() * _PyUnicodeWriter_PrepareKind() * _PyUnicodeWriter_WriteASCIIString() * _PyUnicodeWriter_WriteChar() * _PyUnicodeWriter_WriteLatin1String() * _PyUnicodeWriter_WriteStr() * _PyUnicodeWriter_WriteSubstring() * _PyUnicode_AsString() * _PyUnicode_FromId() * _PyVectorcall_Function() * _Py_HashDouble() * _Py_HashPointer() * _Py_IDENTIFIER() * _Py_c_abs() * _Py_c_diff() * _Py_c_neg() * _Py_c_pow() * _Py_c_prod() * _Py_c_quot() * _Py_c_sum() * _Py_static_string() * _Py_static_string_init()
* gh-111569: Implement Python critical section API (gh-111571)Sam Gross2023-11-081-0/+7
| | | | | | | | Critical sections are helpers to replace the global interpreter lock with finer grained locking. They provide similar guarantees to the GIL and avoid the deadlock risk that plain locking involves. Critical sections are implicitly ended whenever the GIL would be released. They are resumed when the GIL would be acquired. Nested critical sections behave as if the sections were interleaved.
* gh-76785: Move the Cross-Interpreter Code to Its Own File (gh-111502)Eric Snow2023-10-301-77/+0
| | | This is partly to clear this stuff out of pystate.c, but also in preparation for moving some code out of _xxsubinterpretersmodule.c. This change also moves this stuff to the internal API (new: Include/internal/pycore_crossinterp.h). @vstinner did this previously and I undid it. Now I'm re-doing it. :/
* gh-76785: Clean Up the Channels Module (gh-110568)Eric Snow2023-10-171-1/+1
|
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-051-0/+4
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* gh-108867: Add PyThreadState_GetUnchecked() function (#108870)Victor Stinner2023-10-031-1/+1
| | | | | | Add PyThreadState_GetUnchecked() function: similar to PyThreadState_Get(), but don't issue a fatal error if it is NULL. The caller is responsible to check if the result is NULL. Previously, this function was private and known as _PyThreadState_UncheckedGet().
* gh-109860: Use a New Thread State When Switching Interpreters, When ↵Eric Snow2023-10-031-0/+9
| | | | | | | | | Necessary (gh-110245) In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread. For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()). There was a good chance that tstate wasn't actually in use by another thread. However, there are no guarantees of that. Furthermore, re-using an already used tstate is currently fragile. To address this, now we create a new thread state in each of those places and use it. One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
* gh-105716: Support Background Threads in Subinterpreters Consistently ↵Eric Snow2023-10-021-0/+1
| | | | | | | (gh-109921) The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish. We add PyInterpreterState.threads.main, with some internal C-API functions.
* gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)Eric Snow2023-09-191-0/+1
| | | This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
* GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)Victor Stinner2023-09-081-10/+9
| | | | | | | Symbols of the C API should be prefixed by "Py_" to avoid conflict with existing names in 3rd party C extensions on "#include <Python.h>". test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other _testcapi and _testinternalcapi constants.
* gh-106320: Remove private _PyGILState_GetInterpreterStateUnsafe() (#108603)Victor Stinner2023-08-291-9/+0
| | | | | The remove private _PyGILState_GetInterpreterStateUnsafe() function from the public C API: move it the internal C API (pycore_pystate.h). No longer export the function.
* GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for ↵Mark Shannon2023-08-171-23/+2
| | | | performance. (GH-108036)
* GH-107263: Increase C stack limit for most functions, except ↵Mark Shannon2023-08-041-1/+2
| | | | | | `_PyEval_EvalFrameDefault()` (GH-107535) * Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
* gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)Eric Snow2023-07-271-0/+1
| | | The _xxsubinterpreters module should not rely on internal API. Some of the functions it uses were recently moved there however. Here we move them back (and expose them properly).
* gh-106320: Remove _PyInterpreterState_HasFeature() (#106425)Victor Stinner2023-07-041-32/+0
| | | | | Remove the _PyInterpreterState_HasFeature() function from the C API: move it to the internal C API (pycore_interp.h). No longer export the function.
generated by cgit v0.12 at 2026-04-03 19:35:40 (GMT)