summaryrefslogtreecommitdiffstats
path: root/Include/cpython/pystate.h
Commit message (Collapse)AuthorAgeFilesLines
* [3.13] gh-120973: Fix thread-safety issues with `threading.local` ↵Miss Islington (bot)2024-07-191-0/+8
| | | | | | | | | | | (GH-121655) (#122042) This is a small refactoring to the current design that allows us to avoid manually iterating over threads. This should also fix gh-118490. (cherry picked from commit e059aa6b01030310125477c3ed1da0491020fe10) Co-authored-by: mpage <mpage@meta.com>
* [3.13] gh-121621: Move asyncio_running_loop to private struct (GH-121939) ↵Miss Islington (bot)2024-07-181-2/+0
| | | | | | | | | | (#121943) gh-121621: Move asyncio_running_loop to private struct (GH-121939) This avoids changing the ABI and keeps the field in the private struct. (cherry picked from commit 81fd625b5c30cc6f417c93bad404923676ad8ca3) Co-authored-by: Sam Gross <colesbury@gmail.com>
* [3.13] gh-121621: Move asyncio running loop to thread state (GH-121695) ↵Miss Islington (bot)2024-07-161-0/+2
| | | | | | | | (GH-121864) gh-121621: Move asyncio running loop to thread state (GH-121695) (cherry picked from commit 69c68de43aef03dd52fabd21f99cb3b0f9329201) Co-authored-by: Ken Jin <kenjin@python.org>
* [3.13] gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121013)Miss Islington (bot)2024-06-261-4/+5
| | | | | | | | | We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence(). This change only affects internal API. (cherry picked from commit a905721b9c5c15279e67c2f7785034b7356b2d46, AKA gh-121010) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.13] gh-118727: Don't drop the GIL in `drop_gil()` unless the current ↵Miss Islington (bot)2024-05-231-1/+3
| | | | | | | | | | | | | | | | | | | | thread holds it (GH-118745) (#119474) `drop_gil()` assumes that its caller is attached, which means that the current thread holds the GIL if and only if the GIL is enabled, and the enabled-state of the GIL won't change. This isn't true, though, because `detach_thread()` calls `_PyEval_ReleaseLock()` after detaching and `_PyThreadState_DeleteCurrent()` calls it after removing the current thread from consideration for stop-the-world requests (effectively detaching it). Fix this by remembering whether or not a thread acquired the GIL when it last attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()` instead of `gil->enabled`. This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've reenabled it. (cherry picked from commit be1dfccdf2c5c7671b8a549e969b8cf7d60d9936) Co-authored-by: Brett Simmers <swtaarrs@users.noreply.github.com>
* gh-112075: use per-thread dict version pool (#118676)Dino Viehland2024-05-071-0/+1
| | | use thread state set of dict versions
* GH-117760: Streamline the trashcan mechanism (GH-117763)Mark Shannon2024-04-171-6/+1
|
* gh-117645: Increase WASI stack size from 512 KiB to 8 MiB (#117674)Victor Stinner2024-04-161-8/+3
| | | | | Increase also the initial memory from 10 MiB to 20 MiB. Reenable test_dynamic on WASI build.
* gh-117008: Fix functools test_recursive_pickle() (#117009)Victor Stinner2024-03-231-0/+2
| | | | | | | Use support.infinite_recursion() in test_recursive_pickle() of test_functools to prevent a stack overflow on "ARM64 Windows Non-Debug" buildbot. Lower Py_C_RECURSION_LIMIT to 1,000 frames on Windows ARM64.
* gh-71052: Add Android build script and instructions (#116426)Malcolm Smith2024-03-211-0/+4
|
* gh-114271: Fix race in `Thread.join()` (#114839)mpage2024-03-161-26/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()` and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution involving threads A, B, and C: 1. A starts. 2. B joins A, blocking on its `_tstate_lock`. 3. C joins A, blocking on its `_tstate_lock`. 4. A finishes and releases its `_tstate_lock`. 5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped out before calling `_stop()`. 6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped out before releasing it. 7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held. However, C holds it, so the assertion fails. The race can be reproduced[^3] by inserting sleeps at the appropriate points in the threading code. To do so, run the `repro_join_race.py` from the linked repo. There are two main parts to this PR: 1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`. The event is set by the runtime prior to the thread being cleared (in the same place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the event to be set. 2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all non-daemon threads to exit. To do so, an `is_daemon` predicate was added to `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()` now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on `_tstate_lock`s. [^1]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1201 [^2]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1115 [^3]: https://github.com/mpage/cpython/commit/81946532792f938cd6f6ab4c4ff92a4edf61314f --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Antoine Pitrou <antoine@python.org>
* gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)Brett Simmers2024-02-201-0/+5
| | | | | | | | | | | This change adds an `eval_breaker` field to `PyThreadState`. The primary motivation is for performance in free-threaded builds: with thread-local eval breakers, we can stop a specific thread (e.g., for an async exception) without interrupting other threads. The source of truth for the global instrumentation version is stored in the `instrumentation_version` field in PyInterpreterState. Threads usually read the version from their local `eval_breaker`, where it continues to be colocated with the eval breaker bits.
* GH-112354: Initial implementation of warm up on exits and trace-stitching ↵Mark Shannon2024-02-201-0/+2
| | | | (GH-114142)
* GH-113655 Lower C recursion limit from 4000 to 3000 on Windows. (GH-114896)Mark Shannon2024-02-021-1/+1
|
* GH-114456: lower the recursion limit under WASI for debug builds (GH-114457)Brett Cannon2024-01-231-4/+7
| | | Testing under wasmtime 16.0.0 w/ code from https://github.com/python/cpython/issues/114413 is how the value was found.
* gh-111964: Implement stop-the-world pauses (gh-112471)Sam Gross2024-01-231-1/+1
| | | | | | | | | | | | | | | | | The `--disable-gil` builds occasionally need to pause all but one thread. Some examples include: * Cyclic garbage collection, where this is often called a "stop the world event" * Before calling `fork()`, to ensure a consistent state for internal data structures * During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects This adds the following functions to implement global and per-interpreter pauses: * `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime) * `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter) (The function names may change.) These functions are no-ops outside of the `--disable-gil` build.
* GH-113655: Lower the C recursion limit on various platforms (GH-113944)Mark Shannon2024-01-161-2/+6
|
* GH-112215: Increase C recursion limit for non debug builds (GH-113397)Mark Shannon2023-12-221-1/+3
|
* gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864)Hugo van Kemenade2023-11-201-1/+1
| | | Rename Py_NOGIL to Py_GIL_DISABLED
* gh-111798: Use lower Py_C_RECURSION_LIMIT in debug mode (#112124)Victor Stinner2023-11-161-1/+5
| | | | | | | * Run again test_ast_recursion_limit() on WASI platform. * Add _testinternalcapi.get_c_recursion_remaining(). * Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and test_trace_unpack_long_sequence() now adjust the maximum recursion depth depending on the the remaining C recursion.
* gh-112026: Add again _PyThreadState_UncheckedGet() function (#112121)Victor Stinner2023-11-151-0/+3
| | | | Add again the private _PyThreadState_UncheckedGet() function as an alias to the new public PyThreadState_GetUnchecked() function.
* gh-112026: Restore removed private C API (#112115)Victor Stinner2023-11-151-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Restore removed private C API functions, macros and structures which have no simple replacement for now: * _PyDict_GetItem_KnownHash() * _PyDict_NewPresized() * _PyHASH_BITS * _PyHASH_IMAG * _PyHASH_INF * _PyHASH_MODULUS * _PyHASH_MULTIPLIER * _PyLong_Copy() * _PyLong_FromDigits() * _PyLong_New() * _PyLong_Sign() * _PyObject_CallMethodId() * _PyObject_CallMethodNoArgs() * _PyObject_CallMethodOneArg() * _PyObject_CallOneArg() * _PyObject_EXTRA_INIT * _PyObject_FastCallDict() * _PyObject_GetAttrId() * _PyObject_Vectorcall() * _PyObject_VectorcallMethod() * _PyStack_AsDict() * _PyThread_CurrentFrames() * _PyUnicodeWriter structure * _PyUnicodeWriter_Dealloc() * _PyUnicodeWriter_Finish() * _PyUnicodeWriter_Init() * _PyUnicodeWriter_Prepare() * _PyUnicodeWriter_PrepareKind() * _PyUnicodeWriter_WriteASCIIString() * _PyUnicodeWriter_WriteChar() * _PyUnicodeWriter_WriteLatin1String() * _PyUnicodeWriter_WriteStr() * _PyUnicodeWriter_WriteSubstring() * _PyUnicode_AsString() * _PyUnicode_FromId() * _PyVectorcall_Function() * _Py_HashDouble() * _Py_HashPointer() * _Py_IDENTIFIER() * _Py_c_abs() * _Py_c_diff() * _Py_c_neg() * _Py_c_pow() * _Py_c_prod() * _Py_c_quot() * _Py_c_sum() * _Py_static_string() * _Py_static_string_init()
* gh-111569: Implement Python critical section API (gh-111571)Sam Gross2023-11-081-0/+7
| | | | | | | | Critical sections are helpers to replace the global interpreter lock with finer grained locking. They provide similar guarantees to the GIL and avoid the deadlock risk that plain locking involves. Critical sections are implicitly ended whenever the GIL would be released. They are resumed when the GIL would be acquired. Nested critical sections behave as if the sections were interleaved.
* gh-76785: Move the Cross-Interpreter Code to Its Own File (gh-111502)Eric Snow2023-10-301-77/+0
| | | This is partly to clear this stuff out of pystate.c, but also in preparation for moving some code out of _xxsubinterpretersmodule.c. This change also moves this stuff to the internal API (new: Include/internal/pycore_crossinterp.h). @vstinner did this previously and I undid it. Now I'm re-doing it. :/
* gh-76785: Clean Up the Channels Module (gh-110568)Eric Snow2023-10-171-1/+1
|
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-051-0/+4
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* gh-108867: Add PyThreadState_GetUnchecked() function (#108870)Victor Stinner2023-10-031-1/+1
| | | | | | Add PyThreadState_GetUnchecked() function: similar to PyThreadState_Get(), but don't issue a fatal error if it is NULL. The caller is responsible to check if the result is NULL. Previously, this function was private and known as _PyThreadState_UncheckedGet().
* gh-109860: Use a New Thread State When Switching Interpreters, When ↵Eric Snow2023-10-031-0/+9
| | | | | | | | | Necessary (gh-110245) In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread. For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()). There was a good chance that tstate wasn't actually in use by another thread. However, there are no guarantees of that. Furthermore, re-using an already used tstate is currently fragile. To address this, now we create a new thread state in each of those places and use it. One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
* gh-105716: Support Background Threads in Subinterpreters Consistently ↵Eric Snow2023-10-021-0/+1
| | | | | | | (gh-109921) The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish. We add PyInterpreterState.threads.main, with some internal C-API functions.
* gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)Eric Snow2023-09-191-0/+1
| | | This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
* GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)Victor Stinner2023-09-081-10/+9
| | | | | | | Symbols of the C API should be prefixed by "Py_" to avoid conflict with existing names in 3rd party C extensions on "#include <Python.h>". test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other _testcapi and _testinternalcapi constants.
* gh-106320: Remove private _PyGILState_GetInterpreterStateUnsafe() (#108603)Victor Stinner2023-08-291-9/+0
| | | | | The remove private _PyGILState_GetInterpreterStateUnsafe() function from the public C API: move it the internal C API (pycore_pystate.h). No longer export the function.
* GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for ↵Mark Shannon2023-08-171-23/+2
| | | | performance. (GH-108036)
* GH-107263: Increase C stack limit for most functions, except ↵Mark Shannon2023-08-041-1/+2
| | | | | | `_PyEval_EvalFrameDefault()` (GH-107535) * Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
* gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)Eric Snow2023-07-271-0/+1
| | | The _xxsubinterpreters module should not rely on internal API. Some of the functions it uses were recently moved there however. Here we move them back (and expose them properly).
* gh-106320: Remove _PyInterpreterState_HasFeature() (#106425)Victor Stinner2023-07-041-32/+0
| | | | | Remove the _PyInterpreterState_HasFeature() function from the C API: move it to the internal C API (pycore_interp.h). No longer export the function.
* gh-106320: Remove private _PyInterpreterState functions (#106325)Victor Stinner2023-07-021-57/+0
| | | | | | | | | Remove private _PyThreadState and _PyInterpreterState C API functions: move them to the internal C API (pycore_pystate.h and pycore_interp.h). Don't export most of these functions anymore, but still export functions used by tests. Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C API, but keep it in the stable API.
* gh-106320: Remove _PyInterpreterState_Get() alias (#106321)Victor Stinner2023-07-011-4/+1
| | | | Replace calls to the (removed) slow _PyInterpreterState_Get() with fast inlined _PyInterpreterState_GET() function.
* gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)Eric Snow2023-04-241-0/+4
| | | | | | | | | | | | | | This is strictly about moving the "obmalloc" runtime state from `_PyRuntimeState` to `PyInterpreterState`. Doing so improves isolation between interpreters, specifically most of the memory (incl. objects) allocated for each interpreter's use. This is important for a per-interpreter GIL, but such isolation is valuable even without it. FWIW, a per-interpreter obmalloc is the proverbial canary-in-the-coalmine when it comes to the isolation of objects between interpreters. Any object that leaks (unintentionally) to another interpreter is highly likely to cause a crash (on debug builds at least). That's a useful thing to know, relative to interpreter isolation.
* GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython ↵Mark Shannon2023-04-121-10/+1
| | | | | | | | | | (GH-103083) * The majority of the monitoring code is in instrumentation.c * The new instrumentation bytecodes are in bytecodes.c * legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
* gh-98627: Add an Optional Check for Extension Module Subinterpreter ↵Eric Snow2023-02-161-0/+3
| | | | | | | | | | | | | | | Compatibility (gh-99040) Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL. This change includes the following: * add tests for extension module subinterpreter compatibility * add _PyInterpreterConfig.check_multi_interp_extensions * add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS * add _PyImport_CheckSubinterpIncompatibleExtensionAllowed() * fail iff the module does not implement multi-phase init and the current interpreter is configured to check https://github.com/python/cpython/issues/98627
* GH-101578: Normalize the current exception (GH-101607)Mark Shannon2023-02-081-3/+1
| | | | | | | | | | * Make sure that the current exception is always normalized. * Remove redundant type and traceback fields for the current exception. * Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException * Add new API functions: PyException_GetArgs, PyException_SetArgs
* gh-59956: Clarify Runtime State Status Expectations (gh-101308)Eric Snow2023-01-301-1/+26
| | | | | | | | | | | | | A PyThreadState can be in one of many states in its lifecycle, represented by some status value. Those statuses haven't been particularly clear, so we're addressing that here. Specifically: * made the distinct lifecycle statuses clear on PyThreadState * identified expectations of how various lifecycle-related functions relate to status * noted the various places where those expectations don't match the actual behavior At some point we'll need to address the mismatches. (This change also includes some cleanup.) https://github.com/python/cpython/issues/59956
* gh-59956: Allow the "Trashcan" Mechanism to Work Without a Thread State ↵Eric Snow2023-01-231-2/+6
| | | | | | | | | | | | (gh-101209) We've factored out a struct from the two PyThreadState fields. This accomplishes two things: * make it clear that the trashcan-related code doesn't need any other parts of PyThreadState * allows us to use the trashcan mechanism even when there isn't a "current" thread state We still expect the caller to hold the GIL. https://github.com/python/cpython/issues/59956
* gh-59956: Clarify GILState-related Code (gh-101161)Eric Snow2023-01-191-5/+1
| | | | | | | | | | The objective of this change is to help make the GILState-related code easier to understand. This mostly involves moving code around and some semantically equivalent refactors. However, there are a also a small number of slight changes in structure and behavior: * tstate_current is moved out of _PyRuntimeState.gilstate * autoTSSkey is moved out of _PyRuntimeState.gilstate * autoTSSkey is initialized earlier * autoTSSkey is re-initialized (after fork) earlier https://github.com/python/cpython/issues/59956
* gh-99741: Implement Multi-Phase Init for the _xxsubinterpreters Module ↵Eric Snow2022-12-051-3/+18
| | | | | | | (gh-99742) _xxsubinterpreters is an internal module used for testing. https://github.com/python/cpython/issues/99741
* gh-99741: Clean Up the _xxsubinterpreters Module (gh-99940)Eric Snow2022-12-021-1/+1
| | | | | This cleanup up resolves a few subtle bugs and makes the implementation for multi-phase init much cleaner. https://github.com/python/cpython/issues/99741
* gh-99741: Fix the Cross-Interpreter Data API (gh-99939)Eric Snow2022-12-021-0/+1
| | | | | There were some minor issues that showed up while I was working on porting _xxsubinterpreters to multi-phase init. This fixes them. https://github.com/python/cpython/issues/99741
* GH-99205: remove `_static` field from `PyThreadState` and ↵Kumar Aditya2022-11-151-3/+0
| | | | `PyInterpreterState` (GH-99385)
* gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)Eric Snow2022-10-311-6/+7
| | | | | | | Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads. By default, we were disallowing all three for "isolated" interpreters. We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API. Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads. The default for "isolated" interpreters disables fork, exec, and daemon threads. Regular threads are allowed by default. We continue always allowing everything For the main interpreter and the legacy API. In the code, we add `_PyInterpreterConfig.allow_exec` and `_PyInterpreterConfig.allow_daemon_threads`. We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
generated by cgit v0.12 at 2026-01-04 01:14:04 (GMT)