summaryrefslogtreecommitdiffstats
path: root/Include/internal/pycore_runtime.h
Commit message (Collapse)AuthorAgeFilesLines
* [3.12] gh-110310: Add a Per-Interpreter XID Registry for Heap Types ↵Eric Snow2023-11-281-4/+1
| | | | | | | | | | | | | | | (gh-110311) (gh-110714) We do the following: * add a per-interpreter XID registry (PyInterpreterState.xidregistry) * put heap types there (keep static types in _PyRuntimeState.xidregistry) * clear the registries during interpreter/runtime finalization * avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type) * use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup() The per-interpreter registry helps preserve isolation between interpreters. This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon. (cherry-picked from commit 80dc39e1dc2abc809f448cba5d2c5b9c1c631e11)
* [3.12] gh-109793: Allow Switching Interpreters During Finalization ↵Eric Snow2023-11-281-0/+17
| | | | | (gh-109794) (gh-110705) Essentially, we should check the thread ID rather than the thread state pointer.
* [3.12] gh-106931: Intern Statically Allocated Strings Globally (gh-107272) ↵Eric Snow2023-11-271-0/+1
| | | | | | | | | (gh-110713) We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict. Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter. (cherry-picked from commit b72947a8d26915156323ccfd04d273199ecb870c)
* [3.12] gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994) (gh-110701)Eric Snow2023-11-271-0/+5
| | | | | | | This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This change does not address the broader concerns from gh-109853. (cherry-picked from commit a040a32ea2f13f16172394d3e3e3f80f47f25a68)
* [3.12] gh-106140: Reorder some fields to facilitate out-of-process ↵Miss Islington (bot)2023-06-271-8/+14
| | | | | | | | | inspection (GH-106143) (#106147) gh-106140: Reorder some fields to facilitate out-of-process inspection (GH-106143) (cherry picked from commit 2d5a1c281161d037148ffb5983decc6d31c2557d) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-100227: Lock Around Adding Global Audit Hooks (gh-105515) (gh-105525)Eric Snow2023-06-081-1/+4
| | | | The risk of a race with this state is relatively low, but we play it safe anyway. (cherry picked from commit e822a676f1f3bef6c5413e9b856db481c08ac2a5)
* gh-99113: A Per-Interpreter GIL! (gh-104210)Eric Snow2023-05-081-2/+0
| | | | | This is the culmination of PEP 684 (and of my 8-year long multi-core Python project)! Each subinterpreter may now be created with its own GIL (via Py_NewInterpreterFromConfig()). If not so configured then the interpreter will share with the main interpreter--the status quo since subinterpreters were added decades ago. The main interpreter always has its own GIL and subinterpreters from Py_NewInterpreter() will always share with the main interpreter.
* gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)Eric Snow2023-04-241-2/+1
| | | | | | | | | | | | | | This is strictly about moving the "obmalloc" runtime state from `_PyRuntimeState` to `PyInterpreterState`. Doing so improves isolation between interpreters, specifically most of the memory (incl. objects) allocated for each interpreter's use. This is important for a per-interpreter GIL, but such isolation is valuable even without it. FWIW, a per-interpreter obmalloc is the proverbial canary-in-the-coalmine when it comes to the isolation of objects between interpreters. Any object that leaks (unintentionally) to another interpreter is highly likely to cause a crash (on debug builds at least). That's a useful thing to know, relative to interpreter isolation.
* gh-95795: Move types.next_version_tag to PyInterpreterState (gh-102343)Eric Snow2023-04-241-7/+2
| | | | | | Core static types will continue to use the global value. All other types will use the per-interpreter value. They all share the same range, where the global types use values < 2^16 and each interpreter uses values higher than that.
* gh-103323: Get the "Current" Thread State from a Thread-Local Variable ↵Eric Snow2023-04-241-3/+0
| | | | | | | | | | | (gh-103324) We replace _PyRuntime.tstate_current with a thread-local variable. As part of this change, we add a _Py_thread_local macro in pyport.h (only for the core runtime) to smooth out the compiler differences. The main motivation here is in support of a per-interpreter GIL, but this change also provides some performance improvement opportunities. Note that we do not provide a fallback to the thread-local, either falling back to the old tstate_current or to thread-specific storage (PyThread_tss_*()). If that proves problematic then we can circle back. I consider it unlikely, but will run the buildbots to double-check. Also note that this does not change any of the code related to the GILState API, where it uses a thread state stored in thread-specific storage. I suspect we can combine that with _Py_tss_tstate (from here). However, that can be addressed separately and is not urgent (nor critical). (While this change was mostly done independently, I did take some inspiration from earlier (~2020) work by @markshannon (main...markshannon:threadstate_in_tls) and @vstinner (#23976).)
* gh-101659: Add _Py_AtExit() (gh-103298)Eric Snow2023-04-061-3/+2
| | | | | The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on gh-101660.)
* gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339)Eric Snow2023-03-281-1/+0
| | | | | We can revisit the options for keeping it global later, if desired. For now the approach seems quite complex, so we've gone with the simpler isolation solution in the meantime. https://github.com/python/cpython/issues/100227
* gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)Eric Snow2023-03-201-0/+2
| | | | | | | The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState. Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead. https://github.com/python/cpython/issues/102304
* gh-100227: Move dict_state.global_version to PyInterpreterState (gh-102338)Eric Snow2023-03-091-2/+0
| | | https://github.com/python/cpython/issues/100227
* gh-100227: Move func_state.next_version to PyInterpreterState (gh-102334)Eric Snow2023-03-081-2/+0
| | | https://github.com/python/cpython/issues/100227
* gh-100227: Move the dtoa State to PyInterpreterState (gh-102331)Eric Snow2023-02-281-2/+0
| | | https://github.com/python/cpython/issues/100227
* gh-59956: Allow the "Trashcan" Mechanism to Work Without a Thread State ↵Eric Snow2023-01-231-0/+3
| | | | | | | | | | | | (gh-101209) We've factored out a struct from the two PyThreadState fields. This accomplishes two things: * make it clear that the trashcan-related code doesn't need any other parts of PyThreadState * allows us to use the trashcan mechanism even when there isn't a "current" thread state We still expect the caller to hold the GIL. https://github.com/python/cpython/issues/59956
* gh-59956: Clarify GILState-related Code (gh-101161)Eric Snow2023-01-191-4/+6
| | | | | | | | | | The objective of this change is to help make the GILState-related code easier to understand. This mostly involves moving code around and some semantically equivalent refactors. However, there are a also a small number of slight changes in structure and behavior: * tstate_current is moved out of _PyRuntimeState.gilstate * autoTSSkey is moved out of _PyRuntimeState.gilstate * autoTSSkey is initialized earlier * autoTSSkey is re-initialized (after fork) earlier https://github.com/python/cpython/issues/59956
* gh-90111: Minor Cleanup for Runtime-Global Objects (gh-100254)Eric Snow2022-12-141-1/+1
| | | | | | | | * move _PyRuntime.global_objects.interned to _PyRuntime.cached_objects.interned_strings (and use _Py_CACHED_OBJECT()) * rename _PyRuntime.global_objects to _PyRuntime.static_objects (This also relates to gh-96075.) https://github.com/python/cpython/issues/90111
* gh-81057: Move Signal-Related Globals to _PyRuntimeState (gh-100085)Eric Snow2022-12-121-5/+2
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move faulthandler Globals to _PyRuntimeState (gh-100152)Eric Snow2022-12-121-0/+2
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move tracemalloc Globals to _PyRuntimeState (gh-100151)Eric Snow2022-12-121-3/+2
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move Threading-Related Globals to _PyRuntimeState (#100084)Eric Snow2022-12-091-0/+2
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move Ceval Trampoline Globals to _PyRuntimeState (gh-100083)Eric Snow2022-12-091-10/+1
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move time Globals to _PyRuntimeState (gh-100122)Eric Snow2022-12-081-2/+2
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move OS-Related Globals to _PyRuntimeState (gh-100082)Eric Snow2022-12-081-0/+2
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move More Globals to _PyRuntimeState (gh-100092)Eric Snow2022-12-071-0/+5
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move the global Dict-Related Versions to _PyRuntimeState (gh-99497)Eric Snow2022-11-161-0/+4
| | | | | We also move the global func version. https://github.com/python/cpython/issues/81057
* gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)Eric Snow2022-11-161-1/+11
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)Eric Snow2022-11-151-1/+7
| | | | | This is the first of several changes to consolidate non-object globals in core code. https://github.com/python/cpython/issues/81057
* gh-81057: Move Global Variables Holding Objects to _PyRuntimeState. (gh-99487)Eric Snow2022-11-141-0/+8
| | | | | This moves nearly all remaining object-holding globals in core code (other than static types). https://github.com/python/cpython/issues/81057
* gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)Eric Snow2022-11-111-0/+5
| | | | | The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject. State for the "small block" allocator was stored in another 13. That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057. (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.) https://github.com/python/cpython/issues/81057
* gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)Eric Snow2022-11-111-0/+2
| | | | | We also move the closely related max_module_number and add comments documenting the group of struct members. https://github.com/python/cpython/issues/81057
* gh-96947: Fix comment on `pyruntimestate->pyinterpreters` struct for ↵Виталий Дмитриев2022-09-201-2/+2
| | | | | | | `next_id` (GH-96949) `_next_interp_id` appeared on [this commit](https://github.com/python/cpython/commit/e377416c10eb0bf055b0728cdcdc4488fdfd3b5f#diff-7ac11e526f79b42d6ea9d3592cb99da46775640c69fa5510f4a6de87cced7141R68) renamed to `next_id` ([by this commit](https://github.com/python/cpython/commit/2ebc5ce42a8a9e047e790aefbf9a94811569b2b6#diff-bccfc01bd96b58c022dde77486b8a896cbb31d7581bd4a4156b32c3654afe468R59)). Also, now, `next_id` gets initialized in` _PyInterpreterState_Enable()` https://github.com/python/cpython/blob/12c5f328d2479ac3432df5e266adc4e59adeabfe/Python/pystate.c#L241-L244 because `_PyInterpreterState_Init()` function doesn't exist at all.
* GH-95909: Make `_PyArg_Parser` initialization thread safe (GH-95958)Kumar Aditya2022-08-161-0/+4
|
* bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)Eric Snow2022-05-271-2/+0
| | | | | | | This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing: * the configure option * the macro * the code enabled by the macro
* bpo-45459: Use type names in the internal C API (GH-31669)Victor Stinner2022-03-031-1/+1
| | | Replace "struct xxx" with "xxx" types in the internal C API.
* bpo-46370: Move the static initializer for _PyRuntime to its own header ↵Eric Snow2022-01-131-15/+0
| | | | | file. (gh-30587) https://bugs.python.org/issue46370
* bpo-45953: Statically allocate the main interpreter (and initial thread ↵Eric Snow2022-01-121-10/+38
| | | | | | | | | state). (gh-29883) Previously, the main interpreter was allocated on the heap during runtime initialization. Here we instead embed it into _PyRuntimeState, which means it is statically allocated as part of the _PyRuntime global. The same goes for the initial thread state (of each interpreter, including the main one). Consequently there are fewer allocations during runtime/interpreter init, fewer possible failures, and better memory locality. FYI, this also helps efforts to consolidate globals, which in turns helps work on subinterpreter isolation. https://bugs.python.org/issue45953
* bpo-45953: Statically initialize the small ints. (gh-30092)Eric Snow2021-12-141-9/+10
| | | | | | | The array of small PyLong objects has been statically declared. Here I also statically initialize them. Consequently they are no longer initialized dynamically during runtime init. I've also moved them under a new sub-struct in _PyRuntimeState, in preparation for static allocation and initialization of other global objects. https://bugs.python.org/issue45953
* bpo-46008: Make runtime-global object/type lifecycle functions and state ↵Eric Snow2021-12-091-21/+3
| | | | | | | | | | | | consistent. (gh-29998) This change is strictly renames and moving code around. It helps in the following ways: * ensures type-related init functions focus strictly on one of the three aspects (state, objects, types) * passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter * consistent naming conventions help make what's going on more clear * keeping API related to a type in the corresponding header file makes it more obvious where to look for it https://bugs.python.org/issue46008
* bpo-46008: Move Py*State init into distinct functions. (gh-29977)Eric Snow2021-12-081-1/+16
| | | | | Previously, basic initialization of PyInterprterState happened in PyInterpreterState_New() (along with allocation and adding the new interpreter to the runtime state). This prevented us from initializing interpreter states that were allocated separately (e.g. statically or in a free list). We've addressed that here by factoring out a separate function just for initialization. We've done the same for PyThreadState. _PyRuntimeState was sorted out when we added it since _PyRuntime is statically allocated. However, here we update the existing init code to line up with the functions for PyInterpreterState and PyThreadState. https://bugs.python.org/issue46008
* bpo-45691: Make array of small ints static to fix use-after-free error. ↵Mark Shannon2021-11-031-0/+15
| | | | (GH-29366)
* bpo-42882: _PyRuntimeState_Init() leaves unicode next_index unchanged (GH-24193)Victor Stinner2021-01-121-0/+4
| | | | | | | | Fix the _PyUnicode_FromId() function (_Py_IDENTIFIER(var) API) when Py_Initialize() / Py_Finalize() is called multiple times: preserve _PyRuntime.unicode_ids.next_index value. Use _PyRuntimeState_INIT macro instead memset(0) to reset _PyRuntimeState members to zero.
* bpo-39465: Fix _PyUnicode_FromId() for subinterpreters (GH-20058)Victor Stinner2020-12-251-0/+7
| | | | | | | | | | | | | | | | | Make _PyUnicode_FromId() function compatible with subinterpreters. Each interpreter now has an array of identifier objects (interned strings decoded from UTF-8). * Add PyInterpreterState.unicode.identifiers: array of identifiers objects. * Add _PyRuntimeState.unicode_ids used to allocate unique indexes to _Py_Identifier. * Rewrite the _Py_Identifier structure. Microbenchmark on _PyUnicode_FromId(&PyId_a) with _Py_IDENTIFIER(a): [ref] 2.42 ns +- 0.00 ns -> [atomic] 3.39 ns +- 0.00 ns: 1.40x slower This change adds 1 ns per _PyUnicode_FromId() call in average.
* PyOS_AfterFork_Child() uses PyStatus (GH-20596)Victor Stinner2020-06-021-1/+1
| | | | | | | | PyOS_AfterFork_Child() helper functions now return a PyStatus: PyOS_AfterFork_Child() is now responsible to handle errors. * Move _PySignal_AfterFork() to the internal C API * Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork() and _PyInterpreterState_DeleteExceptMain().
* bpo-40513: Per-interpreter GIL (GH-19943)Victor Stinner2020-05-051-0/+2
| | | | | | | | In the experimental isolated subinterpreters build mode, the GIL is now per-interpreter. Move gil from _PyRuntimeState.ceval to PyInterpreterState.ceval. new_interpreter() always get the config from the main interpreter.
* Revert "bpo-40513: Per-interpreter signals pending (GH-19924)" (GH-19932)Victor Stinner2020-05-051-0/+5
| | | This reverts commit 4e01946cafca0cf49f796c3118e0d65237bcad69.
* bpo-40513: Per-interpreter recursion_limit (GH-19929)Victor Stinner2020-05-051-1/+0
| | | | | | | | | | | Move recursion_limit member from _PyRuntimeState.ceval to PyInterpreterState.ceval. * Py_SetRecursionLimit() now only sets _Py_CheckRecursionLimit of ceval.c if the current Python thread is part of the main interpreter. * Inline _Py_MakeEndRecCheck() into _Py_LeaveRecursiveCall(). * Convert _Py_RecursionLimitLowerWaterMark() macro into a static inline function.
* bpo-40513: Per-interpreter gil_drop_request (GH-19927)Victor Stinner2020-05-051-2/+0
| | | | Move gil_drop_request member from _PyRuntimeState.ceval to PyInterpreterState.ceval.
generated by cgit v0.12 at 2025-09-09 06:41:47 (GMT)