summaryrefslogtreecommitdiffstats
path: root/Include/internal/pycore_interp.h
Commit message (Collapse)AuthorAgeFilesLines
* [3.12] gh-110310: Add a Per-Interpreter XID Registry for Heap Types ↵Eric Snow2023-11-281-15/+28
| | | | | | | | | | | | | | | (gh-110311) (gh-110714) We do the following: * add a per-interpreter XID registry (PyInterpreterState.xidregistry) * put heap types there (keep static types in _PyRuntimeState.xidregistry) * clear the registries during interpreter/runtime finalization * avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type) * use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup() The per-interpreter registry helps preserve isolation between interpreters. This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon. (cherry-picked from commit 80dc39e1dc2abc809f448cba5d2c5b9c1c631e11)
* [3.12] gh-105716: Support Background Threads in Subinterpreters Consistently ↵Eric Snow2023-11-281-0/+2
| | | | | | | | | (gh-109921) (gh-110707) The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish. We add PyInterpreterState.threads.main, with some internal C-API functions. (cherry-picked from commit 1dd9dee45d2591b4e701039d1673282380696849)
* [3.12] gh-109793: Allow Switching Interpreters During Finalization ↵Eric Snow2023-11-281-0/+18
| | | | | (gh-109794) (gh-110705) Essentially, we should check the thread ID rather than the thread state pointer.
* [3.12] GH-108390: Prevent non-local events being set with ↵Mark Shannon2023-09-051-1/+1
| | | | | | | | | | | `sys.monitoring.set_local_events()` (GH-108420) (#108899) * GH-108390: Prevent non-local events being set with `sys.monitoring.set_local_events()` (GH-108420) * Restore generated objects * Restore size of monitoring arrays in code object for 3.12 ABI compatibility. * Update ABI file
* [3.12] GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS ↵Victor Stinner2023-07-221-2/+2
| | | | | | | | | | | | (#107069) (#107075) GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069) Rename private C API constants: * Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS * Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (cherry picked from commit 0927a2b25c059988e237108605ed8ab0c5459c53)
* [3.12] gh-106140: Reorder some more fields to facilitate out-of-process ↵Pablo Galindo Salgado2023-06-271-7/+8
| | | | | inspection (GH-106148) (#106155) (cherry picked from commit 9126a6a9ce3772d5dc785cbee159b07a1ff7d531)
* [3.12] gh-106140: Reorder some fields to facilitate out-of-process ↵Miss Islington (bot)2023-06-271-33/+38
| | | | | | | | | inspection (GH-106143) (#106147) gh-106140: Reorder some fields to facilitate out-of-process inspection (GH-106143) (cherry picked from commit 2d5a1c281161d037148ffb5983decc6d31c2557d) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* GH-104580: Don't cache eval breaker in interpreter (GH-104581)Mark Shannon2023-05-181-1/+1
| | | Move eval-breaker to the front of the interpreter state.
* gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization ↵Eric Snow2023-05-151-0/+18
| | | | | (gh-104437) With the move to a per-interpreter GIL, this check slipped through the cracks.
* gh-99113: A Per-Interpreter GIL! (gh-104210)Eric Snow2023-05-081-0/+3
| | | | | This is the culmination of PEP 684 (and of my 8-year long multi-core Python project)! Each subinterpreter may now be created with its own GIL (via Py_NewInterpreterFromConfig()). If not so configured then the interpreter will share with the main interpreter--the status quo since subinterpreters were added decades ago. The main interpreter always has its own GIL and subinterpreters from Py_NewInterpreter() will always share with the main interpreter.
* gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)Eric Snow2023-04-241-1/+4
| | | | | | | | | | | | | | This is strictly about moving the "obmalloc" runtime state from `_PyRuntimeState` to `PyInterpreterState`. Doing so improves isolation between interpreters, specifically most of the memory (incl. objects) allocated for each interpreter's use. This is important for a per-interpreter GIL, but such isolation is valuable even without it. FWIW, a per-interpreter obmalloc is the proverbial canary-in-the-coalmine when it comes to the isolation of objects between interpreters. Any object that leaks (unintentionally) to another interpreter is highly likely to cause a crash (on debug builds at least). That's a useful thing to know, relative to interpreter isolation.
* GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython ↵Mark Shannon2023-04-121-1/+13
| | | | | | | | | | (GH-103083) * The majority of the monitoring code is in instrumentation.c * The new instrumentation bytecodes are in bytecodes.c * legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
* gh-101659: Add _Py_AtExit() (gh-103298)Eric Snow2023-04-061-15/+2
| | | | | The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on gh-101660.)
* gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545)Eric Snow2023-03-211-0/+2
| | | | | Moving it valuable with a per-interpreter GIL. However, it is also useful without one, since it allows us to identify refleaks within a single interpreter or where references are escaping an interpreter. This becomes more important as we move the obmalloc state to PyInterpreterState. https://github.com/python/cpython/issues/102304
* gh-102660: Handle m_copy Specially for the sys and builtins Modules (gh-102661)Eric Snow2023-03-141-0/+1
| | | | | | | It doesn't make sense to use multi-phase init for these modules. Using a per-interpreter "m_copy" (instead of PyModuleDef.m_base.m_copy) makes this work okay. (This came up while working on gh-101660.) Note that we might instead end up disallowing re-load for sys/builtins since they are so special. https://github.com/python/cpython/issues/102660
* gh-100227: Move func_state.next_version to PyInterpreterState (gh-102334)Eric Snow2023-03-081-0/+1
| | | https://github.com/python/cpython/issues/100227
* gh-100227: Move the dtoa State to PyInterpreterState (gh-102331)Eric Snow2023-02-281-0/+2
| | | https://github.com/python/cpython/issues/100227
* gh-101758: Clean Up Uses of Import State (gh-101919)Eric Snow2023-02-151-32/+3
| | | | | | | | | | | | | | | This change is almost entirely moving code around and hiding import state behind internal API. We introduce no changes to behavior, nor to non-internal API. (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.) The motivation is to simplify a number of upcoming changes. Specific changes: * move existing import-related code to import.c, wherever possible * add internal API for interacting with import state (both global and per-interpreter) * use only API outside of import.c (to limit churn there when changing the location, etc.) * consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly) * add macros for import state in import.c (to simplify changing the location) * group code in import.c into sections *remove _PyState_AddModule() https://github.com/python/cpython/issues/101758
* GH-100000: Cleanup and polish various watchers code (GH-99998)Itamar Ostricher2022-12-141-1/+0
| | | | | | * Initialize `type_watchers` array to `NULL`s * Optimize code watchers notification * Optimize func watchers notification
* gh-81057: Move Ceval Trampoline Globals to _PyRuntimeState (gh-100083)Eric Snow2022-12-091-31/+1
| | | https://github.com/python/cpython/issues/81057
* gh-99741: Fix the Cross-Interpreter Data API (gh-99939)Eric Snow2022-12-021-2/+3
| | | | | There were some minor issues that showed up while I was working on porting _xxsubinterpreters to multi-phase init. This fixes them. https://github.com/python/cpython/issues/99741
* GH-91054: Add code object watchers API (GH-99859)Itamar Ostricher2022-12-021-0/+3
| | | | | | * Add API to allow extensions to set callback function on creation and destruction of PyCodeObject Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
* gh-91053: Add an optional callback that is invoked whenever a function is ↵mpage2022-11-221-0/+6
| | | | modified (#98175)
* gh-81057: Move the global Dict-Related Versions to _PyRuntimeState (gh-99497)Eric Snow2022-11-161-3/+1
| | | | | We also move the global func version. https://github.com/python/cpython/issues/81057
* gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)Eric Snow2022-11-161-0/+1
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)Eric Snow2022-11-151-0/+1
| | | | | This is the first of several changes to consolidate non-object globals in core code. https://github.com/python/cpython/issues/81057
* GH-99205: remove `_static` field from `PyThreadState` and ↵Kumar Aditya2022-11-151-3/+0
| | | | `PyInterpreterState` (GH-99385)
* gh-81057: Add PyInterpreterState.static_objects (gh-99397)Eric Snow2022-11-111-0/+4
| | | | | As we consolidate global variables, we find some objects that are almost suitable to add to _PyRuntimeState.global_objects, but have some small/sneaky bit of per-interpreter state (e.g. a weakref list). We're adding PyInterpreterState.static_objects so we can move such objects there. (We'll removed the _not_used field once we've added others.) https://github.com/python/cpython/issues/81057
* gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)Eric Snow2022-11-111-0/+19
| | | | | We also move the closely related max_module_number and add comments documenting the group of struct members. https://github.com/python/cpython/issues/81057
* GH-96421: Insert shim frame on entry to interpreter (GH-96319)Mark Shannon2022-11-101-0/+1
| | | | | | * Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault() * Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
* gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() ↵Eric Snow2022-10-261-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | (gh-98609) (see https://github.com/python/cpython/issues/98608) This change does the following: 1. change the argument to a new `_PyInterpreterConfig` struct 2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig` instead of `isolated_subinterpreter`) 3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings * allow_fork * allow_subprocess * allow_threads 4. add `PyInterpreterState.feature_flags` to store those settings 5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *` 6. drop `PyConfig._isolated_interpreter` The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three. None of that changes here; the defaults are preserved. Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved. This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`. I'd rather just avoid that mess from the start for `_PyInterpreterConfig`. We can preserve it later if we find an actual need. This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead). (Note that this PR adds "private" symbols. We'll probably make them public, and add docs, in a separate change.)
* gh-91051: allow setting a callback hook on PyType_Modified (GH-97875)Carl Meyer2022-10-211-0/+1
|
* gh-98417: Store int_max_str_digits on the Interpreter State (GH-98418)Eric Snow2022-10-191-0/+6
|
* gh-97922: Run the GC only on eval breaker (#97920)Pablo Galindo Salgado2022-10-081-0/+2
|
* GH-91052: Add C API for watching dictionaries (GH-31787)Carl Meyer2022-10-071-0/+2
|
* gh-96512: Move int_max_str_digits setting to PyConfig (#96944)Gregory P. Smith2022-10-031-2/+0
| | | | | | | | | | | It had to live as a global outside of PyConfig for stable ABI reasons in the pre-3.12 backports. This removes the `_Py_global_config_int_max_str_digits` and gets rid of the equivalent field in the internal `struct _is PyInterpreterState` as code can just use the existing nested config struct within that. Adds tests to verify unique settings and configs in subinterpreters.
* gh-90110: Fix the c-analyzer Tool (gh-96731)Eric Snow2022-09-121-1/+0
| | | | | | | This includes: * update the whitelists * fixes so we can stop ignoring some of the files * ensure Include/cpython/*.h get analyzed
* gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96499)Gregory P. Smith2022-09-021-0/+2
| | | | | | | | | | | | | | | | Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds. This PR comes fresh from a pile of work done in our private PSRT security response team repo. Signed-off-by: Christian Heimes [Red Hat] <christian@python.org> Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org> Reviews via the private PSRT repo via many others (see the NEWS entry in the PR). <!-- gh-issue-number: gh-95778 --> * Issue: gh-95778 <!-- /gh-issue-number --> I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
* gh-94673: Add Per-Interpreter Storage for Static Builtin Types (#95255)Eric Snow2022-07-261-1/+1
| | | | | | | | | | | | | | This is the last precursor to storing tp_subclasses (and tp_weaklist) on the interpreter state for static builtin types. Here we add per-type storage on PyInterpreterState, but only for the static builtin types. This involves the following: * add PyInterpreterState.types * move PyInterpreterState.type_cache to it * add a "num_builtins_initialized" field * add a "builtins" field (a static array big enough for all the static builtin types) * add _PyStaticType_GetState() to look up a static builtin type's state * (temporarily) add PyTypeObject.tp_static_builtin_index (to hold the type's index into PyInterpreterState.types.builtins) We will be eliminating tp_static_builtin_index in a later change.
* pycore_interp.h: Fix comment about pytypedefs.h (#93188)Itamar Ostricher2022-07-091-1/+1
| | | | GH-31527 moved this typedef to `Include/pytypedefs.h`, so this comment should point at the correct location
* bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)Eric Snow2022-05-271-3/+0
| | | | | | | This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing: * the configure option * the macro * the code enabled by the macro
* Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API ↵Victor Stinner2022-04-061-15/+1
| | | | | | | | | | | (GH-32052)" (GH-32343) * Revert "bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API (GH-32054)" This reverts commit f877b40e3f7e0d97878884d80fbec879a85ab7e8. * Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API (GH-32052)" This reverts commit b9a5522dd952125a99ff554f01f311cae25f5a91.
* bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API ↵Victor Stinner2022-04-011-1/+15
| | | | | | | | | | | (GH-32054) Move the private _PyFrameEvalFunction type, and private _PyInterpreterState_GetEvalFrameFunc() and _PyInterpreterState_SetEvalFrameFunc() functions to the internal C API. The _PyFrameEvalFunction callback function type now uses the _PyInterpreterFrame type which is part of the internal C API. Update the _PyFrameEvalFunction documentation.
* bpo-46841: Use inline caching for calls (GH-31709)Brandt Bucher2022-03-071-0/+2
|
* bpo-45459: Use type names in the internal C API (GH-31669)Victor Stinner2022-03-031-7/+7
| | | Replace "struct xxx" with "xxx" types in the internal C API.
* bpo-45953: Statically allocate the main interpreter (and initial thread ↵Eric Snow2022-01-121-1/+29
| | | | | | | | | state). (gh-29883) Previously, the main interpreter was allocated on the heap during runtime initialization. Here we instead embed it into _PyRuntimeState, which means it is statically allocated as part of the _PyRuntime global. The same goes for the initial thread state (of each interpreter, including the main one). Consequently there are fewer allocations during runtime/interpreter init, fewer possible failures, and better memory locality. FYI, this also helps efforts to consolidate globals, which in turns helps work on subinterpreter isolation. https://bugs.python.org/issue45953
* bpo-45953: Statically allocate and initialize global bytes objects. (gh-30096)Eric Snow2022-01-111-2/+0
| | | | | The empty bytes object (b'') and the 256 one-character bytes objects were allocated at runtime init. Now we statically allocate and initialize them. https://bugs.python.org/issue45953
* bpo-46008: Make runtime-global object/type lifecycle functions and state ↵Eric Snow2021-12-091-173/+10
| | | | | | | | | | | | consistent. (gh-29998) This change is strictly renames and moving code around. It helps in the following ways: * ensures type-related init functions focus strictly on one of the three aspects (state, objects, types) * passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter * consistent naming conventions help make what's going on more clear * keeping API related to a type in the corresponding header file makes it more obvious where to look for it https://bugs.python.org/issue46008
* bpo-46008: Move Py*State init into distinct functions. (gh-29977)Eric Snow2021-12-081-1/+5
| | | | | Previously, basic initialization of PyInterprterState happened in PyInterpreterState_New() (along with allocation and adding the new interpreter to the runtime state). This prevented us from initializing interpreter states that were allocated separately (e.g. statically or in a free list). We've addressed that here by factoring out a separate function just for initialization. We've done the same for PyThreadState. _PyRuntimeState was sorted out when we added it since _PyRuntime is statically allocated. However, here we update the existing init code to line up with the functions for PyInterpreterState and PyThreadState. https://bugs.python.org/issue46008
* bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)Eric Snow2021-12-071-11/+13
| | | | | This parallels _PyRuntimeState.interpreters. Doing this helps make it more clear what part of PyInterpreterState relates to its threads. https://bugs.python.org/issue46008
generated by cgit v0.12 at 2026-01-06 22:16:36 (GMT)