summaryrefslogtreecommitdiffstats
path: root/Include/internal/pycore_interp.h
Commit message (Collapse)AuthorAgeFilesLines
* [3.13] gh-126914: Store the Preallocated Thread State's Pointer in a ↵Eric Snow2024-12-021-0/+2
| | | | | | | PyInterpreterState Field (gh-127114) This approach eliminates the originally reported race. It also gets rid of the deadlock reported in gh-96071, so we can remove the workaround added then. This is mostly a cherry-pick of 1c0a104 (AKA gh-126989). The difference is we add PyInterpreterState.threads_preallocated at the end of PyInterpreterState, instead of adding PyInterpreterState.threads.preallocated. That avoids ABI disruption.
* [3.13] gh-120642: Move private PyCode APIs to the internal C API (#120643) ↵Victor Stinner2024-06-261-0/+1
| | | | | | | | | | | | | | | | | | | | | (#121043) gh-120642: Move private PyCode APIs to the internal C API (#120643) * Move _Py_CODEUNIT and related functions to pycore_code.h. * Move _Py_BackoffCounter to pycore_backoff.h. * Move Include/cpython/optimizer.h content to pycore_optimizer.h. * Remove Include/cpython/optimizer.h. * Remove PyUnstable_Replace_Executor(). Rename functions: * PyUnstable_GetExecutor() => _Py_GetExecutor() * PyUnstable_GetOptimizer() => _Py_GetOptimizer() * PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer() * PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter() * PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer() (cherry picked from commit 9e4a81f00fef689c6e18a64245aa064eaadc7ac7)
* [3.13] gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165) ↵Miss Islington (bot)2024-06-121-1/+4
| | | | | | | | | | (GH-120403) gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165) (cherry picked from commit e16aed63f64b18a26859eff3de976ded373e66b8) Co-authored-by: Ken Jin <kenjin@python.org> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
* gh-118527: Intern code consts in free-threaded build (#118667)Sam Gross2024-05-071-0/+1
| | | | | | We already intern and immortalize most string constants. In the free-threaded build, other constants can be a source of reference count contention because they are shared by all threads running the same code objects.
* gh-116738: Make `_codecs` module thread-safe (#117530)Brett Simmers2024-05-021-4/+2
| | | | | | | | | | | | | | | The module itself is a thin wrapper around calls to functions in `Python/codecs.c`, so that's where the meaningful changes happened: - Move codecs-related state that lives on `PyInterpreterState` to a struct declared in `pycore_codecs.h`. - In free-threaded builds, add a mutex to `codecs_state` to synchronize operations on `search_path`. Because `search_path_mutex` is used as a normal mutex and not a critical section, we must be extremely careful with operations called while holding it. - The codec registry is explicitly initialized as part of `_PyUnicode_InitEncodings` to simplify thread-safety.
* gh-76785: Handle Legacy Interpreters Properly (gh-117490)Eric Snow2024-04-111-1/+4
| | | This is similar to the situation with threading._DummyThread. The methods (incl. __del__()) of interpreters.Interpreter objects must be careful with interpreters not created by interpreters.create(). The simplest thing to start with is to disable any method that modifies or runs in the interpreter. As part of this, the runtime keeps track of where an interpreter was created. We also handle interpreter "refcounts" properly.
* gh-76785: Add More Tests to test_interpreters.test_api (gh-117662)Eric Snow2024-04-111-0/+16
| | | In addition to the increase test coverage, this is a precursor to sorting out how we handle interpreters created directly via the C-API.
* gh-111926: Make weakrefs thread-safe in free-threaded builds (#117168)mpage2024-04-081-0/+7
| | | | | | | | | Most mutable data is protected by a striped lock that is keyed on the referenced object's address. The weakref's hash is protected using the weakref's per-object lock. Note that this only affects free-threaded builds. Apart from some minor refactoring, the added code is all either gated by `ifdef`s or is a no-op (e.g. `Py_BEGIN_CRITICAL_SECTION`).
* gh-116968: Reimplement Tier 2 counters (#117144)Guido van Rossum2024-04-041-6/+0
| | | | | | | | | | | | Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``), shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The API used for adaptive specialization counters is changed but the behavior is (supposed to be) identical. The behavior of the Tier 2 counters is changed: - There are no longer dynamic thresholds (we never varied these). - All counters now use the same exponential backoff. - The counter for ``JUMP_BACKWARD`` starts counting down from 16. - The ``temperature`` in side exits starts counting down from 64.
* gh-76785: Drop PyInterpreterID_Type (gh-117101)Eric Snow2024-03-211-3/+2
| | | I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation. Relatively recently I refactored that implementation to no longer rely on InterpreterID objects. Thus now I'm removing it.
* gh-76785: Clean Up Interpreter ID Conversions (gh-117048)Eric Snow2024-03-211-0/+3
| | | Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t. We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
* gh-116916: Remove separate next_func_version counter (#116918)Guido van Rossum2024-03-181-1/+0
| | | | | Somehow we ended up with two separate counter variables tracking "the next function version". Most likely this was a historical accident where an old branch was updated incorrectly. This PR merges the two counters into a single one: `interp->func_state.next_version`.
* chore: fix typos (#116345)cui fliter2024-03-051-1/+1
| | | Signed-off-by: cui fliter <imcusg@gmail.com>
* gh-115103: Implement delayed free mechanism for free-threaded builds (#115367)Sam Gross2024-02-201-0/+1
| | | | | | This adds `_PyMem_FreeDelayed()` and supporting functions. The `_PyMem_FreeDelayed()` function frees memory with the same allocator as `PyMem_Free()`, but after some delay to ensure that concurrent lock-free readers have finished.
* GH-112354: Initial implementation of warm up on exits and trace-stitching ↵Mark Shannon2024-02-201-1/+4
| | | | (GH-114142)
* gh-115103: Implement delayed memory reclamation (QSBR) (#115180)Sam Gross2024-02-161-0/+2
| | | | | | This adds a safe memory reclamation scheme based on FreeBSD's "GUS" and quiescent state based reclamation (QSBR). The API provides a mechanism for callers to detect when it is safe to free memory that may be concurrently accessed by readers.
* gh-111968: Rename freelist related struct names to Eric's suggestion (gh-115329)Donghee Na2024-02-141-4/+0
|
* GH-113710: Backedge counter improvements. (GH-115166)Mark Shannon2024-02-131-2/+4
|
* gh-114271: Make `PyInterpreterState.threads.count` thread-safe in ↵mpage2024-02-121-1/+1
| | | | | free-threaded builds (gh-115093) Use atomics to mutate PyInterpreterState.threads.count.
* gh-110481: Implement inter-thread queue for biased reference counting (#114824)Sam Gross2024-02-091-0/+1
| | | | | | | | | Biased reference counting maintains two refcount fields in each object: `ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two fields. In some cases, when refcounting operations are split across threads, the ob_ref_shared field can be negative (although the total refcount must be at least zero). In this case, the thread that decremented the refcount requests that the owning thread give up ownership and merge the refcount fields.
* GH-113710: Add a "globals to constants" pass (GH-114592)Mark Shannon2024-02-021-1/+1
| | | Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
* gh-111968: Use per-thread freelists for dict in free-threading (gh-114323)Donghee Na2024-02-011-1/+1
|
* gh-113055: Use pointer for interp->obmalloc state (gh-113412)Neil Schemenauer2024-01-271-1/+11
| | | | | | | | | For interpreters that share state with the main interpreter, this points to the same static memory structure. For interpreters with their own obmalloc state, it is heap allocated. Add free_obmalloc_arenas() which will free the obmalloc arenas and radix tree structures for interpreters with their own obmalloc state. Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* gh-114312: Collect stats for unlikely events (GH-114493)Michael Droettboom2024-01-251-0/+29
|
* gh-111964: Implement stop-the-world pauses (gh-112471)Sam Gross2024-01-231-0/+17
| | | | | | | | | | | | | | | | | The `--disable-gil` builds occasionally need to pause all but one thread. Some examples include: * Cyclic garbage collection, where this is often called a "stop the world event" * Before calling `fork()`, to ensure a consistent state for internal data structures * During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects This adds the following functions to implement global and per-interpreter pauses: * `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime) * `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter) (The function names may change.) These functions are no-ops outside of the `--disable-gil` build.
* gh-111968: Use per-thread freelists for generator in free-threading (gh-114189)Donghee Na2024-01-181-1/+0
|
* gh-111968: Use per-thread freelists for PyContext in free-threading (gh-114122)Donghee Na2024-01-161-1/+0
|
* gh-111968: Use per-thread slice_cache in free-threading (gh-113972)Donghee Na2024-01-151-3/+0
|
* gh-111968: Use per-thread freelists for float in free-threading (gh-113886)Donghee Na2024-01-101-1/+0
|
* gh-111968: Introduce _PyFreeListState and _PyFreeListState_GET API (gh-113584)Donghee Na2024-01-091-1/+3
|
* gh-112532: Isolate abandoned segments by interpreter (#113717)Sam Gross2024-01-041-0/+5
| | | | | | | | | | | | | | | * gh-112532: Isolate abandoned segments by interpreter Mimalloc segments are data structures that contain memory allocations along with metadata. Each segment is "owned" by a thread. When a thread exits, it abandons its segments to a global pool to be later reclaimed by other threads. This changes the pool to be per-interpreter instead of process-wide. This will be important for when we use mimalloc to find GC objects in the `--disable-gil` builds. We want heaps to only store Python objects from a single interpreter. Absent this change, the abandoning and reclaiming process could break this isolation. * Add missing '&_mi_abandoned_default' to 'tld_empty'
* gh-76785: Fixes for test.support.interpreters (gh-112982)Eric Snow2023-12-121-3/+3
| | | This involves a number of changes for PEP 734.
* gh-112538: Add internal-only _PyThreadStateImpl "wrapper" for PyThreadState ↵Sam Gross2023-12-071-2/+3
| | | | | | | | (gh-112560) Every PyThreadState instance is now actually a _PyThreadStateImpl. It is safe to cast from `PyThreadState*` to `_PyThreadStateImpl*` and back. The _PyThreadStateImpl will contain fields that we do not want to expose in the public C API.
* gh-103615: Use local events for opcode tracing (GH-109472)Tian Gao2023-11-031-1/+0
| | | | | | | * Use local monitoring for opcode trace * Remove f_opcode_trace_set * Add test for setting f_trace_opcodes after settrace
* gh-76785: Crossinterp utils additions (gh-111530)Eric Snow2023-11-011-2/+2
| | | | | | | | | | | | This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers). Specifically: * _Py_excinfo, etc.: the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c) * _PyXI_exception_info, etc.: helpers for passing an exception beween interpreters (wraps _Py_excinfo) * _PyXI_namespace, etc.: helpers for copying a dict of attrs between interpreters * _PyXI_Enter(), _PyXI_Exit(): functions that abstract out the transitions between one interpreter and a second that will do some work temporarily Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations. I plan on proposing these as public API at some point.
* gh-76785: Move the Cross-Interpreter Code to Its Own File (gh-111502)Eric Snow2023-10-301-22/+1
| | | This is partly to clear this stuff out of pystate.c, but also in preparation for moving some code out of _xxsubinterpretersmodule.c. This change also moves this stuff to the internal API (new: Include/internal/pycore_crossinterp.h). @vstinner did this previously and I undid it. Now I'm re-doing it. :/
* GH-109369: Add machinery for deoptimizing tier2 executors, both individually ↵Mark Shannon2023-10-231-0/+1
| | | | and globally. (GH-110384)
* gh-110805: Allow the repl to show source code and complete tracebacks (#110775)Pablo Galindo Salgado2023-10-131-0/+1
|
* gh-109693: Update pycore_interp.h to use pyatomic.h (#110604)Donghee Na2023-10-101-4/+3
|
* gh-76785: Add SendChannel.send_buffer() (#110246)Eric Snow2023-10-091-1/+2
| | | (This is still a test module.)
* gh-110310: Add a Per-Interpreter XID Registry for Heap Types (gh-110311)Eric Snow2023-10-041-15/+29
| | | | | | | | | | | We do the following: * add a per-interpreter XID registry (PyInterpreterState.xidregistry) * put heap types there (keep static types in _PyRuntimeState.xidregistry) * clear the registries during interpreter/runtime finalization * avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type) * use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup() The per-interpreter registry helps preserve isolation between interpreters. This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.
* GH-109369: Merge all eval-breaker flags and monitoring version into one ↵Mark Shannon2023-10-041-2/+1
| | | | word. (GH-109846)
* gh-105716: Support Background Threads in Subinterpreters Consistently ↵Eric Snow2023-10-021-0/+2
| | | | | | | (gh-109921) The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish. We add PyInterpreterState.threads.main, with some internal C-API functions.
* gh-109793: Allow Switching Interpreters During Finalization (gh-109794)Eric Snow2023-09-271-0/+16
| | | Essentially, we should check the thread ID rather than the thread state pointer.
* GH-108716: Turn off deep-freezing of code objects. (GH-108722)Mark Shannon2023-09-081-0/+1
|
* GH-108390: Prevent non-local events being set with ↵Mark Shannon2023-09-051-1/+1
| | | | `sys.monitoring.set_local_events()` (GH-108420)
* gh-108634: PyInterpreterState_New() no longer calls Py_FatalError() (#108748)Victor Stinner2023-09-011-0/+4
| | | | | | | | | | | | pycore_create_interpreter() now returns a status, rather than calling Py_FatalError(). * PyInterpreterState_New() now calls Py_ExitStatusException() instead of calling Py_FatalError() directly. * Replace Py_FatalError() with PyStatus in init_interpreter() and _PyObject_InitState(). * _PyErr_SetFromPyStatus() now raises RuntimeError, instead of ValueError. It can now call PyErr_NoMemory(), raise MemoryError, if it detects _PyStatus_NO_MEMORY() error message.
* gh-107211: No longer export internal functions (6) (#108424)Victor Stinner2023-08-241-31/+31
| | | | | | | | | | | | No longer export these 5 internal C API functions: * _PyArena_AddPyObject() * _PyArena_Free() * _PyArena_Malloc() * _PyArena_New() * _Py_FatalRefcountErrorFunc() Change comment style to "// comment" and add comment explaining why other functions have to be exported.
* gh-108216: Cleanup #include in internal header files (#108228)Victor Stinner2023-08-211-6/+6
| | | | | | | | | | | * Add missing includes. * Remove unused includes. * Update old include/symbol names to newer names. * Mention at least one included symbol. * Sort includes. * Update Tools/cases_generator/generate_cases.py used to generated pycore_opcode_metadata.h. * Update Parser/asdl_c.py used to generate pycore_ast.h. * Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
* GH-104580: Put `eval_breaker` back at the start of the interpreter state. ↵Mark Shannon2023-07-281-2/+5
| | | | (GH-107383)
generated by cgit v0.12 at 2026-01-10 14:02:23 (GMT)