summaryrefslogtreecommitdiffstats
path: root/Python/ceval_gil.c
Commit message (Collapse)AuthorAgeFilesLines
* GH-142513: Reimplement executor management (GH-142931)Mark Shannon2025-12-181-0/+6
| | | | | * Invalidating an executor does not cause arbitrary code to run * Executors are only freed at safe points
* gh-135953: Add GIL contention markers to sampling profiler Gecko format ↵Pablo Galindo Salgado2025-11-171-0/+4
| | | | | | (#139485) This commit enhances the Gecko format reporter in the sampling profiler to include markers for GIL acquisition events.
* gh-138050: [WIP] JIT - Streamline MAKE_WARM - move coldness check to ↵alm2025-10-271-1/+1
| | | | executor creation (GH-138240)
* gh-137514: Add a free-threading wrapper for mutexes (GH-137515)Peter Bierma2025-08-071-6/+2
| | | Add `FT_MUTEX_LOCK`/`FT_MUTEX_UNLOCK`, which call `PyMutex_Lock` and `PyMutex_Unlock` on the free-threaded build, and no-op otherwise.
* GH-133136: Revise QSBR to reduce excess memory held (gh-135473)Neil Schemenauer2025-06-251-0/+4
| | | | | | | | | | | | | | | | | The free threading build uses QSBR to delay the freeing of dictionary keys and list arrays when the objects are accessed by multiple threads in order to allow concurrent reads to proceed with holding the object lock. The requests are processed in batches to reduce execution overhead, but for large memory blocks this can lead to excess memory usage. Take into account the size of the memory block when deciding when to process QSBR requests. Also track the amount of memory being held by QSBR for mimalloc pages. Advance the write sequence if this memory exceeds a limit. Advancing the sequence will allow it to be freed more quickly. Process the held QSBR items from the "eval breaker", rather than from `_PyMem_FreeDelayed()`. This gives a higher chance that the global read sequence has advanced enough so that items can be freed. Co-authored-by: Sam Gross <colesbury@gmail.com>
* gh-135543: Emit sys.remote_exec audit event when sys.remote_exec is called ↵Nadeshiko Manju2025-06-191-1/+1
| | | | (GH-135544)
* gh-133886: Fix sys.remote_exec() for non-UTF-8 paths (GH-133887)Serhiy Storchaka2025-05-131-9/+16
| | | | | It now supports non-ASCII paths in non-UTF-8 locales and non-UTF-8 paths in UTF-8 locales.
* gh-132859: Run debugger scripts in their own namespaces (#132860)Matt Wozniski2025-04-231-13/+25
| | | | | | | | Run debugger scripts in their own namespaces Previously scripts injected by `sys.remote_exec` were run with the globals of the `__main__` module. Instead, run each injected script with an empty set of globals. If someone really wants to use the `__main__` module's namespace, they can always `import __main__`.
* gh-131591: Check for remote debug in PyErr_CheckSignals (#132853)Pablo Galindo Salgado2025-04-231-26/+33
| | | | | | | | For the same reasons as running the GC, this will allow sections that run in native code for long periods without executing bytecode to also run the remote debugger protocol without having to wait until bytecode is executed Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* gh-131591: Execute the source and not the file to avoid locking it in ↵Pablo Galindo Salgado2025-04-191-35/+21
| | | | | Windows (#132712) Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* gh-131591: Implement PEP 768 (#131937)Pablo Galindo Salgado2025-04-031-0/+95
| | | | Co-authored-by: Ivona Stojanovic <stojanovic.i@hotmail.com> Co-authored-by: Matt Wozniski <godlygeek@gmail.com>
* gh-131238: Remove includes from pycore_interp.h (#131495)Victor Stinner2025-03-201-4/+2
| | | Remove also now unused includes in C files.
* gh-131238: Remove more includes from pycore_interp.h (#131480)Victor Stinner2025-03-191-0/+2
|
* gh-124878: Fix race conditions during interpreter finalization (#130649)Sam Gross2025-03-061-10/+9
| | | | | | | | | | | | | | | | | | | | The PyThreadState field gains a reference count field to avoid issues with PyThreadState being a dangling pointer to freed memory. The refcount starts with a value of two: one reference is owned by the interpreter's linked list of thread states and one reference is owned by the OS thread. The reference count is decremented when the thread state is removed from the interpreter's linked list and before the OS thread calls `PyThread_hang_thread()`. The thread that decrements it to zero frees the `PyThreadState` memory. The `holds_gil` field is moved out of the `_status` bit field, to avoid a data race where on thread calls `PyThreadState_Clear()`, modifying the `_status` bit field while the OS thread reads `holds_gil` when attempting to acquire the GIL. The `PyThreadState.state` field now has `_Py_THREAD_SHUTTING_DOWN` as a possible value. This corresponds to the `_PyThreadState_MustExit()` check. This avoids race conditions in the free threading build when checking `_PyThreadState_MustExit()`.
* gh-130605: Use relaxed atomics to set the GIL switch interval (gh-130654)Sam Gross2025-02-281-3/+6
| | | | The interval may be concurrently read by a thread attempting to acquire the GIL.
* gh-128360: Add `_Py_AssertHoldsTstate` as assertion for holding a thread ↵Peter Bierma2025-01-201-2/+2
| | | | | state (#128361) Co-authored-by: Kumar Aditya <kumaraditya@python.org>
* gh-114940: Add _Py_FOR_EACH_TSTATE_UNLOCKED(), and Friends (gh-127077)Eric Snow2024-11-211-10/+4
| | | This is a precursor to the actual fix for gh-114940, where we will change these macros to use the new lock. This change is almost entirely mechanical; the exceptions are the loops in codeobject.c and ceval.c, which now hold the "head" lock. Note that almost all of the uses of _Py_FOR_EACH_TSTATE_UNLOCKED() here will change to _Py_FOR_EACH_TSTATE_BEGIN() once we add the new per-interpreter lock.
* gh-87135: Hang non-main threads that attempt to acquire the GIL during ↵Jeremy Maitin-Shepard2024-10-021-9/+17
| | | | | | | finalization (GH-105805) Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option. Co-authored-by: Gregory P. Smith <greg@krypto.org>
* GH-123516: Improve JIT memory consumption by invalidating cold executors ↵Savannah Ostrowski2024-09-271-0/+6
| | | | | (GH-124443) Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* gh-122860: Remove unused macro `_Py_atomic_load_relaxed_int32` (#122861)Sam Gross2024-08-111-7/+0
|
* gh-122201: Lock mutex when setting handling_thread to NULL (#122204)Sam Gross2024-07-261-3/+15
| | | | | In the free-threaded build, we need to lock pending->mutex when clearing the handling_thread in order not to race with a concurrent make_pending_calls in the same interpreter.
* gh-118297: Make Sure All Pending Calls Run in _Py_FinishPendingCalls() ↵Eric Snow2024-07-151-6/+28
| | | | (gh-118298)
* gh-118727: Don't drop the GIL in `drop_gil()` unless the current thread ↵Brett Simmers2024-05-231-39/+57
| | | | | | | | | | | | | | | | | holds it (#118745) `drop_gil()` assumes that its caller is attached, which means that the current thread holds the GIL if and only if the GIL is enabled, and the enabled-state of the GIL won't change. This isn't true, though, because `detach_thread()` calls `_PyEval_ReleaseLock()` after detaching and `_PyThreadState_DeleteCurrent()` calls it after removing the current thread from consideration for stop-the-world requests (effectively detaching it). Fix this by remembering whether or not a thread acquired the GIL when it last attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()` instead of `gil->enabled`. This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've reenabled it.
* gh-116322: Enable the GIL while loading C extension modules (#118560)Brett Simmers2024-05-071-13/+145
| | | | | | | | | | Add the ability to enable/disable the GIL at runtime, and use that in the C module loading code. We can't know before running a module init function if it supports free-threading, so the GIL is temporarily enabled before doing so. If the module declares support for running without the GIL, the GIL is later disabled. Otherwise, the GIL is permanently enabled, and will never be disabled again for the life of the current interpreter.
* gh-118534: Fix load of `gil->locked` (#118553)Sam Gross2024-05-031-1/+1
|
* GH-118095: Make sure that progress is made if there are pending calls being ↵Mark Shannon2024-05-011-11/+10
| | | | handled. (GH-118484)
* gh-116749: Disable GIL by default in free-threaded build (#118295)Sam Gross2024-04-261-3/+1
| | | | Switch GIL to disabled by default in free-threaded build so that the free-threaded CIs catch thread-safety issues.
* gh-110693: Pending Calls Machinery Cleanups (gh-118296)Eric Snow2024-04-261-55/+104
| | | This does some cleanup in preparation for later changes.
* gh-117929: Restore removed PyEval_InitThreads() function (#117931)Victor Stinner2024-04-171-2/+1
|
* gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)Sam Gross2024-03-211-5/+2
| | | | | | | | | | | Split `_PyThreadState_DeleteExcept` into two functions: - `_PyThreadState_RemoveExcept` removes all thread states other than one passed as an argument. It returns the removed thread states as a linked list. - `_PyThreadState_DeleteList` deletes those dead thread states. It may call destructors, so we want to "start the world" before calling `_PyThreadState_DeleteList` to avoid potential deadlocks.
* gh-116908: Only write to `_pending_calls.calls_to_do` with atomic operations ↵Brett Simmers2024-03-201-2/+2
| | | | | | (#117044) These writes to `pending->calls_to_do` need to be atomic, because other threads can read (atomically) from `calls_to_do` without holding `pending->mutex`.
* gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338)Brett Simmers2024-03-111-0/+15
| | | | | | | | | In free-threaded builds, running with `PYTHON_GIL=0` will now disable the GIL. Follow-up issues track work to re-enable the GIL when loading an incompatible extension, and to disable the GIL by default. In order to support re-enabling the GIL at runtime, all GIL-related data structures are initialized as usual, and disabling the GIL simply sets a flag that causes `take_gil()` and `drop_gil()` to return early.
* gh-116590: Fix unused `current_thread_holds_gil` function warning (#116591)Nikita Sobolev2024-03-111-0/+2
|
* gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)Brett Simmers2024-02-201-93/+150
| | | | | | | | | | | This change adds an `eval_breaker` field to `PyThreadState`. The primary motivation is for performance in free-threaded builds: with thread-local eval breakers, we can stop a specific thread (e.g., for an async exception) without interrupting other threads. The source of truth for the global instrumentation version is stored in the `instrumentation_version` field in PyInterpreterState. Threads usually read the version from their local `eval_breaker`, where it continues to be colocated with the eval breaker bits.
* gh-110481: Implement inter-thread queue for biased reference counting (#114824)Sam Gross2024-02-091-0/+8
| | | | | | | | | Biased reference counting maintains two refcount fields in each object: `ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two fields. In some cases, when refcounting operations are split across threads, the ob_ref_shared field can be negative (although the total refcount must be at least zero). In this case, the thread that decremented the refcount requests that the owning thread give up ownership and merge the refcount fields.
* gh-104530: Enable native Win32 condition variables by default (GH-104531)Andrew Rogers2024-02-021-0/+8
|
* gh-111964: Implement stop-the-world pauses (gh-112471)Sam Gross2024-01-231-0/+9
| | | | | | | | | | | | | | | | | The `--disable-gil` builds occasionally need to pause all but one thread. Some examples include: * Cyclic garbage collection, where this is often called a "stop the world event" * Before calling `fork()`, to ensure a consistent state for internal data structures * During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects This adds the following functions to implement global and per-interpreter pauses: * `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime) * `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter) (The function names may change.) These functions are no-ops outside of the `--disable-gil` build.
* gh-112723: Call `PyThreadState_Clear()` from the correct interpreter (#112776)Sam Gross2023-12-131-3/+1
| | | | | | | | | | | | | The `PyThreadState_Clear()` function must only be called with the GIL held and must be called from the same interpreter as the passed in thread state. Otherwise, any Python objects on the thread state may be destroyed using the wrong interpreter, leading to memory corruption. This is also important for `Py_GIL_DISABLED` builds because free lists will be associated with PyThreadStates and cleared in `PyThreadState_Clear()`. This fixes two places that called `PyThreadState_Clear()` from the wrong interpreter and adds an assertion to `PyThreadState_Clear()`.
* gh-112978: Remove redundant condition inside `take_gil` (gh-112979)Yan Yanchii2023-12-111-5/+0
|
* gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)Sam Gross2023-12-071-28/+9
| | | | | This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize. This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
* gh-109693: Remove pycore_atomic.h (gh-110992)Donghee Na2023-10-171-1/+0
|
* gh-109693: Update _gil_runtime_state.locked to use pyatomic.h (gh-110836)Donghee Na2023-10-161-16/+11
|
* gh-109693: Update _gil_runtime_state.last_holder to use pyatomic.h (#110605)Donghee Na2023-10-131-7/+7
|
* gh-109693: Use pyatomic.h for signal module (gh-110480)Donghee Na2023-10-091-1/+1
|
* gh-76785: Add SendChannel.send_buffer() (#110246)Eric Snow2023-10-091-12/+19
| | | (This is still a test module.)
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-051-36/+11
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* GH-109369: Merge all eval-breaker flags and monitoring version into one ↵Mark Shannon2023-10-041-179/+72
| | | | word. (GH-109846)
* gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)Eric Snow2023-09-191-4/+4
| | | This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)Victor Stinner2023-09-111-25/+3
| | | | | | | | | | | | | Fix _thread.start_new_thread() race condition. If a thread is created during Python finalization, the newly spawned thread now exits immediately instead of trying to access freed memory and lead to a crash. thread_run() calls PyEval_AcquireThread() which checks if the thread must exit. The problem was that tstate was dereferenced earlier in _PyThreadState_Bind() which leads to a crash most of the time. Move _PyThreadState_CheckConsistency() from thread_run() to _PyThreadState_Bind().
* gh-104690: thread_run() checks for tstate dangling pointer (#109056)Victor Stinner2023-09-081-18/+8
| | | | | | | | thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
generated by cgit v0.12 at 2026-01-15 18:16:20 (GMT)