summaryrefslogtreecommitdiffstats
path: root/Python/ceval_gil.c
Commit message (Collapse)AuthorAgeFilesLines
* gh-132859: Run debugger scripts in their own namespaces (#132860)Matt Wozniski2025-04-231-13/+25
| | | | | | | | Run debugger scripts in their own namespaces Previously scripts injected by `sys.remote_exec` were run with the globals of the `__main__` module. Instead, run each injected script with an empty set of globals. If someone really wants to use the `__main__` module's namespace, they can always `import __main__`.
* gh-131591: Check for remote debug in PyErr_CheckSignals (#132853)Pablo Galindo Salgado2025-04-231-26/+33
| | | | | | | | For the same reasons as running the GC, this will allow sections that run in native code for long periods without executing bytecode to also run the remote debugger protocol without having to wait until bytecode is executed Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* gh-131591: Execute the source and not the file to avoid locking it in ↵Pablo Galindo Salgado2025-04-191-35/+21
| | | | | Windows (#132712) Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* gh-131591: Implement PEP 768 (#131937)Pablo Galindo Salgado2025-04-031-0/+95
| | | | Co-authored-by: Ivona Stojanovic <stojanovic.i@hotmail.com> Co-authored-by: Matt Wozniski <godlygeek@gmail.com>
* gh-131238: Remove includes from pycore_interp.h (#131495)Victor Stinner2025-03-201-4/+2
| | | Remove also now unused includes in C files.
* gh-131238: Remove more includes from pycore_interp.h (#131480)Victor Stinner2025-03-191-0/+2
|
* gh-124878: Fix race conditions during interpreter finalization (#130649)Sam Gross2025-03-061-10/+9
| | | | | | | | | | | | | | | | | | | | The PyThreadState field gains a reference count field to avoid issues with PyThreadState being a dangling pointer to freed memory. The refcount starts with a value of two: one reference is owned by the interpreter's linked list of thread states and one reference is owned by the OS thread. The reference count is decremented when the thread state is removed from the interpreter's linked list and before the OS thread calls `PyThread_hang_thread()`. The thread that decrements it to zero frees the `PyThreadState` memory. The `holds_gil` field is moved out of the `_status` bit field, to avoid a data race where on thread calls `PyThreadState_Clear()`, modifying the `_status` bit field while the OS thread reads `holds_gil` when attempting to acquire the GIL. The `PyThreadState.state` field now has `_Py_THREAD_SHUTTING_DOWN` as a possible value. This corresponds to the `_PyThreadState_MustExit()` check. This avoids race conditions in the free threading build when checking `_PyThreadState_MustExit()`.
* gh-130605: Use relaxed atomics to set the GIL switch interval (gh-130654)Sam Gross2025-02-281-3/+6
| | | | The interval may be concurrently read by a thread attempting to acquire the GIL.
* gh-128360: Add `_Py_AssertHoldsTstate` as assertion for holding a thread ↵Peter Bierma2025-01-201-2/+2
| | | | | state (#128361) Co-authored-by: Kumar Aditya <kumaraditya@python.org>
* gh-114940: Add _Py_FOR_EACH_TSTATE_UNLOCKED(), and Friends (gh-127077)Eric Snow2024-11-211-10/+4
| | | This is a precursor to the actual fix for gh-114940, where we will change these macros to use the new lock. This change is almost entirely mechanical; the exceptions are the loops in codeobject.c and ceval.c, which now hold the "head" lock. Note that almost all of the uses of _Py_FOR_EACH_TSTATE_UNLOCKED() here will change to _Py_FOR_EACH_TSTATE_BEGIN() once we add the new per-interpreter lock.
* gh-87135: Hang non-main threads that attempt to acquire the GIL during ↵Jeremy Maitin-Shepard2024-10-021-9/+17
| | | | | | | finalization (GH-105805) Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option. Co-authored-by: Gregory P. Smith <greg@krypto.org>
* GH-123516: Improve JIT memory consumption by invalidating cold executors ↵Savannah Ostrowski2024-09-271-0/+6
| | | | | (GH-124443) Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* gh-122860: Remove unused macro `_Py_atomic_load_relaxed_int32` (#122861)Sam Gross2024-08-111-7/+0
|
* gh-122201: Lock mutex when setting handling_thread to NULL (#122204)Sam Gross2024-07-261-3/+15
| | | | | In the free-threaded build, we need to lock pending->mutex when clearing the handling_thread in order not to race with a concurrent make_pending_calls in the same interpreter.
* gh-118297: Make Sure All Pending Calls Run in _Py_FinishPendingCalls() ↵Eric Snow2024-07-151-6/+28
| | | | (gh-118298)
* gh-118727: Don't drop the GIL in `drop_gil()` unless the current thread ↵Brett Simmers2024-05-231-39/+57
| | | | | | | | | | | | | | | | | holds it (#118745) `drop_gil()` assumes that its caller is attached, which means that the current thread holds the GIL if and only if the GIL is enabled, and the enabled-state of the GIL won't change. This isn't true, though, because `detach_thread()` calls `_PyEval_ReleaseLock()` after detaching and `_PyThreadState_DeleteCurrent()` calls it after removing the current thread from consideration for stop-the-world requests (effectively detaching it). Fix this by remembering whether or not a thread acquired the GIL when it last attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()` instead of `gil->enabled`. This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've reenabled it.
* gh-116322: Enable the GIL while loading C extension modules (#118560)Brett Simmers2024-05-071-13/+145
| | | | | | | | | | Add the ability to enable/disable the GIL at runtime, and use that in the C module loading code. We can't know before running a module init function if it supports free-threading, so the GIL is temporarily enabled before doing so. If the module declares support for running without the GIL, the GIL is later disabled. Otherwise, the GIL is permanently enabled, and will never be disabled again for the life of the current interpreter.
* gh-118534: Fix load of `gil->locked` (#118553)Sam Gross2024-05-031-1/+1
|
* GH-118095: Make sure that progress is made if there are pending calls being ↵Mark Shannon2024-05-011-11/+10
| | | | handled. (GH-118484)
* gh-116749: Disable GIL by default in free-threaded build (#118295)Sam Gross2024-04-261-3/+1
| | | | Switch GIL to disabled by default in free-threaded build so that the free-threaded CIs catch thread-safety issues.
* gh-110693: Pending Calls Machinery Cleanups (gh-118296)Eric Snow2024-04-261-55/+104
| | | This does some cleanup in preparation for later changes.
* gh-117929: Restore removed PyEval_InitThreads() function (#117931)Victor Stinner2024-04-171-2/+1
|
* gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)Sam Gross2024-03-211-5/+2
| | | | | | | | | | | Split `_PyThreadState_DeleteExcept` into two functions: - `_PyThreadState_RemoveExcept` removes all thread states other than one passed as an argument. It returns the removed thread states as a linked list. - `_PyThreadState_DeleteList` deletes those dead thread states. It may call destructors, so we want to "start the world" before calling `_PyThreadState_DeleteList` to avoid potential deadlocks.
* gh-116908: Only write to `_pending_calls.calls_to_do` with atomic operations ↵Brett Simmers2024-03-201-2/+2
| | | | | | (#117044) These writes to `pending->calls_to_do` need to be atomic, because other threads can read (atomically) from `calls_to_do` without holding `pending->mutex`.
* gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338)Brett Simmers2024-03-111-0/+15
| | | | | | | | | In free-threaded builds, running with `PYTHON_GIL=0` will now disable the GIL. Follow-up issues track work to re-enable the GIL when loading an incompatible extension, and to disable the GIL by default. In order to support re-enabling the GIL at runtime, all GIL-related data structures are initialized as usual, and disabling the GIL simply sets a flag that causes `take_gil()` and `drop_gil()` to return early.
* gh-116590: Fix unused `current_thread_holds_gil` function warning (#116591)Nikita Sobolev2024-03-111-0/+2
|
* gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)Brett Simmers2024-02-201-93/+150
| | | | | | | | | | | This change adds an `eval_breaker` field to `PyThreadState`. The primary motivation is for performance in free-threaded builds: with thread-local eval breakers, we can stop a specific thread (e.g., for an async exception) without interrupting other threads. The source of truth for the global instrumentation version is stored in the `instrumentation_version` field in PyInterpreterState. Threads usually read the version from their local `eval_breaker`, where it continues to be colocated with the eval breaker bits.
* gh-110481: Implement inter-thread queue for biased reference counting (#114824)Sam Gross2024-02-091-0/+8
| | | | | | | | | Biased reference counting maintains two refcount fields in each object: `ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two fields. In some cases, when refcounting operations are split across threads, the ob_ref_shared field can be negative (although the total refcount must be at least zero). In this case, the thread that decremented the refcount requests that the owning thread give up ownership and merge the refcount fields.
* gh-104530: Enable native Win32 condition variables by default (GH-104531)Andrew Rogers2024-02-021-0/+8
|
* gh-111964: Implement stop-the-world pauses (gh-112471)Sam Gross2024-01-231-0/+9
| | | | | | | | | | | | | | | | | The `--disable-gil` builds occasionally need to pause all but one thread. Some examples include: * Cyclic garbage collection, where this is often called a "stop the world event" * Before calling `fork()`, to ensure a consistent state for internal data structures * During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects This adds the following functions to implement global and per-interpreter pauses: * `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime) * `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter) (The function names may change.) These functions are no-ops outside of the `--disable-gil` build.
* gh-112723: Call `PyThreadState_Clear()` from the correct interpreter (#112776)Sam Gross2023-12-131-3/+1
| | | | | | | | | | | | | The `PyThreadState_Clear()` function must only be called with the GIL held and must be called from the same interpreter as the passed in thread state. Otherwise, any Python objects on the thread state may be destroyed using the wrong interpreter, leading to memory corruption. This is also important for `Py_GIL_DISABLED` builds because free lists will be associated with PyThreadStates and cleared in `PyThreadState_Clear()`. This fixes two places that called `PyThreadState_Clear()` from the wrong interpreter and adds an assertion to `PyThreadState_Clear()`.
* gh-112978: Remove redundant condition inside `take_gil` (gh-112979)Yan Yanchii2023-12-111-5/+0
|
* gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)Sam Gross2023-12-071-28/+9
| | | | | This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize. This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
* gh-109693: Remove pycore_atomic.h (gh-110992)Donghee Na2023-10-171-1/+0
|
* gh-109693: Update _gil_runtime_state.locked to use pyatomic.h (gh-110836)Donghee Na2023-10-161-16/+11
|
* gh-109693: Update _gil_runtime_state.last_holder to use pyatomic.h (#110605)Donghee Na2023-10-131-7/+7
|
* gh-109693: Use pyatomic.h for signal module (gh-110480)Donghee Na2023-10-091-1/+1
|
* gh-76785: Add SendChannel.send_buffer() (#110246)Eric Snow2023-10-091-12/+19
| | | (This is still a test module.)
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-051-36/+11
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* GH-109369: Merge all eval-breaker flags and monitoring version into one ↵Mark Shannon2023-10-041-179/+72
| | | | word. (GH-109846)
* gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)Eric Snow2023-09-191-4/+4
| | | This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)Victor Stinner2023-09-111-25/+3
| | | | | | | | | | | | | Fix _thread.start_new_thread() race condition. If a thread is created during Python finalization, the newly spawned thread now exits immediately instead of trying to access freed memory and lead to a crash. thread_run() calls PyEval_AcquireThread() which checks if the thread must exit. The problem was that tstate was dereferenced earlier in _PyThreadState_Bind() which leads to a crash most of the time. Move _PyThreadState_CheckConsistency() from thread_run() to _PyThreadState_Bind().
* gh-104690: thread_run() checks for tstate dangling pointer (#109056)Victor Stinner2023-09-081-18/+8
| | | | | | | | thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
* gh-108753: Enhance pystats (#108754)Victor Stinner2023-09-061-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Statistics gathering is now off by default. Use the "-X pystats" command line option or set the new PYTHONSTATS environment variable to 1 to turn statistics gathering on at Python startup. Statistics are no longer dumped at exit if statistics gathering was off or statistics have been cleared. Changes: * Add PYTHONSTATS environment variable. * sys._stats_dump() now returns False if statistics are not dumped because they are all equal to zero. * Add PyConfig._pystats member. * Add tests on sys functions and on setting PyConfig._pystats to 1. * Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h header files. * Rename '_py_stats' variable to '_Py_stats'. * Exclude Include/cpython/pystats.h from the Py_LIMITED_API. * Move pystats.h include from object.h to Python.h. * Add _Py_StatsOn() and _Py_StatsOff() functions. Remove '_py_stats_struct' variable from the API: make it static in specialize.c. * Document API in Include/pystats.h and Include/cpython/pystats.h. * Complete pystats documentation in Doc/using/configure.rst. * Don't write "all zeros" stats: if _stats_off() and _stats_clear() or _stats_dump() were called. * _PyEval_Fini() now always call _Py_PrintSpecializationStats() which does nothing if stats are all zeros. Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* GH-104584: Fix ENTER_EXECUTOR (GH-106141)Mark Shannon2023-07-031-2/+59
| | | | | | * Check eval-breaker in ENTER_EXECUTOR. * Make sure that frame->prev_instr is set before entering executor.
* gh-106320: Remove _PyInterpreterState_Get() alias (#106321)Victor Stinner2023-07-011-2/+2
| | | | Replace calls to the (removed) slow _PyInterpreterState_Get() with fast inlined _PyInterpreterState_GET() function.
* gh-104812: Run Pending Calls in any Thread (gh-104813)Eric Snow2023-06-131-82/+131
| | | For a while now, pending calls only run in the main thread (in the main interpreter). This PR changes things to allow any thread run a pending call, unless the pending call was explicitly added for the main thread to run.
* gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current ↵Eric Snow2023-06-011-4/+26
| | | | | | | Thread (gh-105109) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.)
* gh-105182: Remove PyEval_AcquireLock() and PyEval_InitThreads() (#105183)Victor Stinner2023-06-011-4/+9
| | | | | | | | | | | | Remove functions in the C API: * PyEval_AcquireLock() * PyEval_ReleaseLock() * PyEval_InitThreads() * PyEval_ThreadsInitialized() But keep these functions in the stable ABI. Mention "make regen-limited-abi" in "make regen-all".
* gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization ↵Eric Snow2023-05-151-0/+3
| | | | | (gh-104437) With the move to a per-interpreter GIL, this check slipped through the cracks.
generated by cgit v0.12 at 2026-01-16 19:30:48 (GMT)