summaryrefslogtreecommitdiffstats
path: root/Python/ceval_gil.c
Commit message (Collapse)AuthorAgeFilesLines
* gh-118727: Don't drop the GIL in `drop_gil()` unless the current thread ↵Brett Simmers2024-05-231-39/+57
| | | | | | | | | | | | | | | | | holds it (#118745) `drop_gil()` assumes that its caller is attached, which means that the current thread holds the GIL if and only if the GIL is enabled, and the enabled-state of the GIL won't change. This isn't true, though, because `detach_thread()` calls `_PyEval_ReleaseLock()` after detaching and `_PyThreadState_DeleteCurrent()` calls it after removing the current thread from consideration for stop-the-world requests (effectively detaching it). Fix this by remembering whether or not a thread acquired the GIL when it last attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()` instead of `gil->enabled`. This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've reenabled it.
* gh-116322: Enable the GIL while loading C extension modules (#118560)Brett Simmers2024-05-071-13/+145
| | | | | | | | | | Add the ability to enable/disable the GIL at runtime, and use that in the C module loading code. We can't know before running a module init function if it supports free-threading, so the GIL is temporarily enabled before doing so. If the module declares support for running without the GIL, the GIL is later disabled. Otherwise, the GIL is permanently enabled, and will never be disabled again for the life of the current interpreter.
* gh-118534: Fix load of `gil->locked` (#118553)Sam Gross2024-05-031-1/+1
|
* GH-118095: Make sure that progress is made if there are pending calls being ↵Mark Shannon2024-05-011-11/+10
| | | | handled. (GH-118484)
* gh-116749: Disable GIL by default in free-threaded build (#118295)Sam Gross2024-04-261-3/+1
| | | | Switch GIL to disabled by default in free-threaded build so that the free-threaded CIs catch thread-safety issues.
* gh-110693: Pending Calls Machinery Cleanups (gh-118296)Eric Snow2024-04-261-55/+104
| | | This does some cleanup in preparation for later changes.
* gh-117929: Restore removed PyEval_InitThreads() function (#117931)Victor Stinner2024-04-171-2/+1
|
* gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)Sam Gross2024-03-211-5/+2
| | | | | | | | | | | Split `_PyThreadState_DeleteExcept` into two functions: - `_PyThreadState_RemoveExcept` removes all thread states other than one passed as an argument. It returns the removed thread states as a linked list. - `_PyThreadState_DeleteList` deletes those dead thread states. It may call destructors, so we want to "start the world" before calling `_PyThreadState_DeleteList` to avoid potential deadlocks.
* gh-116908: Only write to `_pending_calls.calls_to_do` with atomic operations ↵Brett Simmers2024-03-201-2/+2
| | | | | | (#117044) These writes to `pending->calls_to_do` need to be atomic, because other threads can read (atomically) from `calls_to_do` without holding `pending->mutex`.
* gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338)Brett Simmers2024-03-111-0/+15
| | | | | | | | | In free-threaded builds, running with `PYTHON_GIL=0` will now disable the GIL. Follow-up issues track work to re-enable the GIL when loading an incompatible extension, and to disable the GIL by default. In order to support re-enabling the GIL at runtime, all GIL-related data structures are initialized as usual, and disabling the GIL simply sets a flag that causes `take_gil()` and `drop_gil()` to return early.
* gh-116590: Fix unused `current_thread_holds_gil` function warning (#116591)Nikita Sobolev2024-03-111-0/+2
|
* gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)Brett Simmers2024-02-201-93/+150
| | | | | | | | | | | This change adds an `eval_breaker` field to `PyThreadState`. The primary motivation is for performance in free-threaded builds: with thread-local eval breakers, we can stop a specific thread (e.g., for an async exception) without interrupting other threads. The source of truth for the global instrumentation version is stored in the `instrumentation_version` field in PyInterpreterState. Threads usually read the version from their local `eval_breaker`, where it continues to be colocated with the eval breaker bits.
* gh-110481: Implement inter-thread queue for biased reference counting (#114824)Sam Gross2024-02-091-0/+8
| | | | | | | | | Biased reference counting maintains two refcount fields in each object: `ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two fields. In some cases, when refcounting operations are split across threads, the ob_ref_shared field can be negative (although the total refcount must be at least zero). In this case, the thread that decremented the refcount requests that the owning thread give up ownership and merge the refcount fields.
* gh-104530: Enable native Win32 condition variables by default (GH-104531)Andrew Rogers2024-02-021-0/+8
|
* gh-111964: Implement stop-the-world pauses (gh-112471)Sam Gross2024-01-231-0/+9
| | | | | | | | | | | | | | | | | The `--disable-gil` builds occasionally need to pause all but one thread. Some examples include: * Cyclic garbage collection, where this is often called a "stop the world event" * Before calling `fork()`, to ensure a consistent state for internal data structures * During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects This adds the following functions to implement global and per-interpreter pauses: * `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime) * `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter) (The function names may change.) These functions are no-ops outside of the `--disable-gil` build.
* gh-112723: Call `PyThreadState_Clear()` from the correct interpreter (#112776)Sam Gross2023-12-131-3/+1
| | | | | | | | | | | | | The `PyThreadState_Clear()` function must only be called with the GIL held and must be called from the same interpreter as the passed in thread state. Otherwise, any Python objects on the thread state may be destroyed using the wrong interpreter, leading to memory corruption. This is also important for `Py_GIL_DISABLED` builds because free lists will be associated with PyThreadStates and cleared in `PyThreadState_Clear()`. This fixes two places that called `PyThreadState_Clear()` from the wrong interpreter and adds an assertion to `PyThreadState_Clear()`.
* gh-112978: Remove redundant condition inside `take_gil` (gh-112979)Yan Yanchii2023-12-111-5/+0
|
* gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)Sam Gross2023-12-071-28/+9
| | | | | This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize. This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
* gh-109693: Remove pycore_atomic.h (gh-110992)Donghee Na2023-10-171-1/+0
|
* gh-109693: Update _gil_runtime_state.locked to use pyatomic.h (gh-110836)Donghee Na2023-10-161-16/+11
|
* gh-109693: Update _gil_runtime_state.last_holder to use pyatomic.h (#110605)Donghee Na2023-10-131-7/+7
|
* gh-109693: Use pyatomic.h for signal module (gh-110480)Donghee Na2023-10-091-1/+1
|
* gh-76785: Add SendChannel.send_buffer() (#110246)Eric Snow2023-10-091-12/+19
| | | (This is still a test module.)
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-051-36/+11
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* GH-109369: Merge all eval-breaker flags and monitoring version into one ↵Mark Shannon2023-10-041-179/+72
| | | | word. (GH-109846)
* gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)Eric Snow2023-09-191-4/+4
| | | This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)Victor Stinner2023-09-111-25/+3
| | | | | | | | | | | | | Fix _thread.start_new_thread() race condition. If a thread is created during Python finalization, the newly spawned thread now exits immediately instead of trying to access freed memory and lead to a crash. thread_run() calls PyEval_AcquireThread() which checks if the thread must exit. The problem was that tstate was dereferenced earlier in _PyThreadState_Bind() which leads to a crash most of the time. Move _PyThreadState_CheckConsistency() from thread_run() to _PyThreadState_Bind().
* gh-104690: thread_run() checks for tstate dangling pointer (#109056)Victor Stinner2023-09-081-18/+8
| | | | | | | | thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
* gh-108753: Enhance pystats (#108754)Victor Stinner2023-09-061-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Statistics gathering is now off by default. Use the "-X pystats" command line option or set the new PYTHONSTATS environment variable to 1 to turn statistics gathering on at Python startup. Statistics are no longer dumped at exit if statistics gathering was off or statistics have been cleared. Changes: * Add PYTHONSTATS environment variable. * sys._stats_dump() now returns False if statistics are not dumped because they are all equal to zero. * Add PyConfig._pystats member. * Add tests on sys functions and on setting PyConfig._pystats to 1. * Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h header files. * Rename '_py_stats' variable to '_Py_stats'. * Exclude Include/cpython/pystats.h from the Py_LIMITED_API. * Move pystats.h include from object.h to Python.h. * Add _Py_StatsOn() and _Py_StatsOff() functions. Remove '_py_stats_struct' variable from the API: make it static in specialize.c. * Document API in Include/pystats.h and Include/cpython/pystats.h. * Complete pystats documentation in Doc/using/configure.rst. * Don't write "all zeros" stats: if _stats_off() and _stats_clear() or _stats_dump() were called. * _PyEval_Fini() now always call _Py_PrintSpecializationStats() which does nothing if stats are all zeros. Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* GH-104584: Fix ENTER_EXECUTOR (GH-106141)Mark Shannon2023-07-031-2/+59
| | | | | | * Check eval-breaker in ENTER_EXECUTOR. * Make sure that frame->prev_instr is set before entering executor.
* gh-106320: Remove _PyInterpreterState_Get() alias (#106321)Victor Stinner2023-07-011-2/+2
| | | | Replace calls to the (removed) slow _PyInterpreterState_Get() with fast inlined _PyInterpreterState_GET() function.
* gh-104812: Run Pending Calls in any Thread (gh-104813)Eric Snow2023-06-131-82/+131
| | | For a while now, pending calls only run in the main thread (in the main interpreter). This PR changes things to allow any thread run a pending call, unless the pending call was explicitly added for the main thread to run.
* gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current ↵Eric Snow2023-06-011-4/+26
| | | | | | | Thread (gh-105109) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.)
* gh-105182: Remove PyEval_AcquireLock() and PyEval_InitThreads() (#105183)Victor Stinner2023-06-011-4/+9
| | | | | | | | | | | | Remove functions in the C API: * PyEval_AcquireLock() * PyEval_ReleaseLock() * PyEval_InitThreads() * PyEval_ThreadsInitialized() But keep these functions in the stable ABI. Mention "make regen-limited-abi" in "make regen-all".
* gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization ↵Eric Snow2023-05-151-0/+3
| | | | | (gh-104437) With the move to a per-interpreter GIL, this check slipped through the cracks.
* gh-99113: A Per-Interpreter GIL! (gh-104210)Eric Snow2023-05-081-42/+13
| | | | | This is the culmination of PEP 684 (and of my 8-year long multi-core Python project)! Each subinterpreter may now be created with its own GIL (via Py_NewInterpreterFromConfig()). If not so configured then the interpreter will share with the main interpreter--the status quo since subinterpreters were added decades ago. The main interpreter always has its own GIL and subinterpreters from Py_NewInterpreter() will always share with the main interpreter.
* gh-99113: Make Sure the GIL is Acquired at the Right Places (gh-104208)Eric Snow2023-05-061-27/+57
| | | This is a pre-requisite for a per-interpreter GIL. Without it this change isn't strictly necessary. However, there is no real downside otherwise.
* gh-104233: Fix "unused variable" warning in `ceval_gil.c` (#104234)Nikita Sobolev2023-05-061-0/+2
|
* gh-99113: Add PyInterpreterConfig.own_gil (gh-104204)Eric Snow2023-05-051-1/+22
| | | | | We also add PyInterpreterState.ceval.own_gil to record if the interpreter actually has its own GIL. Note that for now we don't actually respect own_gil; all interpreters still share the one GIL. However, PyInterpreterState.ceval.own_gil does reflect PyInterpreterConfig.own_gil. That lie is a temporary one that we will fix when the GIL really becomes per-interpreter.
* gh-99113: Share the GIL via PyInterpreterState.ceval.gil (gh-104203)Eric Snow2023-05-051-38/+59
| | | In preparation for a per-interpreter GIL, we add PyInterpreterState.ceval.gil, set it to the shared GIL for each interpreter, and use that rather than using _PyRuntime.ceval.gil directly. Note that _PyRuntime.ceval.gil is still the actual GIL.
* gh-103323: Remove PyRuntimeState_GetThreadState() (#104171)Victor Stinner2023-05-041-3/+2
| | | | | This function no longer makes sense, since its runtime parameter is no longer used. Use directly _PyThreadState_GET() and _PyInterpreterState_GET() instead.
* gh-103167: Fix `-Wstrict-prototypes` warnings by using `(void)` for ↵Nikita Sobolev2023-04-051-1/+1
| | | | functions with no args (GH-103168)
* gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives ↵Irit Katriel2023-02-281-4/+3
| | | | (in Python/) (#102193)
* gh-59956: Clarify Runtime State Status Expectations (gh-101308)Eric Snow2023-01-301-1/+1
| | | | | | | | | | | | | A PyThreadState can be in one of many states in its lifecycle, represented by some status value. Those statuses haven't been particularly clear, so we're addressing that here. Specifically: * made the distinct lifecycle statuses clear on PyThreadState * identified expectations of how various lifecycle-related functions relate to status * noted the various places where those expectations don't match the actual behavior At some point we'll need to address the mismatches. (This change also includes some cleanup.) https://github.com/python/cpython/issues/59956
* gh-59956: Clarify GILState-related Code (gh-101161)Eric Snow2023-01-191-8/+6
| | | | | | | | | | The objective of this change is to help make the GILState-related code easier to understand. This mostly involves moving code around and some semantically equivalent refactors. However, there are a also a small number of slight changes in structure and behavior: * tstate_current is moved out of _PyRuntimeState.gilstate * autoTSSkey is moved out of _PyRuntimeState.gilstate * autoTSSkey is initialized earlier * autoTSSkey is re-initialized (after fork) earlier https://github.com/python/cpython/issues/59956
* gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)Eric Snow2022-11-161-5/+4
| | | https://github.com/python/cpython/issues/81057
* gh-97922: Run the GC only on eval breaker (#97920)Pablo Galindo Salgado2022-10-081-10/+20
|
* gh-96387: take_gil() resets drop request before exit (#96869)Victor Stinner2022-09-191-0/+11
| | | | | | | | | | | | | At Python exit, sometimes a thread holding the GIL can wait forever for a thread (usually a daemon thread) which requested to drop the GIL, whereas the thread already exited. To fix the race condition, the thread which requested the GIL drop now resets its request before exiting. take_gil() now calls RESET_GIL_DROP_REQUEST() before PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a race condition with drop_gil(). Issue discovered and analyzed by Mingliang ZHAO.
* GH-96177: Move GIL and eval breaker code out of ceval.c into ceval_gil.c. ↵Mark Shannon2022-08-241-0/+986
(GH-96204)
generated by cgit v0.12 at 2025-06-13 20:51:59 (GMT)