summaryrefslogtreecommitdiffstats
path: root/Python/pystate.c
Commit message (Collapse)AuthorAgeFilesLines
* Remove comment from pystate created in 2003 (#123259)Anthony Shaw2024-08-241-5/+0
|
* Add debug offsets for free threaded builds (#123041)Pablo Galindo Salgado2024-08-151-1/+3
|
* gh-122697: Fix free-threading memory leaks at shutdown (#122703)Sam Gross2024-08-081-1/+1
| | | | | | | | | | | | | | | We were not properly accounting for interpreter memory leaks at shutdown and had two sources of leaks: * Objects that use deferred reference counting and were reachable via static types outlive the final GC. We now disable deferred reference counting on all objects if we are calling the GC due to interpreter shutdown. * `_PyMem_FreeDelayed` did not properly check for interpreter shutdown so we had some memory blocks that were enqueued to be freed, but never actually freed. * `_PyType_FinalizeIdPool` wasn't called at interpreter shutdown.
* gh-122417: Implement per-thread heap type refcounts (#122418)Sam Gross2024-08-061-7/+6
| | | | | | | The free-threaded build partially stores heap type reference counts in distributed manner in per-thread arrays. This avoids reference count contention when creating or destroying instances. Co-authored-by: Ken Jin <kenjin@python.org>
* gh-100240: Use a consistent implementation for freelists (#121934)Sam Gross2024-07-221-2/+2
| | | | | | | | This combines and updates our freelist handling to use a consistent implementation. Objects in the freelist are linked together using the first word of memory block. If configured with freelists disabled, these operations are essentially no-ops.
* gh-120973: Fix thread-safety issues with `threading.local` (#121655)mpage2024-07-191-0/+3
| | | | | | This is a small refactoring to the current design that allows us to avoid manually iterating over threads. This should also fix gh-118490.
* gh-121621: Move asyncio_running_loop to private struct (#121939)Sam Gross2024-07-171-2/+2
| | | | This avoids changing the ABI and keeps the field in the private struct.
* gh-121621: Move asyncio running loop to thread state (GH-121695)Ken Jin2024-07-161-0/+4
|
* gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121010)Eric Snow2024-06-251-5/+9
| | | | | We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence(). This change only affects internal API.
* gh-120726: Fix compiler warnings on is_core_module() (#120727)Kirill Podoprigora2024-06-191-3/+4
| | | | | Fix compiler warnings on is_core_module() and check_interpreter_whence(): only define them when assertions are built.
* gh-117657: Fix race involving GC and heap initialization (#119923)Sam Gross2024-06-041-0/+2
| | | | | | | | | | | | The `_PyThreadState_Bind()` function is called before the first `PyEval_AcquireThread()` so it's not synchronized with the stop the world GC. We had a race where `gc_visit_heaps()` might visit a thread's heap while it's being initialized. Use a simple atomic int to avoid visiting heaps for threads that are not yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called). The race was reproducible by running: `python Lib/test/test_importlib/partial/pool_in_threads.py`.
* gh-117657: Fix race involving immortalizing objects (#119927)Sam Gross2024-06-031-3/+1
| | | | | | | | | The free-threaded build currently immortalizes objects that use deferred reference counting (see gh-117783). This typically happens once the first non-main thread is created, but the behavior can be suppressed for tests, in subinterpreters, or during a compile() call. This fixes a race condition involving the tracking of whether the behavior is suppressed.
* gh-119369: Fix deadlock during thread exit in free-threaded build (#119528)Sam Gross2024-05-311-9/+12
| | | | | | | Release the GIL before calling `_Py_qsbr_unregister`. The deadlock could occur when the GIL was enabled at runtime. The `_Py_qsbr_unregister` call might block while holding the GIL because the thread state was not active, but the GIL was still held.
* gh-119585: Fix crash involving `PyGILState_Release()` and ↵Sam Gross2024-05-311-0/+6
| | | | | | | | | | `PyThreadState_Clear()` (#119753) Make sure that `gilstate_counter` is not zero in when calling `PyThreadState_Clear()`. A destructor called from `PyThreadState_Clear()` may call back into `PyGILState_Ensure()` and `PyGILState_Release()`. If `gilstate_counter` is zero, it will try to create a new thread state before the current active thread state is destroyed, leading to an assertion failure or crash.
* gh-118727: Don't drop the GIL in `drop_gil()` unless the current thread ↵Brett Simmers2024-05-231-7/+4
| | | | | | | | | | | | | | | | | holds it (#118745) `drop_gil()` assumes that its caller is attached, which means that the current thread holds the GIL if and only if the GIL is enabled, and the enabled-state of the GIL won't change. This isn't true, though, because `detach_thread()` calls `_PyEval_ReleaseLock()` after detaching and `_PyThreadState_DeleteCurrent()` calls it after removing the current thread from consideration for stop-the-world requests (effectively detaching it). Fix this by remembering whether or not a thread acquired the GIL when it last attached, in `PyThreadState._status.holds_gil`, and check this in `drop_gil()` instead of `gil->enabled`. This fixes a crash in `test_multiprocessing_pool_circular_import()`, so I've reenabled it.
* gh-117657: Fix QSBR race condition (#118843)Alex Turner2024-05-101-1/+1
| | | | | | `_Py_qsbr_unregister` is called when the PyThreadState is already detached, so the access to `tstate->qsbr` isn't safe without locking the shared mutex. Grab the `struct _qsbr_shared` from the interpreter instead.
* gh-117657: Fix data races reported by TSAN on `interp->threads.main` (#118865)mpage2024-05-101-11/+20
| | | Use relaxed loads/stores when reading/writing to this field.
* gh-116322: Enable the GIL while loading C extension modules (#118560)Brett Simmers2024-05-071-8/+25
| | | | | | | | | | Add the ability to enable/disable the GIL at runtime, and use that in the C module loading code. We can't know before running a module init function if it supports free-threading, so the GIL is temporarily enabled before doing so. If the module declares support for running without the GIL, the GIL is later disabled. Otherwise, the GIL is permanently enabled, and will never be disabled again for the life of the current interpreter.
* gh-112075: use per-thread dict version pool (#118676)Dino Viehland2024-05-071-0/+1
| | | use thread state set of dict versions
* gh-118527: Intern code consts in free-threaded build (#118667)Sam Gross2024-05-071-0/+1
| | | | | | We already intern and immortalize most string constants. In the free-threaded build, other constants can be a source of reference count contention because they are shared by all threads running the same code objects.
* gh-116738: Make `_codecs` module thread-safe (#117530)Brett Simmers2024-05-021-3/+1
| | | | | | | | | | | | | | | The module itself is a thin wrapper around calls to functions in `Python/codecs.c`, so that's where the meaningful changes happened: - Move codecs-related state that lives on `PyInterpreterState` to a struct declared in `pycore_codecs.h`. - In free-threaded builds, add a mutex to `codecs_state` to synchronize operations on `search_path`. Because `search_path_mutex` is used as a normal mutex and not a critical section, we must be extremely careful with operations called while holding it. - The codec registry is explicitly initialized as part of `_PyUnicode_InitEncodings` to simplify thread-safety.
* gh-118335: Configure Tier 2 interpreter at build time (#118339)Guido van Rossum2024-05-011-0/+6
| | | | | | | | | | | | | | | | | | | | | | The code for Tier 2 is now only compiled when configured with `--enable-experimental-jit[=yes|interpreter]`. We drop support for `PYTHON_UOPS` and -`Xuops`, but you can disable the interpreter or JIT at runtime by setting `PYTHON_JIT=0`. You can also build it without enabling it by default using `--enable-experimental-jit=yes-off`; enable with `PYTHON_JIT=1`. On Windows, the `build.bat` script supports `--experimental-jit`, `--experimental-jit-off`, `--experimental-interpreter`. In the C code, `_Py_JIT` is defined as before when the JIT is enabled; the new variable `_Py_TIER2` is defined when the JIT *or* the interpreter is enabled. It is actually a bitmask: 1: JIT; 2: default-off; 4: interpreter.
* gh-118332: Fix deadlock involving stop the world (#118412)Sam Gross2024-04-301-1/+2
| | | | | | Avoid detaching thread state when stopping the world. When re-attaching the thread state, the thread would attempt to resume the top-most critical section, which might now be held by a thread paused for our stop-the-world request.
* gh-117783: Immortalize objects that use deferred reference counting (#118112)Sam Gross2024-04-291-0/+11
| | | | | | | | | Deferred reference counting is not fully implemented yet. As a temporary measure, we immortalize objects that would use deferred reference counting to avoid multi-threaded scaling bottlenecks. This is only performed in the free-threaded build once the first non-main thread is started. Additionally, some tests, including refleak tests, suppress this behavior.
* gh-117657: Quiet TSAN warnings about remaining non-atomic accesses of ↵mpage2024-04-231-1/+1
| | | | | `tstate->state` (#118165) Quiet TSAN warnings about remaining non-atomic accesses of `tstate->state`
* gh-116818: Make `sys.settrace`, `sys.setprofile`, and monitoring thread-safe ↵Dino Viehland2024-04-191-0/+1
| | | | | | | (#116775) Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe. Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version. There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
* GH-117760: Streamline the trashcan mechanism (GH-117763)Mark Shannon2024-04-171-0/+2
|
* gh-117657: Quiet more TSAN warnings due to incorrect modeling of ↵mpage2024-04-151-2/+2
| | | | compare/exchange (#117830)
* gh-117657: Quiet TSAN warning about a data race between `start_the_world()` ↵mpage2024-04-151-1/+2
| | | | | | | | and `tstate_try_attach()` (#117828) TSAN erroneously reports a data race between the `_Py_atomic_compare_exchange_int` on `tstate->state` in `tstate_try_attach()` and the non-atomic load of `tstate->state` in `start_the_world`. The `_Py_atomic_compare_exchange_int` fails, but TSAN erroneously treats it as a store.
* gh-76785: Handle Legacy Interpreters Properly (gh-117490)Eric Snow2024-04-111-0/+7
| | | This is similar to the situation with threading._DummyThread. The methods (incl. __del__()) of interpreters.Interpreter objects must be careful with interpreters not created by interpreters.create(). The simplest thing to start with is to disable any method that modifies or runs in the interpreter. As part of this, the runtime keeps track of where an interpreter was created. We also handle interpreter "refcounts" properly.
* gh-76785: Add More Tests to test_interpreters.test_api (gh-117662)Eric Snow2024-04-111-2/+53
| | | In addition to the increase test coverage, this is a precursor to sorting out how we handle interpreters created directly via the C-API.
* gh-117439: Make refleak checking thread-safe without the GIL (#117469)Sam Gross2024-04-081-0/+8
| | | | | This keeps track of the per-thread total reference count operations in PyThreadState in the free-threaded builds. The count is merged into the interpreter's total when the thread exits.
* gh-111926: Make weakrefs thread-safe in free-threaded builds (#117168)mpage2024-04-081-0/+9
| | | | | | | | | Most mutable data is protected by a striped lock that is keyed on the referenced object's address. The weakref's hash is protected using the weakref's per-object lock. Note that this only affects free-threaded builds. Apart from some minor refactoring, the added code is all either gated by `ifdef`s or is a no-op (e.g. `Py_BEGIN_CRITICAL_SECTION`).
* gh-76785: Raise InterpreterError, Not RuntimeError (gh-117489)Eric Snow2024-04-031-1/+1
| | | | | I had meant to switch everything to InterpreterError when I added it a while back. At the time I missed a few key spots. As part of this, I've added print-the-exception to _PyXI_InitTypes() and fixed an error case in `_PyStaticType_InitBuiltin().
* gh-117303: Don't detach in `PyThreadState_DeleteCurrent()` (#117304)Sam Gross2024-03-291-2/+0
| | | | | | | | | | | This fixes a crash in `test_threading.test_reinit_tls_after_fork()` when running with the GIL disabled. We already properly handle the case where the thread state is `_Py_THREAD_ATTACHED` in `tstate_delete_common()` -- we just need to remove an assertion. Keeping the thread attached means that a stop-the-world pause, such as for a `fork()`, won't commence until we remove our thread state from the interpreter's linked list. This prevents a crash when the child process tries to clean up the dead thread states.
* gh-117300: Use stop the world to make `sys._current_frames` and ↵Sam Gross2024-03-291-0/+4
| | | | | | | | | | `sys._current_exceptions` thread-safe. (#117301) This adds a stop the world pause to make the two functions thread-safe when the GIL is disabled in the free-threaded build. Additionally, the main test thread may call `sys._current_exceptions()` as soon as `g_raised.set()` is called. The background thread may not yet reach the `leave_g.wait()` line.
* gh-110481: Fix biased reference counting queue initialization. (#117271)Sam Gross2024-03-281-4/+6
| | | | | The biased reference counting queue must be initialized from the bound (active) thread because it uses `_Py_ThreadId()` as the key in a hash table.
* gh-105716: Fix _PyInterpreterState_IsRunningMain() For Embedders (gh-117140)Eric Snow2024-03-221-20/+10
| | | | | When I added _PyInterpreterState_IsRunningMain() and friends last year, I tried to accommodate applications that embed Python but don't call _PyInterpreterState_SetRunningMain() (not that they're expected to). That mostly worked fine until my recent changes in gh-117049, where the subtleties with the fallback code led to failures; the change ended up breaking test_tools.test_freeze, which exercises a basic embedding situation. The simplest fix is to drop the fallback code I originally added to _PyInterpreterState_IsRunningMain() (and later to _PyThreadState_IsRunningMain()). I've kept the fallback in the _xxsubinterpreters module though. I've also updated Py_FrozenMain() to call _PyInterpreterState_SetRunningMain().
* gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)Sam Gross2024-03-211-15/+24
| | | | | | | | | | | Split `_PyThreadState_DeleteExcept` into two functions: - `_PyThreadState_RemoveExcept` removes all thread states other than one passed as an argument. It returns the removed thread states as a linked list. - `_PyThreadState_DeleteList` deletes those dead thread states. It may call destructors, so we want to "start the world" before calling `_PyThreadState_DeleteList` to avoid potential deadlocks.
* gh-76785: Drop PyInterpreterID_Type (gh-117101)Eric Snow2024-03-211-5/+0
| | | I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation. Relatively recently I refactored that implementation to no longer rely on InterpreterID objects. Thus now I'm removing it.
* gh-105716: Update interp->threads.main After Fork (gh-117049)Eric Snow2024-03-211-0/+35
| | | | | I missed this in gh-109921. We also update Py_Exit() to call _PyInterpreterState_SetNotRunningMain(), if necessary.
* gh-76785: Clean Up Interpreter ID Conversions (gh-117048)Eric Snow2024-03-211-24/+79
| | | Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t. We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
* gh-116522: Stop the world before fork() and during shutdown (#116607)Sam Gross2024-03-211-0/+6
| | | | | | | | | | | This changes the free-threaded build to perform a stop-the-world pause before deleting other thread states when forking and during shutdown. This fixes some crashes when using multiprocessing and during shutdown when running with `PYTHON_GIL=0`. This also changes `PyOS_BeforeFork` to acquire the runtime lock (i.e., `HEAD_LOCK(&_PyRuntime)`) before forking to ensure that data protected by the runtime lock (and not just the GIL or stop-the-world) is in a consistent state before forking.
* gh-116916: Remove separate next_func_version counter (#116918)Guido van Rossum2024-03-181-1/+0
| | | | | Somehow we ended up with two separate counter variables tracking "the next function version". Most likely this was a historical accident where an old branch was updated incorrectly. This PR merges the two counters into a single one: `interp->func_state.next_version`.
* gh-114271: Fix race in `Thread.join()` (#114839)mpage2024-03-161-24/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()` and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution involving threads A, B, and C: 1. A starts. 2. B joins A, blocking on its `_tstate_lock`. 3. C joins A, blocking on its `_tstate_lock`. 4. A finishes and releases its `_tstate_lock`. 5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped out before calling `_stop()`. 6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped out before releasing it. 7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held. However, C holds it, so the assertion fails. The race can be reproduced[^3] by inserting sleeps at the appropriate points in the threading code. To do so, run the `repro_join_race.py` from the linked repo. There are two main parts to this PR: 1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`. The event is set by the runtime prior to the thread being cleared (in the same place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the event to be set. 2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all non-daemon threads to exit. To do so, an `is_daemon` predicate was added to `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()` now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on `_tstate_lock`s. [^1]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1201 [^2]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1115 [^3]: https://github.com/mpage/cpython/commit/81946532792f938cd6f6ab4c4ff92a4edf61314f --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Antoine Pitrou <antoine@python.org>
* gh-116515: Clear thread-local state before tstate_delete_common() (#116517)Sam Gross2024-03-111-1/+2
| | | | | | | This moves `current_fast_clear()` up so that the current thread state is `NULL` while running `tstate_delete_common()`. This doesn't fix any bugs, but it means that we are more consistent that `_PyThreadState_GET() != NULL` means that the thread is "attached".
* gh-116396: Pass "detached_state" argument to tstate_set_detached (#116398)Sam Gross2024-03-071-6/+6
| | | | The stop-the-world code was incorrectly setting suspended threads' states to _Py_THREAD_DETACHED instead of _Py_THREAD_SUSPENDED.
* gh-115103: Delay reuse of mimalloc pages that store PyObjects (#115435)Sam Gross2024-03-061-0/+7
| | | | | | | | | | | | | | | | | | This implements the delayed reuse of mimalloc pages that contain Python objects in the free-threaded build. Allocations of the same size class are grouped in data structures called pages. These are different from operating system pages. For thread-safety, we want to ensure that memory used to store PyObjects remains valid as long as there may be concurrent lock-free readers; we want to delay using it for other size classes, in other heaps, or returning it to the operating system. When a mimalloc page becomes empty, instead of immediately freeing it, we tag it with a QSBR goal and insert it into a per-thread state linked list of pages to be freed. When mimalloc needs a fresh page, we process the queue and free any still empty pages that are now deemed safe to be freed. Pages waiting to be freed are still available for allocations of the same size class and allocating from a page prevent it from being freed. There is additional logic to handle abandoned pages when threads exit.
* gh-115832: Fix instrumentation version mismatch during interpreter shutdown ↵Brett Simmers2024-03-041-0/+3
| | | | | | | | | | | | | (#115856) A previous commit introduced a bug to `interpreter_clear()`: it set `interp->ceval.instrumentation_version` to 0, without making the corresponding change to `tstate->eval_breaker` (which holds a thread-local copy of the version). After this happens, Python code can still run due to object finalizers during a GC, and the version check in bytecodes.c will see a different result than the one in instrumentation.c causing an infinite loop. The fix itself is straightforward: clear `tstate->eval_breaker` when clearing `interp->ceval.instrumentation_version`.
* gh-116012: Preserve GetLastError() across calls to TlsGetValue on Windows ↵Steve Dower2024-02-281-9/+0
| | | | (GH-116014)