summaryrefslogtreecommitdiffstats
path: root/Modules/_threadmodule.c
Commit message (Collapse)AuthorAgeFilesLines
* gh-114570: Add PythonFinalizationError exception (#115352)Victor Stinner2024-02-141-1/+1
| | | | | | | | | | | | | | | | | Add PythonFinalizationError exception. This exception derived from RuntimeError is raised when an operation is blocked during the Python finalization. The following functions now raise PythonFinalizationError, instead of RuntimeError: * _thread.start_new_thread() * subprocess.Popen * os.fork() * os.fork1() * os.forkpty() Morever, _winapi.Overlapped finalizer now logs an unraisable PythonFinalizationError, instead of an unraisable RuntimeError.
* gh-114271: Make `PyInterpreterState.threads.count` thread-safe in ↵mpage2024-02-121-3/+3
| | | | | free-threaded builds (gh-115093) Use atomics to mutate PyInterpreterState.threads.count.
* gh-115035: Mark ThreadHandles as non-joinable earlier after forking (#115042)Sam Gross2024-02-061-17/+36
| | | | | | This marks dead ThreadHandles as non-joinable earlier in `PyOS_AfterFork_Child()` before we execute any Python code. The handles are stored in a global linked list in `_PyRuntimeState` because `fork()` affects the entire process.
* gh-114315: Make `threading.Lock` a real class, not a factory function (#114479)Nikita Sobolev2024-01-251-4/+29
| | | | | | | | | | `threading.Lock` is now the underlying class and is constructable rather than the old factory function. This allows for type annotations to refer to it which had no non-ugly way to be expressed prior to this. --------- Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Gregory P. Smith <greg@krypto.org>
* gh-114414: Assert PyType_GetModuleByDef result in _threadmodule (#114415)Nikita Sobolev2024-01-221-0/+3
|
* gh-111789: Use PyDict_GetItemRef() in Modules/_threadmodule.c (gh-112077)Serhiy Storchaka2023-11-271-6/+4
|
* gh-111262: Add PyDict_Pop() function (#112028)Victor Stinner2023-11-141-5/+2
| | | | | | | _PyDict_Pop_KnownHash(): remove the default value and the return type becomes an int. Co-authored-by: Stefan Behnel <stefan_ml@behnel.de> Co-authored-by: Antoine Pitrou <pitrou@free.fr>
* GH-110829: Ensure Thread.join() joins the OS thread (#110848)Antoine Pitrou2023-11-041-55/+282
| | | | | | | Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes. --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* gh-108082: Remove _PyErr_WriteUnraisableMsg() (GH-111643)Serhiy Storchaka2023-11-031-2/+2
| | | | Replace the remaining calls with PyErr_FormatUnraisable().
* gh-106320: Re-add some PyLong/PyDict C-API functions (GH-#111162)scoder2023-10-251-1/+0
| | | | | | | | * gh-106320: Re-add _PyLong_FromByteArray(), _PyLong_AsByteArray() and _PyLong_GCD() to the public header files since they are used by third-party packages and there is no efficient replacement. See https://github.com/python/cpython/issues/111140 See https://github.com/python/cpython/issues/111139 * gh-111262: Re-add _PyDict_Pop() to have a C-API until a new public one is designed.
* gh-84570: Add Timeouts to SendChannel.send() and RecvChannel.recv() (gh-110567)Eric Snow2023-10-171-5/+6
|
* gh-84570: Send-Wait Fixes for _xxinterpchannels (gh-111006)Eric Snow2023-10-171-50/+2
| | | | | There were a few things I did in gh-110565 that need to be fixed. I also forgot to add tests in that PR. (Note that this PR exposes a refleak introduced by gh-110246. I'll take care of that separately.)
* gh-109860: Use a New Thread State When Switching Interpreters, When ↵Eric Snow2023-10-031-1/+1
| | | | | | | | | Necessary (gh-110245) In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread. For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()). There was a good chance that tstate wasn't actually in use by another thread. However, there are no guarantees of that. Furthermore, re-using an already used tstate is currently fragile. To address this, now we create a new thread state in each of those places and use it. One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
* gh-105716: Support Background Threads in Subinterpreters Consistently ↵Eric Snow2023-10-021-1/+15
| | | | | | | (gh-109921) The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish. We add PyInterpreterState.threads.main, with some internal C-API functions.
* gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (#109629)Antoine Pitrou2023-09-261-0/+14
| | | | | --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw ↵Radislav Chugunov2023-09-251-3/+6
| | | | memory allocator (#109808)
* gh-109611: Add convenient C API function _PyFile_Flush() (GH-109612)Serhiy Storchaka2023-09-231-3/+1
|
* gh-108987: Fix _thread.start_new_thread() race condition (#109135)Victor Stinner2023-09-111-16/+31
| | | | | | | | | | | | | Fix _thread.start_new_thread() race condition. If a thread is created during Python finalization, the newly spawned thread now exits immediately instead of trying to access freed memory and lead to a crash. thread_run() calls PyEval_AcquireThread() which checks if the thread must exit. The problem was that tstate was dereferenced earlier in _PyThreadState_Bind() which leads to a crash most of the time. Move _PyThreadState_CheckConsistency() from thread_run() to _PyThreadState_Bind().
* gh-104690: thread_run() checks for tstate dangling pointer (#109056)Victor Stinner2023-09-081-2/+5
| | | | | | | | thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
* gh-106320: Remove private _PyErr_WriteUnraisableMsg() (#108863)Victor Stinner2023-09-041-0/+1
| | | | | | Move the private _PyErr_WriteUnraisableMsg() functions to the internal C API (pycore_pyerrors.h). Move write_unraisable_exc() from _testcapi to _testinternalcapi.
* gh-106320: Remove private _PySys functions (#108452)Victor Stinner2023-08-241-0/+1
| | | | | | | | | | Move private functions to the internal C API (pycore_sysmodule.h): * _PySys_GetAttr() * _PySys_GetSizeOf() No longer export most of these functions. Fix also a typo in Include/cpython/optimizer.h: add a missing space.
* gh-106320: Remove private _PyDict functions (#108449)Victor Stinner2023-08-241-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Move private functions to the internal C API (pycore_dict.h): * _PyDictView_Intersect() * _PyDictView_New() * _PyDict_ContainsId() * _PyDict_DelItemId() * _PyDict_DelItem_KnownHash() * _PyDict_GetItemIdWithError() * _PyDict_GetItem_KnownHash() * _PyDict_HasSplitTable() * _PyDict_NewPresized() * _PyDict_Next() * _PyDict_Pop() * _PyDict_SetItemId() * _PyDict_SetItem_KnownHash() * _PyDict_SizeOf() No longer export most of these functions. Move also the _PyDictViewObject structure to the internal C API. Move dict_getitem_knownhash() function from _testcapi to the _testinternalcapi extension. Update test_capi.test_dict for this change.
* gh-106320: Remove private _PyEval function (#108433)Victor Stinner2023-08-241-0/+1
| | | | | | | | | | | | | | Move private _PyEval functions to the internal C API (pycore_ceval.h): * _PyEval_GetBuiltin() * _PyEval_GetBuiltinId() * _PyEval_GetSwitchInterval() * _PyEval_MakePendingCalls() * _PyEval_SetProfile() * _PyEval_SetSwitchInterval() * _PyEval_SetTrace() No longer export most of these functions.
* gh-106869: Use new PyMemberDef constant names (#106871)Victor Stinner2023-07-251-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove '#include "structmember.h"'. * If needed, add <stddef.h> to get offsetof() function. * Update Parser/asdl_c.py to regenerate Python/Python-ast.c. * Replace: * T_SHORT => Py_T_SHORT * T_INT => Py_T_INT * T_LONG => Py_T_LONG * T_FLOAT => Py_T_FLOAT * T_DOUBLE => Py_T_DOUBLE * T_STRING => Py_T_STRING * T_OBJECT => _Py_T_OBJECT * T_CHAR => Py_T_CHAR * T_BYTE => Py_T_BYTE * T_UBYTE => Py_T_UBYTE * T_USHORT => Py_T_USHORT * T_UINT => Py_T_UINT * T_ULONG => Py_T_ULONG * T_STRING_INPLACE => Py_T_STRING_INPLACE * T_BOOL => Py_T_BOOL * T_OBJECT_EX => Py_T_OBJECT_EX * T_LONGLONG => Py_T_LONGLONG * T_ULONGLONG => Py_T_ULONGLONG * T_PYSSIZET => Py_T_PYSSIZET * T_NONE => _Py_T_NONE * READONLY => Py_READONLY * PY_AUDIT_READ => Py_AUDIT_READ * READ_RESTRICTED => Py_AUDIT_READ * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
* gh-86493: Fix possible leaks in some modules initialization (GH-106768)Serhiy Storchaka2023-07-181-2/+2
| | | | Fix _ssl, _stat, _testinternalcapi, _threadmodule, cmath, math, posix, time.
* gh-106521: Remove _PyObject_LookupAttr() function (GH-106642)Serhiy Storchaka2023-07-121-1/+1
|
* gh-106320: Remove _PyInterpreterState_Get() alias (#106321)Victor Stinner2023-07-011-1/+1
| | | | Replace calls to the (removed) slow _PyInterpreterState_Get() with fast inlined _PyInterpreterState_GET() function.
* gh-105927: _abc and _thread use PyWeakref_GetRef() (#105961)Victor Stinner2023-06-211-13/+13
| | | | | | | | Hold a strong reference on the object, rather than using a borrowed reference: replace PyWeakref_GET_OBJECT() with PyWeakref_GetRef() and _PyWeakref_GET_REF(). Remove assert(PyWeakref_CheckRef(localweakref)) since it's already tested by _PyWeakref_GET_REF().
* gh-104812: Run Pending Calls in any Thread (gh-104813)Eric Snow2023-06-131-1/+2
| | | For a while now, pending calls only run in the main thread (in the main interpreter). This PR changes things to allow any thread run a pending call, unless the pending call was explicitly added for the main thread to run.
* gh-104690 Disallow thread creation and fork at interpreter finalization ↵chgnrdv2023-06-041-0/+5
| | | | | | | | | | | | | | | | | (#104826) Disallow thread creation and fork at interpreter finalization. in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message: * `_thread.start_new_thread` and thus `threading` * `posix.fork` * `posix.fork1` * `posix.forkpty` * `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied. --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Gregory P. Smith <greg@krypto.org>
* gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)Eric Snow2023-05-051-0/+1
| | | Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules. We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
* gh-103712: Increase the length of the type name in AttributeError messages ↵Alex Gaynor2023-04-241-1/+1
| | | | (#103713)
* gh-59956: Clarify Runtime State Status Expectations (gh-101308)Eric Snow2023-01-301-1/+1
| | | | | | | | | | | | | A PyThreadState can be in one of many states in its lifecycle, represented by some status value. Those statuses haven't been particularly clear, so we're addressing that here. Specifically: * made the distinct lifecycle statuses clear on PyThreadState * identified expectations of how various lifecycle-related functions relate to status * noted the various places where those expectations don't match the actual behavior At some point we'll need to address the mismatches. (This change also includes some cleanup.) https://github.com/python/cpython/issues/59956
* gh-59956: Clarify GILState-related Code (gh-101161)Eric Snow2023-01-191-7/+1
| | | | | | | | | | The objective of this change is to help make the GILState-related code easier to understand. This mostly involves moving code around and some semantically equivalent refactors. However, there are a also a small number of slight changes in structure and behavior: * tstate_current is moved out of _PyRuntimeState.gilstate * autoTSSkey is moved out of _PyRuntimeState.gilstate * autoTSSkey is initialized earlier * autoTSSkey is re-initialized (after fork) earlier https://github.com/python/cpython/issues/59956
* GH-100892: consolidate `HEAD_LOCK/HEAD_UNLOCK` macros (#100953)Kumar Aditya2023-01-151-5/+0
|
* GH-100892: Fix race in clearing `threading.local` (#100922)Kumar Aditya2023-01-111-10/+20
|
* bpo-15999: Accept arbitrary values for boolean parameters. (#15609)Serhiy Storchaka2022-12-031-1/+1
| | | builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
* gh-99377: Add audit events for thread creation and clear (GH-99378)Steve Dower2022-11-161-1/+9
|
* gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)Eric Snow2022-10-311-0/+20
| | | | | | | Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads. By default, we were disallowing all three for "isolated" interpreters. We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API. Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads. The default for "isolated" interpreters disables fork, exec, and daemon threads. Regular threads are allowed by default. We continue always allowing everything For the main interpreter and the legacy API. In the code, we add `_PyInterpreterConfig.allow_exec` and `_PyInterpreterConfig.allow_daemon_threads`. We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
* gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() ↵Eric Snow2022-10-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | (gh-98609) (see https://github.com/python/cpython/issues/98608) This change does the following: 1. change the argument to a new `_PyInterpreterConfig` struct 2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig` instead of `isolated_subinterpreter`) 3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings * allow_fork * allow_subprocess * allow_threads 4. add `PyInterpreterState.feature_flags` to store those settings 5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *` 6. drop `PyConfig._isolated_interpreter` The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three. None of that changes here; the defaults are preserved. Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved. This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`. I'd rather just avoid that mess from the start for `_PyInterpreterConfig`. We can preserve it later if we find an actual need. This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead). (Note that this PR adds "private" symbols. We'll probably make them public, and add docs, in a separate change.)
* gh-95324: Emit a warning if an object doesn't call PyObject_GC_UnTrack ↵Pablo Galindo Salgado2022-07-271-0/+2
| | | | during deallocation in debug mode (#95325)
* gh-91320: Use _PyCFunction_CAST() (#92251)Victor Stinner2022-05-031-5/+5
| | | | | | | | | | Replace "(PyCFunction)(void(*)(void))func" cast with _PyCFunction_CAST(func). Change generated by the command: sed -i -e \ 's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \ $(find -name "*.c")
* bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081)Petr Viktorin2022-02-111-3/+3
| | | | | * Make PyType_GetModuleByDef public (remove underscore) Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized ↵Eric Snow2022-02-081-21/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | global objects. (gh-30928) We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules. The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings). https://bugs.python.org/issue46541#msg411799 explains the rationale for this change. The core of the change is in: * (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros * Include/internal/pycore_runtime_init.h - added the static initializers for the global strings * Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState * Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config. The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *. The following are not changed (yet): * stop using _Py_IDENTIFIER() in the stdlib modules * (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API * (maybe) intern the strings during runtime init https://bugs.python.org/issue46541
* bpo-39277: Fix PY_TIMEOUT_MAX cast in _threadmodule.c (GH-31195)Victor Stinner2022-02-071-1/+1
| | | | | | | | | | | | Cast PY_TIMEOUT_MAX to double, not to _PyTime_t. Fix the clang warning: Modules/_threadmodule.c:1648:26: warning: implicit conversion from '_PyTime_t' (aka 'long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-const-int-float-conversion] double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6; ^~~~~~~~~~~~~~~~~~~~~~~~~ ~
* bpo-46417: _thread uses PyStructSequence_NewType() (GH-30733)Victor Stinner2022-01-211-12/+14
| | | | | The _thread module now creates its _ExceptHookArgs type as a heap type using PyStructSequence_NewType(), rather than using a static type.
* bpo-46008: Stop calling _PyThreadState_Init() in new_threadstate(). (gh-29973)Eric Snow2021-12-081-2/+2
| | | | | This simplifies new_threadstate(). We also rename _PyThreadState_Init() to _PyThreadState_SetCurrent() to reflect what it actually does. https://bugs.python.org/issue46008
* bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)Eric Snow2021-12-071-4/+4
| | | | | This parallels _PyRuntimeState.interpreters. Doing this helps make it more clear what part of PyInterpreterState relates to its threads. https://bugs.python.org/issue46008
* pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)Victor Stinner2021-10-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | Redefining the PyThreadState_GET() macro in pycore_pystate.h is useless since it doesn't affect files not including it. Either use _PyThreadState_GET() directly, or don't use pycore_pystate.h internal C API. For example, the _testcapi extension don't use the internal C API, but use the public PyThreadState_Get() function instead. Replace PyThreadState_Get() with _PyThreadState_GET(). The _PyThreadState_GET() macro is more efficient than PyThreadState_Get() and PyThreadState_GET() function calls which call fail with a fatal Python error. posixmodule.c and _ctypes extension now include <windows.h> before pycore header files (like pycore_call.h). _PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead of PyErr_Fetch()/PyErr_Restore(). The _decimal and _xxsubinterpreters extensions are now built with the Py_BUILD_CORE_MODULE macro defined to get access to the internal C API.
* Fix typos in the Modules directory (GH-28761)Christian Clauss2021-10-071-1/+1
|
generated by cgit v0.12 at 2026-01-11 10:58:45 (GMT)