summaryrefslogtreecommitdiffstats
path: root/Python/pylifecycle.c
Commit message (Collapse)AuthorAgeFilesLines
* GH-113464: Add a JIT backend for tier 2 (GH-113465)Brandt Bucher2024-01-291-0/+7
| | | | | | | Add an option (--enable-experimental-jit for configure-based builds or --experimental-jit for PCbuild-based ones) to build an *experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf). See Tools/jit/README.md for more information on how to install the required build-time tooling.
* gh-113055: Use pointer for interp->obmalloc state (gh-113412)Neil Schemenauer2024-01-271-0/+16
| | | | | | | | | For interpreters that share state with the main interpreter, this points to the same static memory structure. For interpreters with their own obmalloc state, it is heap allocated. Add free_obmalloc_arenas() which will free the obmalloc arenas and radix tree structures for interpreters with their own obmalloc state. Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* gh-114312: Collect stats for unlikely events (GH-114493)Michael Droettboom2024-01-251-0/+18
|
* gh-111968: Use per-thread freelists for generator in free-threading (gh-114189)Donghee Na2024-01-181-1/+1
|
* gh-111968: Use per-thread freelists for PyContext in free-threading (gh-114122)Donghee Na2024-01-161-1/+1
|
* gh-111968: Use per-thread slice_cache in free-threading (gh-113972)Donghee Na2024-01-151-3/+1
|
* gh-111968: Use per-thread freelists for tuple in free-threading (gh-113921)Donghee Na2024-01-111-1/+1
|
* gh-111968: Use per-thread freelists for float in free-threading (gh-113886)Donghee Na2024-01-101-1/+1
|
* gh-111968: Introduce _PyFreeListState and _PyFreeListState_GET API (gh-113584)Donghee Na2024-01-091-1/+4
|
* gh-112532: Use separate mimalloc heaps for GC objects (gh-113263)Sam Gross2023-12-261-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | * gh-112532: Use separate mimalloc heaps for GC objects In `--disable-gil` builds, we now use four separate heaps in anticipation of using mimalloc to find GC objects when the GIL is disabled. To support this, we also make a few changes to mimalloc: * `mi_heap_t` and `mi_tld_t` initialization is split from allocation. This allows us to have a `mi_tld_t` per-`PyThreadState`, which is important to keep interpreter isolation, since the same OS thread may run in multiple interpreters (using different PyThreadStates.) * Heap abandoning (mi_heap_collect_ex) can now be called from a different thread than the one that created the heap. This is necessary because we may clear and delete the containing PyThreadStates from a different thread during finalization and after fork(). * Use enum instead of defines and guard mimalloc includes. * The enum typedef will be convenient for future PRs that use the type. * Guarding the mimalloc includes allows us to unconditionally include pycore_mimalloc.h from other header files that rely on things like `struct _mimalloc_thread_state`. * Only define _mimalloc_thread_state in Py_GIL_DISABLED builds
* gh-112723: Call `PyThreadState_Clear()` from the correct interpreter (#112776)Sam Gross2023-12-131-46/+26
| | | | | | | | | | | | | The `PyThreadState_Clear()` function must only be called with the GIL held and must be called from the same interpreter as the passed in thread state. Otherwise, any Python objects on the thread state may be destroyed using the wrong interpreter, leading to memory corruption. This is also important for `Py_GIL_DISABLED` builds because free lists will be associated with PyThreadStates and cleared in `PyThreadState_Clear()`. This fixes two places that called `PyThreadState_Clear()` from the wrong interpreter and adds an assertion to `PyThreadState_Clear()`.
* gh-76785: Fixes for test.support.interpreters (gh-112982)Eric Snow2023-12-121-0/+6
| | | This involves a number of changes for PEP 734.
* gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)Sam Gross2023-12-071-7/+7
| | | | | This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize. This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
* gh-111962: Make dtoa thread-safe in `--disable-gil` builds. (#112049)Sam Gross2023-12-071-0/+6
| | | | | | | | | | | | | | | | | | | | This updates `dtoa.c` to avoid using the Bigint free-list in --disable-gil builds and to pre-computes the needed powers of 5 during interpreter initialization. * gh-111962: Make dtoa thread-safe in `--disable-gil` builds. This avoids using the Bigint free-list in `--disable-gil` builds and pre-computes the needed powers of 5 during interpreter initialization. * Fix size of cached powers of 5 array. We need the powers of 5 up to 5**512 because we only jump straight to underflow when the exponent is less than -512 (or larger than 308). * Rename Py_NOGIL to Py_GIL_DISABLED * Changes from review * Fix assertion placement
* gh-112567: Add _Py_GetTicksPerSecond() function (#112587)Victor Stinner2023-12-011-5/+0
| | | | | | | | | | * Move _PyRuntimeState.time to _posixstate.ticks_per_second and time_module_state.ticks_per_second. * Add time_module_state.clocks_per_second. * Rename _PyTime_GetClockWithInfo() to py_clock(). * Rename _PyTime_GetProcessTimeWithInfo() to py_process_time(). * Add process_time_times() helper function, called by py_process_time(). * os.times() is now always built: no longer rely on HAVE_TIMES.
* gh-112367: Only free perf trampoline arenas at shutdown (#112368)Pablo Galindo Salgado2023-12-011-1/+1
| | | Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* gh-108082: Use PyErr_FormatUnraisable() (GH-111580)Serhiy Storchaka2023-11-021-17/+17
| | | | | | Replace most of calls of _PyErr_WriteUnraisableMsg() and some calls of PyErr_WriteUnraisable(NULL) with PyErr_FormatUnraisable(). Co-authored-by: Victor Stinner <vstinner@python.org>
* gh-76785: Crossinterp utils additions (gh-111530)Eric Snow2023-11-011-0/+7
| | | | | | | | | | | | This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers). Specifically: * _Py_excinfo, etc.: the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c) * _PyXI_exception_info, etc.: helpers for passing an exception beween interpreters (wraps _Py_excinfo) * _PyXI_namespace, etc.: helpers for copying a dict of attrs between interpreters * _PyXI_Enter(), _PyXI_Exit(): functions that abstract out the transitions between one interpreter and a second that will do some work temporarily Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations. I plan on proposing these as public API at some point.
* gh-111520: Integrate the Tier 2 interpreter in the Tier 1 interpreter (#111428)Guido van Rossum2023-11-011-1/+1
| | | | | | | | | | | - There is no longer a separate Python/executor.c file. - Conventions in Python/bytecodes.c are slightly different -- don't use `goto error`, you must use `GOTO_ERROR(error)` (same for others like `unused_local_error`). - The `TIER_ONE` and `TIER_TWO` symbols are only valid in the generated (.c.h) files. - In Lib/test/support/__init__.py, `Py_C_RECURSION_LIMIT` is imported from `_testcapi`. - On Windows, in debug mode, stack allocation grows from 8MiB to 12MiB. - **Beware!** This changes the env vars to enable uops and their debugging to `PYTHON_UOPS` and `PYTHON_LLTRACE`.
* gh-110722: Add PYTHON_PRESITE to import a module before site.py is run (#110769)Łukasz Langa2023-10-141-0/+36
|
* gh-109549: Add new states to PyThreadState to support PEP 703 (gh-109915)Sam Gross2023-10-051-9/+5
| | | This adds a new field 'state' to PyThreadState that can take on one of three values: _Py_THREAD_ATTACHED, _Py_THREAD_DETACHED, or _Py_THREAD_GC. The "attached" and "detached" states correspond closely to acquiring and releasing the GIL. The "gc" state is current unused, but will be used to implement stop-the-world GC for --disable-gil builds in the near future.
* gh-109860: Use a New Thread State When Switching Interpreters, When ↵Eric Snow2023-10-031-2/+4
| | | | | | | | | Necessary (gh-110245) In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread. For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()). There was a good chance that tstate wasn't actually in use by another thread. However, there are no guarantees of that. Furthermore, re-using an already used tstate is currently fragile. To address this, now we create a new thread state in each of those places and use it. One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
* gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994)Eric Snow2023-10-021-0/+25
| | | This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This PR does not address the broader concerns from gh-109853.
* gh-110014: Include explicitly <unistd.h> header (#110155)Victor Stinner2023-09-301-0/+3
| | | | | | | | | | | * Remove unused <locale.h> includes. * Remove unused <fcntl.h> include in traceback.h. * Remove redundant <assert.h> and <stddef.h> includes. They are already included by "Python.h". * Remove <object.h> include in faulthandler.c. Python.h already includes it. * Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS is defined. * Fix also warnings in pthread_stubs.h: don't redefine macros if they are already defined, like the __NEED_pthread_t macro.
* gh-110079: Remove extern "C" { ...} in C code (#110080)Victor Stinner2023-09-291-9/+0
|
* gh-109611: Add convenient C API function _PyFile_Flush() (GH-109612)Serhiy Storchaka2023-09-231-14/+3
|
* GH-108716: Turn off deep-freezing of code objects. (GH-108722)Mark Shannon2023-09-081-6/+0
|
* gh-108765: Cleanup #include in Python/*.c files (#108977)Victor Stinner2023-09-061-2/+3
| | | Mention one symbol imported by each #include.
* gh-108634: PyInterpreterState_New() no longer calls Py_FatalError() (#108748)Victor Stinner2023-09-011-3/+5
| | | | | | | | | | | | pycore_create_interpreter() now returns a status, rather than calling Py_FatalError(). * PyInterpreterState_New() now calls Py_ExitStatusException() instead of calling Py_FatalError() directly. * Replace Py_FatalError() with PyStatus in init_interpreter() and _PyObject_InitState(). * _PyErr_SetFromPyStatus() now raises RuntimeError, instead of ValueError. It can now call PyErr_NoMemory(), raise MemoryError, if it detects _PyStatus_NO_MEMORY() error message.
* gh-108634: Py_TRACE_REFS uses a hash table (#108663)Victor Stinner2023-08-311-13/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Python built with "configure --with-trace-refs" (tracing references) is now ABI compatible with Python release build and debug build. Moreover, it now also supports the Limited API. Change Py_TRACE_REFS build: * Remove _PyObject_EXTRA_INIT macro. * The PyObject structure no longer has two extra members (_ob_prev and _ob_next). * Use a hash table (_Py_hashtable_t) to trace references (all objects): PyInterpreterState.object_state.refchain. * Py_TRACE_REFS build is now ABI compatible with release build and debug build. * Limited C API extensions can now be built with Py_TRACE_REFS: xxlimited, xxlimited_35, _testclinic_limited. * No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2() functions to PyModule_Create2TraceRefs() and PyModule_FromDefAndSpec2TraceRefs(). * _Py_PrintReferenceAddresses() is now called before finalize_interp_delete() which deletes the refchain hash table. * test_tracemalloc find_trace() now also filters by size to ignore the memory allocated by _PyRefchain_Trace(). Test changes for Py_TRACE_REFS: * Add test.support.Py_TRACE_REFS constant. * Add test_sys.test_getobjects() to test sys.getobjects() function. * test_exceptions skips test_recursion_normalizing_with_no_memory() and test_memory_error_in_PyErr_PrintEx() if Python is built with Py_TRACE_REFS. * test_repl skips test_no_memory(). * test_capi skisp test_set_nomemory().
* gh-108314: Add PyDict_ContainsString() function (#108323)Victor Stinner2023-08-241-4/+5
|
* gh-108308: Replace _PyDict_GetItemStringWithError() (#108372)Victor Stinner2023-08-231-12/+14
| | | | | | | Replace _PyDict_GetItemStringWithError() calls with PyDict_GetItemStringRef() which returns a strong reference to the item. Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-108308: Replace PyDict_GetItem() with PyDict_GetItemRef() (#108309)Victor Stinner2023-08-231-6/+18
| | | | | | | | | Replace PyDict_GetItem() calls with PyDict_GetItemRef() or PyDict_GetItemWithError() to handle errors. * Replace PyLong_AS_LONG() with _PyLong_AsInt() and check for errors. * Check for PyDict_Contains() error. * pycore_init_builtins() checks for _PyType_Lookup() failure.
* Ignore _Py_write_noraise() result: cast to (void) (#108291)Victor Stinner2023-08-221-1/+1
| | | | Code using _Py_write_noraise() usually cannot report. Ignore errors is the least surprising behavior for users.
* gh-108014: Add Py_IsFinalizing() function (#108032)Victor Stinner2023-08-181-1/+1
| | | Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for ↵Mark Shannon2023-08-171-1/+1
| | | | performance. (GH-108036)
* gh-107630: Initialize Each Interpreter's refchain Properly (gh-107733)Eric Snow2023-08-071-0/+2
| | | This finishes fixing the crashes in Py_TRACE_REFS builds. We missed this part in gh-107567.
* gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)Eric Snow2023-08-031-4/+4
| | | The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
* gh-106320: Move private _PySet API to the internal API (#107041)Victor Stinner2023-07-221-0/+1
| | | | | | | | * Add pycore_setobject.h header file. * Move the following API to the internal C API: * _PySet_Dummy * _PySet_NextEntry() * _PySet_Update()
* GH-104584: Miscellaneous fixes for -Xuops (GH-106908)Brandt Bucher2023-07-201-0/+4
|
* Remove private _PyCodec_Lookup() function (#106269)Victor Stinner2023-06-301-0/+1
| | | | | | | | | | | | | | | Remove the following private functions of the C API: * _PyCodecInfo_GetIncrementalDecoder() * _PyCodecInfo_GetIncrementalEncoder() * _PyCodec_DecodeText() * _PyCodec_EncodeText() * _PyCodec_Forget() * _PyCodec_Lookup() * _PyCodec_LookupTextEncoding() Move these functions to a new pycore_codecs.h internal header file. These functions are no longer exported.
* gh-106084: Remove _PyObject_CallMethod() function (#106159)Victor Stinner2023-06-271-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the following private functions from the public C API: * _Py_CheckFunctionResult() * _PyObject_CallMethod() * _PyObject_CallMethodId() * _PyObject_CallMethodIdNoArgs() * _PyObject_CallMethodIdObjArgs() * _PyObject_CallMethodIdOneArg() * _PyObject_MakeTpCall() * _PyObject_VectorcallMethodId() * _PyStack_AsDict() Move these functions to the internal C API (pycore_call.h). No longer export the following functions: * _PyObject_Call() * _PyObject_CallMethod() * _PyObject_CallMethodId() * _PyObject_CallMethodIdObjArgs() * _PyObject_Call_Prepend() * _PyObject_FastCallDictTstate() * _PyStack_AsDict() The following functions are still exported for stdlib shared extensions: * _Py_CheckFunctionResult() * _PyObject_MakeTpCall() Mark the following internal functions as extern: * _PyStack_UnpackDict() * _PyStack_UnpackDict_Free() * _PyStack_UnpackDict_FreeNoDecRef()
* gh-104584: Baby steps towards generating and executing traces (#105924)Guido van Rossum2023-06-271-0/+13
| | | | | Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose). All of this code is likely to change dramatically before the 3.13 feature freeze. But this is a first step.
* gh-105927: finalize_modules_clear_weaklist() uses _PyWeakref_GET_REF() (#105971)Victor Stinner2023-06-211-4/+5
| | | | | finalize_modules_clear_weaklist() now holds a strong reference to the module longer than before: replace PyWeakref_GET_OBJECT() with _PyWeakref_GET_REF().
* GH-100987: Allow objects other than code objects as the "executable" of an ↵Mark Shannon2023-06-141-20/+0
| | | | | | | | | | internal frame. (GH-105727) * Add table describing possible executable classes for out-of-process debuggers. * Remove shim code object creation code as it is no longer needed. * Make lltrace a bit more robust w.r.t. non-standard frames.
* gh-104812: Run Pending Calls in any Thread (gh-104813)Eric Snow2023-06-131-0/+3
| | | For a while now, pending calls only run in the main thread (in the main interpreter). This PR changes things to allow any thread run a pending call, unless the pending call was explicitly added for the main thread to run.
* gh-105603: Change the PyInterpreterConfig.own gil Field (gh-105620)Eric Snow2023-06-131-4/+15
| | | We are changing it to be more flexible that a strict bool can be for possible future expanded used cases.
* gh-100227: Lock Around Use of the Global "atexit" State (gh-105514)Eric Snow2023-06-081-3/+14
| | | The risk of a race with this state is relatively low, but we play it safe anyway.
* gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current ↵Eric Snow2023-06-011-1/+1
| | | | | | | Thread (gh-105109) This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads. (The idea for this approach came out of discussions with @markshannon.)
* gh-105145: Remove old functions to config Python init (#105154)Victor Stinner2023-06-011-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the following old functions to configure the Python initialization, deprecated in Python 3.11: * PySys_AddWarnOptionUnicode() * PySys_AddWarnOption() * PySys_AddXOption() * PySys_HasWarnOptions() * PySys_SetArgvEx() * PySys_SetArgv() * PySys_SetPath() * Py_SetPath() * Py_SetProgramName() * Py_SetPythonHome() * Py_SetStandardStreamEncoding() * _Py_SetProgramFullPath() Most of these functions are kept in the stable ABI, except: * Py_SetStandardStreamEncoding() * _Py_SetProgramFullPath() Update Doc/extending/embedding.rst and Doc/extending/extending.rst to use the new PyConfig API. _testembed.c: * check_stdio_details() now sets stdio_encoding and stdio_errors of PyConfig. * Add definitions of functions removed from the API but kept in the stable ABI. * test_init_from_config() and test_init_read_set() now use PyConfig_SetString() instead of PyConfig_SetBytesString(). Remove _Py_ClearStandardStreamEncoding() internal function.
generated by cgit v0.12 at 2025-07-25 22:44:57 (GMT)