summaryrefslogtreecommitdiffstats
path: root/Modules/_xxsubinterpretersmodule.c
Commit message (Collapse)AuthorAgeFilesLines
* [3.12] gh-105716: Support Background Threads in Subinterpreters Consistently ↵Eric Snow2023-11-281-43/+47
| | | | | | | | | (gh-109921) (gh-110707) The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish. We add PyInterpreterState.threads.main, with some internal C-API functions. (cherry-picked from commit 1dd9dee45d2591b4e701039d1673282380696849)
* [3.12] gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data ↵Eric Snow2023-11-271-18/+11
| | | | | | (gh-109556) (gh-112288) This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters. (cherry picked from commit fd7e08a6f35581e1189b9bf12feb51f7167a86c5)
* [3.12] gh-101524: Only Use Public C-API in the _xxsubinterpreters Module ↵Miss Islington (bot)2023-07-271-9/+3
| | | | | | | | (gh-105258) (gh-107303) The _xxsubinterpreters module was meant to only use public API. Some internal C-API usage snuck in over the last few years (e.g. gh-28969). This fixes that. (cherry picked from commit e6373c0d8b59512aa7f0dea7f3fb162b6ed10fa4) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)Eric Snow2023-05-051-0/+1
| | | Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules. We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
* gh-99113: Share the GIL via PyInterpreterState.ceval.gil (gh-104203)Eric Snow2023-05-051-0/+1
| | | In preparation for a per-interpreter GIL, we add PyInterpreterState.ceval.gil, set it to the shared GIL for each interpreter, and use that rather than using _PyRuntime.ceval.gil directly. Note that _PyRuntime.ceval.gil is still the actual GIL.
* gh-104109: Expose Py_NewInterpreterFromConfig() in the Public C-API (gh-104110)Eric Snow2023-05-031-4/+4
| | | | | We also expose PyInterpreterConfig. This is part of the PEP 684 (per-interpreter GIL) implementation. We will add docs as soon as we can. FYI, I'm adding the new config field for per-interpreter GIL in gh-99114.
* gh-101659: Add _Py_AtExit() (gh-103298)Eric Snow2023-04-061-10/+1
| | | | | The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on gh-101660.)
* Fix a compiler warning in _xxsubinterpretermodule.c (#103245)T. Wouters2023-04-041-1/+1
| | | | | | | Fix a (correct) warning about potential uses of uninitialized memory in _xxsubinterpreter. Unlike newly allocated PyObject structs or global structs, stack-allocated structs are not initialised, and a few places in the code expect the _sharedexception struct data to be either NULL or initialised.
* gh-98608: Stop Treating All Errors from _Py_NewInterpreterFromConfig() as ↵Eric Snow2023-03-211-2/+7
| | | | | | | Fatal (gh-102657) Prior to this change, errors in _Py_NewInterpreterFromConfig() were always fatal. Instead, callers should be able to handle such errors and keep going. That's what this change supports. (This was an oversight in the original implementation of _Py_NewInterpreterFromConfig().) Note that the existing [fatal] behavior of the public Py_NewInterpreter() is preserved. https://github.com/python/cpython/issues/98608
* gh-101659: Avoid Allocation for Shared Exceptions in the _xxsubinterpreters ↵Eric Snow2023-03-131-74/+49
| | | | | Module (gh-102659) https://github.com/python/cpython/issues/101659
* gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives in ↵Irit Katriel2023-03-061-14/+8
| | | | sub interpreters module (#102472)
* gh-101524: Split Up the _xxsubinterpreters Module (gh-101526)Eric Snow2023-02-041-2178/+11
| | | | | This is step 1 in potentially dropping all the "channel"-related code. Channels have already been removed from PEP 554. https://github.com/python/cpython/issues/101524
* gh-99984: Fix Compiler Warnings (#100036)Eric Snow2022-12-051-2/+6
| | | https://github.com/python/cpython/issues/99984
* gh-99741: Implement Multi-Phase Init for the _xxsubinterpreters Module ↵Eric Snow2022-12-051-123/+226
| | | | | | | (gh-99742) _xxsubinterpreters is an internal module used for testing. https://github.com/python/cpython/issues/99741
* gh-99741: Clean Up the _xxsubinterpreters Module (gh-99940)Eric Snow2022-12-021-299/+615
| | | | | This cleanup up resolves a few subtle bugs and makes the implementation for multi-phase init much cleaner. https://github.com/python/cpython/issues/99741
* gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)Victor Stinner2022-11-231-2/+1
| | | Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
* gh-99537: Use Py_CLEAR() function in C code (#99686)Victor Stinner2022-11-221-2/+1
| | | | | | | Replace "Py_XDECREF(var); var = NULL;" with "Py_CLEAR(var);". Don't replace "Py_DECREF(var); var = NULL;" with "Py_CLEAR(var);". It would add an useless "if (var)" test in code path where var cannot be NULL.
* gh-99300: Use Py_NewRef() in Modules/ directory (#99468)Victor Stinner2022-11-141-6/+5
| | | | Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in test C files of the Modules/ directory.
* gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)Eric Snow2022-10-311-5/+3
| | | | | | | Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads. By default, we were disallowing all three for "isolated" interpreters. We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API. Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads. The default for "isolated" interpreters disables fork, exec, and daemon threads. Regular threads are allowed by default. We continue always allowing everything For the main interpreter and the legacy API. In the code, we add `_PyInterpreterConfig.allow_exec` and `_PyInterpreterConfig.allow_daemon_threads`. We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
* gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() ↵Eric Snow2022-10-261-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | (gh-98609) (see https://github.com/python/cpython/issues/98608) This change does the following: 1. change the argument to a new `_PyInterpreterConfig` struct 2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig` instead of `isolated_subinterpreter`) 3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings * allow_fork * allow_subprocess * allow_threads 4. add `PyInterpreterState.feature_flags` to store those settings 5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *` 6. drop `PyConfig._isolated_interpreter` The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three. None of that changes here; the defaults are preserved. Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved. This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`. I'd rather just avoid that mess from the start for `_PyInterpreterConfig`. We can preserve it later if we find an actual need. This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead). (Note that this PR adds "private" symbols. We'll probably make them public, and add docs, in a separate change.)
* gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)Victor Stinner2022-06-191-1/+0
| | | | | | | | | | | | | | | | Move the follow functions and type from frameobject.h to pyframe.h, so the standard <Python.h> provide frame getter functions: * PyFrame_Check() * PyFrame_GetBack() * PyFrame_GetBuiltins() * PyFrame_GetGenerator() * PyFrame_GetGlobals() * PyFrame_GetLasti() * PyFrame_GetLocals() * PyFrame_Type Remove #include "frameobject.h" from many C files. It's no longer needed.
* bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)Eric Snow2022-05-271-15/+0
| | | | | | | This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing: * the configure option * the macro * the code enabled by the macro
* gh-92434: Silence a compiler warning in _xxsubinterpretersmodule.c for 32bit ↵neonene2022-05-251-1/+1
| | | | version (gh-93091)
* gh-91320: Use _PyCFunction_CAST() (#92251)Victor Stinner2022-05-031-12/+12
| | | | | | | | | | Replace "(PyCFunction)(void(*)(void))func" cast with _PyCFunction_CAST(func). Change generated by the command: sed -i -e \ 's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \ $(find -name "*.c")
* bpo-47045: Remove `f_state` field (GH-31963)Mark Shannon2022-03-221-4/+1
| | | | | * Remove the f_state field from _PyInterpreterFrame * Make ownership of the frame explicit, replacing the is_generator field with an owner field.
* bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583)Victor Stinner2022-02-251-1/+1
| | | | | Rename also struct _interpreter_frame to struct _PyInterpreterFrame. Reduce risk of name conflicts if a project includes pycore_frame.h.
* bpo-45637: Store the frame pointer in the cframe (GH-29267)Mark Shannon2021-10-281-1/+1
| | | * Rename 'frame' to 'current_frame'
* bpo-43974: Move Py_BUILD_CORE_MODULE into module code (GH-29157)Christian Heimes2021-10-221-0/+3
| | | | | | | | | | | | | | setup.py no longer defines Py_BUILD_CORE_MODULE. Instead every module defines the macro before #include "Python.h" unless Py_BUILD_CORE_BUILTIN is already defined. Py_BUILD_CORE_BUILTIN is defined for every module that is built by Modules/Setup. The PR also simplifies Modules/Setup. Makefile and makesetup already define Py_BUILD_CORE_BUILTIN and include Modules/internal for us. Signed-off-by: Christian Heimes <christian@python.org>
* bpo-35081: Move interpreteridobject.h to Include/internal/ (GH-28969)Victor Stinner2021-10-151-1/+1
| | | | Move the interpreteridobject.h header file from Include/ to Include/internal/. It only provides private functions.
* pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)Victor Stinner2021-10-131-1/+2
| | | | | | | | | | | | | | | | | | | | | | Redefining the PyThreadState_GET() macro in pycore_pystate.h is useless since it doesn't affect files not including it. Either use _PyThreadState_GET() directly, or don't use pycore_pystate.h internal C API. For example, the _testcapi extension don't use the internal C API, but use the public PyThreadState_Get() function instead. Replace PyThreadState_Get() with _PyThreadState_GET(). The _PyThreadState_GET() macro is more efficient than PyThreadState_Get() and PyThreadState_GET() function calls which call fail with a fatal Python error. posixmodule.c and _ctypes extension now include <windows.h> before pycore header files (like pycore_call.h). _PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead of PyErr_Fetch()/PyErr_Restore(). The _decimal and _xxsubinterpreters extensions are now built with the Py_BUILD_CORE_MODULE macro defined to get access to the internal C API.
* bpo-44590: Lazily allocate frame objects (GH-27077)Mark Shannon2021-07-261-2/+2
| | | | | | | | | | | | | | * Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it. * Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector. * Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation. * Move ownership of InterpreterFrame in generator from frame object to generator object. * Do not create frame objects for Python calls. * Do not create frame objects for generators.
* bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)Victor Stinner2021-04-301-14/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow creating type instances: set tp_new to NULL and don't create the "__new__" key in the type dictionary. The flag is set automatically on static types if tp_base is NULL or &PyBaseObject_Type and tp_new is NULL. Use the flag on the following types: * _curses.ncurses_version type * _curses_panel.panel * _tkinter.Tcl_Obj * _tkinter.tkapp * _tkinter.tktimertoken * _xxsubinterpretersmodule.ChannelID * sys.flags type * sys.getwindowsversion() type * sys.version_info type Update MyStr example in the C API documentation to use Py_TPFLAGS_DISALLOW_INSTANTIATION. Add _PyStructSequence_InitType() function to create a structseq type with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set. type_new() calls _PyType_CheckConsistency() at exit.
* bpo-43472: Ensure PyInterpreterState_New audit events are raised when called ↵Steve Dower2021-04-211-1/+1
| | | | through _xxsubinterpreters module (GH-25506)
* bpo-40941: Unify implicit and explicit state in the frame and generator ↵Mark Shannon2020-07-171-1/+1
| | | | | | | objects into a single value. (GH-20803) * Merge gen and frame state variables into one. * Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
* Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" ↵Victor Stinner2020-05-141-1026/+121
| | | | | | | | | | | (GH-20089) * Revert "bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)" This reverts commit fa0a66e62d087765dbc5c1b89d6149a23ecfb0a6. * Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" This reverts commit a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57.
* bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)Dong-hee Na2020-05-131-0/+8
|
* bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)Eric Snow2020-05-071-121/+1018
| | | | | | | | | (Note: PEP 554 is not accepted and the implementation in the code base is a private one for use in the test suite.) If code running in a subinterpreter raises an uncaught exception then the "run" call in the calling interpreter fails. A RunFailedError is raised there that summarizes the original exception as a string. The actual exception type, __cause__, __context__, state, etc. are all discarded. This turned out to be functionally insufficient in practice. There is a more helpful solution (and PEP 554 has been updated appropriately). This change adds the exception propagation behavior described in PEP 554 to the _xxsubinterpreters module. With this change a copy of the original exception is set to __cause__ on the RunFailedError. For now we are using "pickle", which preserves the exception's state. We also preserve the original __cause__, __context__, and __traceback__ (since "pickle" does not preserve those). https://bugs.python.org/issue32604
* bpo-40513: _xxsubinterpreters.run_string() releases the GIL (GH-19944)Victor Stinner2020-05-051-0/+15
| | | | In the experimental isolated subinterpreters build mode, _xxsubinterpreters.run_string() now releases the GIL.
* bpo-40453: Add PyConfig._isolated_subinterpreter (GH-19820)Victor Stinner2020-05-011-5/+9
| | | | | | | | | An isolated subinterpreter cannot spawn threads, spawn a child process or call os.fork(). * Add private _Py_NewInterpreter(isolated_subinterpreter) function. * Add isolated=True keyword-only parameter to _xxsubinterpreters.create(). * Allow again os.fork() in "non-isolated" subinterpreters.
* bpo-40429: PyThreadState_GetFrame() returns a strong ref (GH-19781)Victor Stinner2020-04-291-4/+7
| | | | The PyThreadState_GetFrame() function now returns a strong reference to the frame.
* bpo-38880: List interpreters associated with a channel end (GH-17323)Lewis Gaul2020-04-291-6/+83
| | | | | This PR adds the functionality requested by https://github.com/ericsnowcurrently/multi-core-python/issues/52. Automerge-Triggered-By: @ericsnowcurrently
* bpo-32604: Add support for a "default" arg in channel_recv(). (GH-19770)Eric Snow2020-04-281-11/+25
| | | | | This allows the caller to avoid creation of an exception when the channel is empty (just like `dict.get()` works). `ChannelEmptyError` is still raised if no default is provided. Automerge-Triggered-By: @ericsnowcurrently
* bpo-40302: Replace PY_INT64_T with int64_t (GH-19573)Victor Stinner2020-04-171-1/+1
| | | | | | | | | * Replace PY_INT64_T with int64_t * Replace PY_UINT32_T with uint32_t * Replace PY_UINT64_T with uint64_t sha3module.c no longer checks if PY_UINT64_T is defined since it's always defined and uint64_t is always available on platforms supported by Python.
* bpo-39947: Use PyThreadState_GetFrame() (GH-19159)Victor Stinner2020-03-251-3/+4
| | | | | _tracemalloc.c and _xxsubinterpretersmodule.c use PyThreadState_GetFrame() and PyThreadState_GetInterpreter() to no longer depend on the PyThreadState structure.
* bpo-39947: Add PyInterpreterState_Get() function (GH-18979)Victor Stinner2020-03-131-3/+3
| | | | | | * Rename _PyInterpreterState_Get() to PyInterpreterState_Get() and move it the limited C API. * Add _PyInterpreterState_Get() alias to PyInterpreterState_Get() for backward compatibility with Python 3.8.
* bpo-39573: Use Py_TYPE() macro in Modules directory (GH-18393)Victor Stinner2020-02-071-1/+1
| | | Replace direct access to PyObject.ob_type with Py_TYPE().
* closes bpo-39262: Use specific out-of-memory message in ↵Alex Henrie2020-01-091-2/+4
| | | | _sharedexception_bind. (GH-17908)
* bpo-36854: Clear the current thread later (GH-17279)Victor Stinner2019-11-201-1/+0
| | | | | | | | | | Clear the current thread later in the Python finalization. * The PyInterpreterState_Delete() function is now responsible to call PyThreadState_Swap(NULL). * The tstate_delete_common() function is now responsible to clear the "autoTSSKey" thread local storage and it only clears it once the thread state is fully cleared. It allows to still get the current thread from TSS in tstate_delete_common().
* bpo-38005: Fixed comparing and creating of InterpreterID and ChannelID. ↵Serhiy Storchaka2019-09-131-85/+65
| | | | | | | | | | | | | (GH-15652) * Fix a crash in comparing with float (and maybe other crashes). * They are now never equal to strings and non-integer numbers. * Comparison with a large number no longer raises OverflowError. * Arbitrary exceptions no longer silenced in constructors and comparisons. * TypeError raised in the constructor contains now the name of the type. * Accept only ChannelID and int-like objects in channel functions. * Accept only InterpreterId, int-like objects and str in the InterpreterId constructor. * Accept int-like objects, not just int in interpreter related functions.
* bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async ↵Jeroen Demeyer2019-05-311-1/+1
| | | | | | | | | (GH-13464) Automatically replace tp_print -> tp_vectorcall_offset tp_compare -> tp_as_async tp_reserved -> tp_as_async
generated by cgit v0.12 at 2026-01-03 17:57:29 (GMT)