summaryrefslogtreecommitdiffstats
path: root/Python/pylifecycle.c
Commit message (Collapse)AuthorAgeFilesLines
* gh-98627: Add an Optional Check for Extension Module Subinterpreter ↵Eric Snow2023-02-161-0/+4
| | | | | | | | | | | | | | | Compatibility (gh-99040) Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL. This change includes the following: * add tests for extension module subinterpreter compatibility * add _PyInterpreterConfig.check_multi_interp_extensions * add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS * add _PyImport_CheckSubinterpIncompatibleExtensionAllowed() * fail iff the module does not implement multi-phase init and the current interpreter is configured to check https://github.com/python/cpython/issues/98627
* gh-101758: Clean Up Uses of Import State (gh-101919)Eric Snow2023-02-151-97/+30
| | | | | | | | | | | | | | | This change is almost entirely moving code around and hiding import state behind internal API. We introduce no changes to behavior, nor to non-internal API. (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.) The motivation is to simplify a number of upcoming changes. Specific changes: * move existing import-related code to import.c, wherever possible * add internal API for interacting with import state (both global and per-interpreter) * use only API outside of import.c (to limit churn there when changing the location, etc.) * consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly) * add macros for import state in import.c (to simplify changing the location) * group code in import.c into sections *remove _PyState_AddModule() https://github.com/python/cpython/issues/101758
* gh-101819: Remove _PyWindowsConsoleIO_Type from the Windows DLL (GH-101904)Erlend E. Aasland2023-02-151-5/+9
| | | Automerge-Triggered-By: GH:erlend-aasland
* gh-59956: Clarify Runtime State Status Expectations (gh-101308)Eric Snow2023-01-301-3/+40
| | | | | | | | | | | | | A PyThreadState can be in one of many states in its lifecycle, represented by some status value. Those statuses haven't been particularly clear, so we're addressing that here. Specifically: * made the distinct lifecycle statuses clear on PyThreadState * identified expectations of how various lifecycle-related functions relate to status * noted the various places where those expectations don't match the actual behavior At some point we'll need to address the mismatches. (This change also includes some cleanup.) https://github.com/python/cpython/issues/59956
* gh-59956: Clarify GILState-related Code (gh-101161)Eric Snow2023-01-191-10/+9
| | | | | | | | | | The objective of this change is to help make the GILState-related code easier to understand. This mostly involves moving code around and some semantically equivalent refactors. However, there are a also a small number of slight changes in structure and behavior: * tstate_current is moved out of _PyRuntimeState.gilstate * autoTSSkey is moved out of _PyRuntimeState.gilstate * autoTSSkey is initialized earlier * autoTSSkey is re-initialized (after fork) earlier https://github.com/python/cpython/issues/59956
* gh-81057: Move Signal-Related Globals to _PyRuntimeState (gh-100085)Eric Snow2022-12-121-1/+0
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move time Globals to _PyRuntimeState (gh-100122)Eric Snow2022-12-081-0/+5
| | | https://github.com/python/cpython/issues/81057
* gh-99337: Fix compile errors with gcc 12 on macOS (#99470)Ronald Oussoren2022-11-211-3/+4
| | | | | | | | Fix a number of compile errors with GCC-12 on macOS: 1. In pylifecycle.c the compile rejects _Pragma within a declaration 2. posixmodule.c was missing a number of ..._RUNTIME macros for non-clang on macOS 3. _ctypes assumed that __builtin_available is always present on macOS
* gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)Eric Snow2022-11-161-2/+0
| | | https://github.com/python/cpython/issues/81057
* gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)Eric Snow2022-11-151-0/+2
| | | | | This is the first of several changes to consolidate non-object globals in core code. https://github.com/python/cpython/issues/81057
* gh-81057: Move PyImport_Inittab to _PyRuntimeState (gh-99402)Eric Snow2022-11-121-0/+5
| | | | | We actually don't move PyImport_Inittab. Instead, we make a copy that we keep on _PyRuntimeState and use only that after Py_Initialize(). We also prevent folks from modifying PyImport_Inittab (the best we can) after that point. https://github.com/python/cpython/issues/81057
* gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)Eric Snow2022-11-111-1/+1
| | | | | The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject. State for the "small block" allocator was stored in another 13. That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057. (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.) https://github.com/python/cpython/issues/81057
* gh-81057: Add PyInterpreterState.static_objects (gh-99397)Eric Snow2022-11-111-1/+1
| | | | | As we consolidate global variables, we find some objects that are almost suitable to add to _PyRuntimeState.global_objects, but have some small/sneaky bit of per-interpreter state (e.g. a weakref list). We're adding PyInterpreterState.static_objects so we can move such objects there. (We'll removed the _not_used field once we've added others.) https://github.com/python/cpython/issues/81057
* GH-96421: Insert shim frame on entry to interpreter (GH-96319)Mark Shannon2022-11-101-1/+20
| | | | | | * Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault() * Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
* gh-99300: Use Py_NewRef() in Python/ directory (#99302)Victor Stinner2022-11-101-6/+3
| | | | Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in C files of the Python/ directory.
* gh-90868: Adjust the Generated Objects (gh-99223)Eric Snow2022-11-081-0/+1
| | | | | | | | | | | We do the following: * move the generated _PyUnicode_InitStaticStrings() to its own file * move the generated _PyStaticObjects_CheckRefcnt() to its own file * include pycore_global_objects.h in extension modules instead of pycore_runtime_init.h These changes help us avoid including things that aren't needed. https://github.com/python/cpython/issues/90868
* gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)Eric Snow2022-10-311-2/+8
| | | | | | | Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads. By default, we were disallowing all three for "isolated" interpreters. We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API. Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads. The default for "isolated" interpreters disables fork, exec, and daemon threads. Regular threads are allowed by default. We continue always allowing everything For the main interpreter and the legacy API. In the code, we add `_PyInterpreterConfig.allow_exec` and `_PyInterpreterConfig.allow_daemon_threads`. We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
* gh-96853: Restore test coverage for Py_Initialize(Ex) (GH-98212)Nick Coghlan2022-10-301-0/+1
| | | | | | | | | | | * As most of `test_embed` now uses `Py_InitializeFromConfig`, add a specific test case to cover `Py_Initialize` (and `Py_InitializeEx`) * Rename `_testembed` init helper to clarify the API used * Add a `PyConfig_Clear` call in `Py_InitializeEx` to make the code more obviously correct (it already didn't leak as none of the dynamically allocated config fields were being populated, but it's clearer if the wrappers follow the documented API usage guidelines)
* gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() ↵Eric Snow2022-10-261-13/+32
| | | | | | | | | | | | | | | | | | | | | | | | | (gh-98609) (see https://github.com/python/cpython/issues/98608) This change does the following: 1. change the argument to a new `_PyInterpreterConfig` struct 2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig` instead of `isolated_subinterpreter`) 3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings * allow_fork * allow_subprocess * allow_threads 4. add `PyInterpreterState.feature_flags` to store those settings 5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *` 6. drop `PyConfig._isolated_interpreter` The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three. None of that changes here; the defaults are preserved. Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved. This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`. I'd rather just avoid that mess from the start for `_PyInterpreterConfig`. We can preserve it later if we find an actual need. This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead). (Note that this PR adds "private" symbols. We'll probably make them public, and add docs, in a separate change.)
* gh-98417: Store int_max_str_digits on the Interpreter State (GH-98418)Eric Snow2022-10-191-0/+2
|
* GH-90699: use statically allocated interned strings in typeobject's slotdefs ↵Kumar Aditya2022-09-071-5/+0
| | | | (GH-94706)
* gh-96143: Allow Linux perf profiler to see Python calls (GH-96123)Pablo Galindo Salgado2022-08-301-0/+11
| | | | | | | :warning: :warning: Note for reviewers, hackers and fellow systems/low-level/compiler engineers :warning: :warning: If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**. If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
* GH-93911: Specialize `LOAD_ATTR` for custom `__getattribute__` (GH-93988)Ken Jin2022-08-171-0/+3
|
* gh-94673: Always Finalize Static Builtin Types (#95153)Eric Snow2022-07-251-1/+2
| | | | | | | | | | | | | Static builtin types are finalized by calling _PyStaticType_Dealloc(). Before this change, we were skipping finalizing such a type if it still had subtypes (i.e. its tp_subclasses hadn't been cleared yet). The problem is that types hold several heap objects, which leak if we skip the type's finalization. This change addresses that. For context, there's an old comment (from e9e3eab0b86) that says the following: // If a type still has subtypes, it cannot be deallocated. // A subtype can inherit attributes and methods of its parent type, // and a type must no longer be used once it's deallocated. However, it isn't clear that is actually still true. Clearing tp_dict should mean it isn't a problem. Furthermore, the only subtypes that might still be around come from extension modules that didn't clean them up when unloaded (i.e. extensions that do not implement multi-phase initialization, AKA PEP 489). Those objects are already leaking, so this change doesn't change anything in that regard. Instead, this change means more objects gets cleaned up that before.
* GH-94851: check refcnt of immortal objects after finalization (GH-95001)Kumar Aditya2022-07-251-0/+3
|
* gh-77560: Report possible errors in restoring builtins at finalization ↵Serhiy Storchaka2022-06-251-1/+1
| | | | | | | (GH-94255) Seems in the past the copy of builtins was not made in some scenarios, and the error was silenced. Write it now to stderr, so we have a chance to see it.
* gh-77782: Py_FdIsInteractive() now uses PyConfig.interactive (#93916)Victor Stinner2022-06-171-10/+12
|
* gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)Serhiy Storchaka2022-06-141-7/+2
| | | | | | It combines PyImport_ImportModule() and PyObject_GetAttrString() and saves 4-6 lines of code on every use. Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
* bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)Eric Snow2022-05-271-2/+0
| | | | | | | This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing: * the configure option * the macro * the code enabled by the macro
* GH-93207: Remove HAVE_STDARG_PROTOTYPES configure check for stdarg.h (#93215)Kumar Aditya2022-05-271-4/+0
|
* gh-88750: Remove the PYTHONTHREADDEBUG env var support. (#92509)Gregory P. Smith2022-05-091-5/+0
| | | | Remove the `PYTHONTHREADDEBUG` env var support. Remove no-op dprintf() macro calls.
* bpo-46864: Suppress even more ob_shash deprecation warnings (GH-32176)Christian Heimes2022-03-301-0/+5
|
* bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() (GH-31874)Victor Stinner2022-03-181-2/+0
| | | | Remove private empty _PyFrame_Fini() and _PyFrame_DebugMallocStats() functions.
* bpo-46841: Use inline caching for calls (GH-31709)Brandt Bucher2022-03-071-0/+10
|
* bpo-46753: Add the empty tuple to the _PyRuntimeState.global_objects. (gh-31345)Eric Snow2022-02-281-5/+0
| | | https://bugs.python.org/issue46753
* Propagate errors (however unlikely) from _Py_Deepfreeze_Init() (GH-31596)Kumar Aditya2022-02-261-1/+3
|
* bpo-45316: Move _PyArg_Fini() to internal C API (GH-31580)Victor Stinner2022-02-251-1/+1
| | | | Move the private unexported _PyArg_Fini() function to the internal C API: to the pycore_pylifecycle.h header file.
* bpo-46430: Fix memory leak in interned strings of deep-frozen modules (GH-31549)Kumar Aditya2022-02-241-2/+4
|
* bpo-46765: Replace Locally Cached Strings with Statically Initialized ↵Eric Snow2022-02-231-4/+0
| | | | | Objects (gh-31366) https://bugs.python.org/issue46765
* bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized ↵Eric Snow2022-02-081-43/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | global objects. (gh-30928) We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules. The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings). https://bugs.python.org/issue46541#msg411799 explains the rationale for this change. The core of the change is in: * (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros * Include/internal/pycore_runtime_init.h - added the static initializers for the global strings * Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState * Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config. The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *. The following are not changed (yet): * stop using _Py_IDENTIFIER() in the stdlib modules * (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API * (maybe) intern the strings during runtime init https://bugs.python.org/issue46541
* bpo-40170: Move _Py_GetAllocatedBlocks() to pycore_pymem.h (GH-30943)Victor Stinner2022-01-271-0/+1
| | | | | Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats() declarations to pycore_pymem.h. These functions are related to memory allocators, not to the PyObject structure.
* bpo-46476: Fix memory leak in code objects generated by deepfreeze (GH-30853)Kumar Aditya2022-01-271-0/+1
| | | Add _Py_Deepfreeze_Fini() and _PyStaticCode_Dealloc() functions.
* bpo-46417: Clear symtable identifiers at exit (GH-30809)Victor Stinner2022-01-221-0/+4
| | | | | | Add _PySymtable_Fini() function, called by finalize_interp_clear(). Update test_cmd_line.test_showrefcount() to tolerate negative reference count.
* bpo-46417: Clear _io module static objects at exit (GH-30807)Victor Stinner2022-01-221-0/+6
| | | | Add _PyIO_Fini() function, called by finalize_interp_clear(). It clears static objects used by the _io extension module.
* bpo-46417: Clear Unicode static types at exit (GH-30806)Victor Stinner2022-01-221-0/+1
| | | | | | | | | | | Add _PyUnicode_FiniTypes() function, called by finalize_interp_types(). It clears these static types: * EncodingMapType * PyFieldNameIter_Type * PyFormatterIter_Type _PyStaticType_Dealloc() now does nothing if tp_subclasses is not NULL.
* bpo-46417: Factorize _PyExc_InitTypes() code (GH-30804)Victor Stinner2022-01-221-6/+4
| | | | | | | | Add 'static_exceptions' list to factorize code between _PyExc_InitTypes() and _PyBuiltins_AddExceptions(). _PyExc_InitTypes() does nothing if it's not the main interpreter. Sort exceptions in Lib/test/exception_hierarchy.txt.
* bpo-46417: Clear more static types (GH-30796)Victor Stinner2022-01-221-1/+1
| | | | | | | * Move PyContext static types into object.c static_types list. * Rename PyContextTokenMissing_Type to _PyContextTokenMissing_Type and declare it in pycore_context.h. * _PyHamtItems types are no long exported: replace PyAPI_DATA() with extern.
* bpo-46417: Py_Finalize() clears static types (GH-30743)Victor Stinner2022-01-211-0/+1
| | | | | | | Add _PyTypes_FiniTypes() best-effort function to clear static types: don't deallocate a type if it still has subclasses. remove_subclass() now sets tp_subclasses to NULL when removing the last subclass.
* bpo-46417: Call _PyDebug_PrintTotalRefs() later (GH-30744)Victor Stinner2022-01-211-6/+6
| | | | | | | | "python -X showrefcount" now shows the total reference count after clearing and destroyed the main Python interpreter. Previously, it was shown before. Py_FinalizeEx() now calls _PyDebug_PrintTotalRefs() after finalize_interp_delete().
* bpo-46417: Finalize structseq types at exit (GH-30645)Victor Stinner2022-01-211-0/+6
| | | | | | | | | | | | | | | | | | Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc() functions to finalize a structseq static type in Py_Finalize(). Currrently, these functions do nothing if Python is built in release mode. Clear static types: * AsyncGenHooksType: sys.set_asyncgen_hooks() * FlagsType: sys.flags * FloatInfoType: sys.float_info * Hash_InfoType: sys.hash_info * Int_InfoType: sys.int_info * ThreadInfoType: sys.thread_info * UnraisableHookArgsType: sys.unraisablehook * VersionInfoType: sys.version * WindowsVersionType: sys.getwindowsversion()
generated by cgit v0.12 at 2025-02-23 08:45:59 (GMT)