summaryrefslogtreecommitdiffstats
path: root/Include/internal/pycore_interp.h
Commit message (Collapse)AuthorAgeFilesLines
* [3.11] gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96500)Gregory P. Smith2022-09-021-0/+2
| | | | | | | | | | | | | | | | | | Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds. This PR comes fresh from a pile of work done in our private PSRT security response team repo. This backports https://github.com/python/cpython/pull/96499 aka 511ca9452033ef95bc7d7fc404b8161068226002 Signed-off-by: Christian Heimes [Red Hat] <christian@python.org> Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org> Reviews via the private PSRT repo via many others (see the NEWS entry in the PR). <!-- gh-issue-number: gh-95778 --> * Issue: gh-95778 <!-- /gh-issue-number --> I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#).
* [3.11] bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185) ↵Eric Snow2022-05-281-3/+0
| | | | | | | | | | | | | (GH-93306) (cherry picked from commit caa279d6fd5f151e57f891cd4f6ba51b532501c6) This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing: * the configure option * the macro * the code enabled by the macro Automerge-Triggered-By: GH:ericsnowcurrently
* Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API ↵Victor Stinner2022-04-061-15/+1
| | | | | | | | | | | (GH-32052)" (GH-32343) * Revert "bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API (GH-32054)" This reverts commit f877b40e3f7e0d97878884d80fbec879a85ab7e8. * Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API (GH-32052)" This reverts commit b9a5522dd952125a99ff554f01f311cae25f5a91.
* bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API ↵Victor Stinner2022-04-011-1/+15
| | | | | | | | | | | (GH-32054) Move the private _PyFrameEvalFunction type, and private _PyInterpreterState_GetEvalFrameFunc() and _PyInterpreterState_SetEvalFrameFunc() functions to the internal C API. The _PyFrameEvalFunction callback function type now uses the _PyInterpreterFrame type which is part of the internal C API. Update the _PyFrameEvalFunction documentation.
* bpo-46841: Use inline caching for calls (GH-31709)Brandt Bucher2022-03-071-0/+2
|
* bpo-45459: Use type names in the internal C API (GH-31669)Victor Stinner2022-03-031-7/+7
| | | Replace "struct xxx" with "xxx" types in the internal C API.
* bpo-45953: Statically allocate the main interpreter (and initial thread ↵Eric Snow2022-01-121-1/+29
| | | | | | | | | state). (gh-29883) Previously, the main interpreter was allocated on the heap during runtime initialization. Here we instead embed it into _PyRuntimeState, which means it is statically allocated as part of the _PyRuntime global. The same goes for the initial thread state (of each interpreter, including the main one). Consequently there are fewer allocations during runtime/interpreter init, fewer possible failures, and better memory locality. FYI, this also helps efforts to consolidate globals, which in turns helps work on subinterpreter isolation. https://bugs.python.org/issue45953
* bpo-45953: Statically allocate and initialize global bytes objects. (gh-30096)Eric Snow2022-01-111-2/+0
| | | | | The empty bytes object (b'') and the 256 one-character bytes objects were allocated at runtime init. Now we statically allocate and initialize them. https://bugs.python.org/issue45953
* bpo-46008: Make runtime-global object/type lifecycle functions and state ↵Eric Snow2021-12-091-173/+10
| | | | | | | | | | | | consistent. (gh-29998) This change is strictly renames and moving code around. It helps in the following ways: * ensures type-related init functions focus strictly on one of the three aspects (state, objects, types) * passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter * consistent naming conventions help make what's going on more clear * keeping API related to a type in the corresponding header file makes it more obvious where to look for it https://bugs.python.org/issue46008
* bpo-46008: Move Py*State init into distinct functions. (gh-29977)Eric Snow2021-12-081-1/+5
| | | | | Previously, basic initialization of PyInterprterState happened in PyInterpreterState_New() (along with allocation and adding the new interpreter to the runtime state). This prevented us from initializing interpreter states that were allocated separately (e.g. statically or in a free list). We've addressed that here by factoring out a separate function just for initialization. We've done the same for PyThreadState. _PyRuntimeState was sorted out when we added it since _PyRuntime is statically allocated. However, here we update the existing init code to line up with the functions for PyInterpreterState and PyThreadState. https://bugs.python.org/issue46008
* bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)Eric Snow2021-12-071-11/+13
| | | | | This parallels _PyRuntimeState.interpreters. Doing this helps make it more clear what part of PyInterpreterState relates to its threads. https://bugs.python.org/issue46008
* bpo-45786: Allocate space for frame in frame object. (GH-29729)Mark Shannon2021-11-291-14/+0
|
* bpo-45691: Make array of small ints static to fix use-after-free error. ↵Mark Shannon2021-11-031-14/+0
| | | | (GH-29366)
* Store actual ints, not pointers to them in the interpreter state. (GH-29274)Mark Shannon2021-10-281-1/+1
|
* bpo-45292: [PEP 654] add the ExceptionGroup and BaseExceptionGroup classes ↵Irit Katriel2021-10-221-0/+2
| | | | (GH-28569)
* bpo-45522: Allow to disable freelists on build time (GH-29056)Christian Heimes2021-10-211-0/+37
| | | | | | | | Freelists for object structs can now be disabled. A new ``configure`` option ``--without-freelists`` can be used to disable all freelists except empty tuple singleton. Internal Py*_MAXFREELIST macros can now be defined as 0 without causing compiler warnings and segfaults. Signed-off-by: Christian Heimes <christian@python.org>
* bpo-45020: Add -X frozen_modules=[on|off] to explicitly control use of ↵Eric Snow2021-09-141-0/+3
| | | | | | | frozen modules. (gh-28320) Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file. This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now. https://bugs.python.org/issue45020
* bpo-43977: Use tp_flags for collection matching (GH-25723)Mark Shannon2021-04-301-4/+0
| | | | | | | | | | | | | * Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes. * Set relevant flags on collections.abc.Sequence and Mapping. * Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes. * Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING. * Add NEWS * Remove interpreter-state map_abc and seq_abc fields.
* bpo-43962: Fix _PyInterpreterState_IDIncref() (GH-25683)Victor Stinner2021-04-281-1/+1
| | | | _PyInterpreterState_IDIncref() now calls _PyInterpreterState_IDInitref() and always increments id_refcount.
* bpo-43760: Speed up check for tracing in interpreter dispatch (#25276)Mark Shannon2021-04-131-6/+0
| | | | | | | | | * Remove redundant tracing_possible field from interpreter state. * Move 'use_tracing' from tstate onto C stack, for fastest possible checking in dispatch logic. * Add comments stressing the importance stack discipline when dealing with CFrames. * Add NEWS
* Revert "bpo-40521: Make dtoa bigint free list per-interpreter (GH-24821)" ↵Victor Stinner2021-03-221-4/+0
| | | | | (GH-24964) This reverts commit 5bd1059184b154d339f1bd53d23c98b5bcf14c8c.
* bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (GH-24907)Victor Stinner2021-03-171-1/+1
|
* bpo-40521: Make dtoa bigint free list per-interpreter (GH-24821)junyixie2021-03-131-0/+4
|
* bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)Brandt Bucher2021-02-261-1/+4
| | | | | Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* bpo-40521: Per-interpreter interned strings (GH-20085)Victor Stinner2020-12-261-0/+11
| | | | | | | | | | | Make the Unicode dictionary of interned strings compatible with subinterpreters. Remove the INTERN_NAME_STRINGS macro in typeobject.c: names are always now interned (even if EXPERIMENTAL_ISOLATED_SUBINTERPRETERS macro is defined). _PyUnicode_ClearInterned() now uses PyDict_Next() to no longer allocate memory, to ensure that the interned dictionary is cleared.
* bpo-42745: Make the type cache per-interpreter (GH-23947)Victor Stinner2020-12-261-0/+22
| | | | | | | | | | | Make the type attribute lookup cache per-interpreter. Add private _PyType_InitCache() function, called by PyInterpreterState_New(). Continue to share next_version_tag between interpreters, since static types are still shared by interpreters. Remove MCACHE macro: the cache is no longer disabled if the EXPERIMENTAL_ISOLATED_SUBINTERPRETERS macro is defined.
* bpo-39465: Fix _PyUnicode_FromId() for subinterpreters (GH-20058)Victor Stinner2020-12-251-0/+7
| | | | | | | | | | | | | | | | | Make _PyUnicode_FromId() function compatible with subinterpreters. Each interpreter now has an array of identifier objects (interned strings decoded from UTF-8). * Add PyInterpreterState.unicode.identifiers: array of identifiers objects. * Add _PyRuntimeState.unicode_ids used to allocate unique indexes to _Py_Identifier. * Rewrite the _Py_Identifier structure. Microbenchmark on _PyUnicode_FromId(&PyId_a) with _Py_IDENTIFIER(a): [ref] 2.42 ns +- 0.00 ns -> [atomic] 3.39 ns +- 0.00 ns: 1.40x slower This change adds 1 ns per _PyUnicode_FromId() call in average.
* bpo-42639: Move atexit state to PyInterpreterState (GH-23763)Victor Stinner2020-12-151-3/+15
| | | | | | | | | * Add _PyAtExit_Call() function and remove pyexitfunc and pyexitmodule members of PyInterpreterState. The function logs atexit callback errors using _PyErr_WriteUnraisableMsg(). * Add _PyAtExit_Init() and _PyAtExit_Fini() functions. * Remove traverse, clear and free functions of the atexit module. Co-authored-by: Dong-hee Na <donghee.na@python.org>
* bpo-42639: atexit now logs callbacks exceptions (GH-23771)Victor Stinner2020-12-141-1/+1
| | | | | | | | | | | At Python exit, if a callback registered with atexit.register() fails, its exception is now logged. Previously, only some exceptions were logged, and the last exception was always silently ignored. Add _PyAtExit_Call() function and remove PyInterpreterState.atexit_func member. call_py_exitfuncs() now calls directly _PyAtExit_Call(). The atexit module must now always be built as a built-in module.
* bpo-42639: Cleanup atexitmodule.c (GH-23770)Victor Stinner2020-12-141-2/+2
| | | | | | | | | * Rename "atexitmodule_state" to "struct atexit_state". * Rename "modstate" to "state". * Rename "self" parameter to "module". * test_atexit uses textwrap.dedent(). * Remove _Py_PyAtExit() function: inline it into atexit_exec(). * PyInterpreterState: rename pyexitfunc to atexit_func, rename pyexitmodule to atexit_module.
* bpo-42260: Initialize time and warnings earlier at startup (GH-23249)Victor Stinner2020-11-121-1/+5
| | | | | | | | | | * Call _PyTime_Init() and _PyWarnings_InitState() earlier during the Python initialization. * Inline _PyImportHooks_Init() into _PySys_InitCore(). * The _warnings initialization function no longer call _PyWarnings_InitState() to prevent resetting filters_version to 0. * _PyWarnings_InitState() now returns an int and no longer clear the state in case of error (it's done anyway at Python exit). * Rework init_importlib(), fix refleaks on errors.
* bpo-42260: Add _PyInterpreterState_SetConfig() (GH-23158)Victor Stinner2020-11-041-6/+0
| | | | | | | | * Inline _PyInterpreterState_SetConfig(): replace it with _PyConfig_Copy(). * Add _PyErr_SetFromPyStatus() * Add _PyInterpreterState_GetConfigCopy() * Add a new _PyInterpreterState_SetConfig() function. * Add an unit which gets, modifies, and sets the config.
* bpo-41796: Make _ast module state per interpreter (GH-23024)Victor Stinner2020-11-021-4/+7
| | | | | | | | | | | The ast module internal state is now per interpreter. * Rename "astmodulestate" to "struct ast_state" * Add pycore_ast.h internal header: the ast_state structure is now declared in pycore_ast.h. * Add PyInterpreterState.ast (struct ast_state) * Remove get_ast_state() * Rename get_global_ast_state() to get_ast_state() * PyAST_obj2mod() now handles get_ast_state() failures
* bpo-42208: Call GC collect earlier in PyInterpreterState_Clear() (GH-23044)Victor Stinner2020-10-301-0/+1
| | | | | | | | The last GC collection is now done before clearing builtins and sys dictionaries. Add also assertions to ensure that gc.collect() is no longer called after _PyGC_Fini(). Pass also the tstate to PyInterpreterState_Clear() to pass the correct tstate to _PyGC_CollectNoFail() and _PyGC_Fini().
* bpo-42161: Add _PyLong_GetZero() and _PyLong_GetOne() (GH-22993)Victor Stinner2020-10-261-2/+5
| | | | | | Add _PyLong_GetZero() and _PyLong_GetOne() functions and a new internal pycore_long.h header file. Python cannot be built without small integer singletons anymore.
* bpo-40939: Remove even more references to the old parser (GH-21642)Lysandros Nikolaou2020-07-271-9/+0
| | | Automerge-Triggered-By: @lysnikolaou
* bpo-40521: Optimize PyBytes_FromStringAndSize(str, 0) (GH-21142)Victor Stinner2020-06-251-2/+2
| | | | | | | | | | | | | | | | Always create the empty bytes string singleton. Optimize PyBytes_FromStringAndSize(str, 0): it no longer has to check if the empty string singleton was created or not, it is always available. Add functions: * _PyBytes_Init() * bytes_get_empty(), bytes_new_empty() * bytes_create_empty_string_singleton() * unicode_create_empty_string_singleton() _Py_unicode_state: rename empty structure member to empty_string.
* bpo-40521: Make Unicode latin1 singletons per interpreter (GH-21101)Victor Stinner2020-06-241-0/+3
| | | | | | | | | Each interpreter now has its own Unicode latin1 singletons. Remove "ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS" and "ifdef LATIN1_SINGLETONS": always enable latin1 singletons. Optimize unicode_result_ready(): only attempt to get a latin1 singleton for PyUnicode_1BYTE_KIND.
* bpo-40521: Make empty Unicode string per interpreter (GH-21096)Victor Stinner2020-06-231-0/+2
| | | Each interpreter now has its own empty Unicode string singleton.
* bpo-40521: Make MemoryError free list per interpreter (GH-21086)Victor Stinner2020-06-231-0/+8
| | | | | | | Each interpreter now has its own MemoryError free list: it is not longer shared by all interpreters. Add _Py_exc_state structure and PyInterpreterState.exc_state member. Move also errnomap into _Py_exc_state.
* bpo-40521: Empty frozenset is no longer a singleton (GH-21085)Raymond Hettinger2020-06-231-2/+0
| | | | | | | | | * Revert "bpo-40521: Make the empty frozenset per interpreter (GH-21068)" This reverts commit 261cfedf7657a515e04428bba58eba2a9bb88208. * bpo-40521: Empty frozensets are no longer singletons * Complete the removal of the frozenset singleton
* bpo-40521: Make bytes singletons per interpreter (GH-21074)Victor Stinner2020-06-231-0/+6
| | | | | | Each interpreter now has its own empty bytes string and single byte character singletons. Replace STRINGLIB_EMPTY macro with STRINGLIB_GET_EMPTY() macro.
* bpo-40521: Make the empty frozenset per interpreter (GH-21068)Victor Stinner2020-06-231-0/+2
| | | Each interpreter now has its own empty frozenset singleton.
* bpo-40521: Make dict free lists per-interpreter (GH-20645)Victor Stinner2020-06-231-13/+24
| | | | | | | | | | | Each interpreter now has its own dict free list: * Move dict free lists into PyInterpreterState. * Move PyDict_MAXFREELIST define to pycore_interp.h * Add _Py_dict_state structure. * Add tstate parameter to _PyDict_ClearFreeList() and _PyDict_Fini(). * In debug mode, ensure that the dict free lists are not used after _PyDict_Fini() is called. * Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
* bpo-39465: Use _PyInterpreterState_GET() (GH-20788)Victor Stinner2020-06-101-7/+7
| | | | | | | | | | | | Replace _PyThreadState_GET() with _PyInterpreterState_GET() in: * get_small_int() * gcmodule.c: add also get_gc_state() function * _PyTrash_deposit_object() * _PyTrash_destroy_chain() * warnings_get_state() * Py_GetRecursionLimit() Cleanup listnode.c: add 'parser' variable.
* bpo-40521: Make context free list per-interpreter (GH-20644)Victor Stinner2020-06-051-0/+7
| | | | | | | | | Each interpreter now has its own context free list: * Move context free list into PyInterpreterState. * Add _Py_context_state structure. * Add tstate parameter to _PyContext_ClearFreeList() and _PyContext_Fini(). * Pass tstate to clear_freelists().
* bpo-40521: Make async gen free lists per-interpreter (GH-20643)Victor Stinner2020-06-051-0/+18
| | | | | | | | | Each interpreter now has its own asynchronous generator free lists: * Move async gen free lists into PyInterpreterState. * Move _PyAsyncGen_MAXFREELIST define to pycore_interp.h * Add _Py_async_gen_state structure. * Add tstate parameter to _PyAsyncGen_ClearFreeLists and _PyAsyncGen_Fini().
* bpo-40521: Make list free list per-interpreter (GH-20642)Victor Stinner2020-06-051-0/+11
| | | | | | | | | | | | Each interpreter now has its own list free list: * Move list numfree and free_list into PyInterpreterState. * Add _Py_list_state structure. * Add tstate parameter to _PyList_ClearFreeList() and _PyList_Fini(). * Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS". * _PyGC_Fini() clears gcstate->garbage list which can be stored in the list free list. Call _PyGC_Fini() before _PyList_Fini() to prevent leaking this list.
* bpo-40521: Make frame free list per-interpreter (GH-20638)Victor Stinner2020-06-041-0/+7
| | | | | | | | | | Each interpreter now has its own frame free list: * Move frame free list into PyInterpreterState. * Add _Py_frame_state structure. * Add tstate parameter to _PyFrame_ClearFreeList() and _PyFrame_Fini(). * Remove "#if PyFrame_MAXFREELIST > 0". * Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
* bpo-40521: Make slice cache per-interpreter (GH-20637)Victor Stinner2020-06-041-0/+4
| | | | | | Each interpreter now has its own slice cache: * Move slice cache into PyInterpreterState. * Add tstate parameter to _PySlice_Fini().
generated by cgit v0.12 at 2026-01-10 02:00:10 (GMT)