summaryrefslogtreecommitdiffstats
path: root/Include
Commit message (Collapse)AuthorAgeFilesLines
* Python 3.14.0b2v3.14.0b2Hugo van Kemenade2025-05-261-2/+2
|
* [3.14] gh-91048: Refactor and optimize remote debugging module (#134652) ↵Pablo Galindo Salgado2025-05-257-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#134673) gh-91048: Refactor and optimize remote debugging module (#134652) Completely refactor Modules/_remote_debugging_module.c with improved code organization, replacing scattered reference counting and error handling with centralized goto error paths. This cleanup improves maintainability and reduces code duplication throughout the module while preserving the same external API. Implement memory page caching optimization in Python/remote_debug.h to avoid repeated reads of the same memory regions during debugging operations. The cache stores previously read memory pages and reuses them for subsequent reads, significantly reducing system calls and improving performance. Add code object caching mechanism with a new code_object_generation field in the interpreter state that tracks when code object caches need invalidation. This allows efficient reuse of parsed code object metadata and eliminates redundant processing of the same code objects across debugging sessions. Optimize memory operations by replacing multiple individual structure copies with single bulk reads for the same data structures. This reduces the number of memory operations and system calls required to gather debugging information from the target process. Update Makefile.pre.in to include Python/remote_debug.h in the headers list, ensuring that changes to the remote debugging header force proper recompilation of dependent modules and maintain build consistency across the codebase. Also, make the module compatible with the free threading build as an extra :) Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit 42b25ad4d3d6bcdc28ddfe07d2bf8831378bb0d1)
* [3.14] gh-115999: Add PyCodeObject.co_tlbc to the debug offsets (GH-134286) ↵Miss Islington (bot)2025-05-251-0/+8
| | | | | | | | | (#134348) gh-115999: Add PyCodeObject.co_tlbc to the debug offsets (GH-134286) (cherry picked from commit dd7f1130570d50461b2a0f81ab01c55b9ce93700) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.14] GH-130397: remove special-casing of C stack depth for WASI ↵Miss Islington (bot)2025-05-221-3/+0
| | | | | | | | | | | | (GH-134469) (GH-134547) GH-130397: remove special-casing of C stack depth for WASI (GH-134469) Removed special-casing for WASI when setting C stack depth limits. Since WASI has its own C stack checking this isn't a security risk. Also disabled some tests that stopped passing. They all happened to have already been disabled under Emscripten. (cherry picked from commit ad42dc1909bdf8ec775b63fb22ed48ff42797a17) Co-authored-by: Brett Cannon <brett@python.org>
* [3.14] gh-132775: Make _PyXI_session Opaque (gh-134522)Miss Islington (bot)2025-05-221-50/+8
| | | | | | | | | | | | | This is mostly a refactor to clean things up a bit, most notably the "XI namespace" code. Making the session opaque requires adding the following internal-only functions: * _PyXI_NewSession() * _PyXI_FreeSession() * _PyXI_GetMainNamespace() (cherry picked from commit 4a4ac3ab4d, gh-134452) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.14] gh-132775: Use _PyFunction_VerifyStateless() and ↵Miss Islington (bot)2025-05-211-2/+2
| | | | | | | _PyCode_VerifyStateless() (gh-134465) (cherry picked from commit a66bae8bb, AKA gh-134439) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.14] gh-132775: Support Fallbacks in _PyObject_GetXIData() (gh-134418)Miss Islington (bot)2025-05-212-4/+25
| | | | | | | | It now supports a "full" fallback to _PyFunction_GetXIData() and then `_PyPickle_GetXIData()`. There's also room for other fallback modes if that later makes sense. (cherry picked from commit 88f8102a8f, AKA gh-133482) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.14] gh-129748: Update mimalloc to use atomic store for mi_block_set_nextx ↵Miss Islington (bot)2025-05-202-5/+5
| | | | | | | | (GH-134238) (gh-134352) gh-129748: Update mimalloc to use atomic store for mi_block_set_nextx (GH-134238) (cherry picked from commit 317c49622397222b7c7fb49837e6b1fd7e82a80d) Co-authored-by: Donghee Na <donghee.na@python.org>
* [3.14] gh-132983: Convert zstd ``__new__`` methods to Argument Clinic ↵Miss Islington (bot)2025-05-204-7/+0
| | | | | (GH-133860) (#133915) Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* [3.14] gh-133779: Revert Windows generation of pyconfig.h and go back to a ↵Steve Dower2025-05-191-0/+13
| | | | | | static header. (GH-133966) Extension builders must specify Py_GIL_DISABLED if they want to link to the free-threaded builds. This was usually the case already, but this change guarantees it in all circumstances.
* [3.14] gh-128045: Mark unknown opcodes as deopting to themselves (GH-128044) ↵Miss Islington (bot)2025-05-191-0/+29
| | | | | (#134228) * gh-128045: Mark unknown opcodes as deopting to themselves (GH-128044)
* [3.14] gh-132641: fix race in `lru_cache` under free-threading (GH-133787) ↵Miss Islington (bot)2025-05-141-0/+2
| | | | | | | | | | (#133979) gh-132641: fix race in `lru_cache` under free-threading (GH-133787) Fix race in `lru_cache` by acquiring critical section on the cache object itself and call the lock held variant of dict functions to modify the underlying dict. (cherry picked from commit 9ad0c7b0f14c5fcda6bfae6692c88abb95502d38) Co-authored-by: Peter Hawkins <phawkins@google.com>
* [3.14] gh-133767: Fix use-after-free in the unicode-escape decoder with an ↵Miss Islington (bot)2025-05-132-6/+11
| | | | | | | | | | | | | | | | | error handler (GH-129648) (GH-133942) If the error handler is used, a new bytes object is created to set as the object attribute of UnicodeDecodeError, and that bytes object then replaces the original data. A pointer to the decoded data will became invalid after destroying that temporary bytes object. So we need other way to return the first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal(). _PyBytes_DecodeEscape() does not have such issue, because it does not use the error handlers registry, but it should be changed for compatibility with _PyUnicode_DecodeUnicodeEscapeInternal(). (cherry picked from commit 9f69a58623bd01349a18ba0c7a9cb1dad6a51e8e) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.14] gh-132775: Add _PyFunction_GetXIData() (gh-133955)Miss Islington (bot)2025-05-121-0/+7
| | | | | (cherry picked from commit 8cf4947b0f, AKA gh-133481) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.14] gh-132775: Add _PyCode_GetScriptXIData() (gh-133676)Miss Islington (bot)2025-05-082-0/+9
| | | | | | | | | | | This converts functions, code, str, bytes, bytearray, and memoryview objects to PyCodeObject, and ensure that the object looks like a script. That means no args, no return, and no closure. _PyCode_GetPureScriptXIData() takes it a step further and ensures there are no globals. We also add _PyObject_SupportedAsScript() to the internal C-API. (cherry picked from commit c81fa2b9cd1ed4d4ff5c13984d6582c27e5f2633, AKA gh-133480) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.14] gh-133583: Add support for fixed size unsigned integers in argument ↵Miss Islington (bot)2025-05-081-0/+5
| | | | | | | | | | | parsing (GH-133584) (GH-133650) * Add Argument Clinic converters: uint8, uint16, uint32, uint64. * Add private C API: _PyLong_UInt8_Converter(), _PyLong_UInt16_Converter(), _PyLong_UInt32_Converter(), _PyLong_UInt64_Converter(). (cherry picked from commit 4c914e7a36033d5e8d52cdef3a4063eb9b012ed2) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.14] gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133625)Eric Snow2025-05-083-0/+50
| | | | | | | | This reverts commit 3c73cf5 (gh-133497), which itself reverted the original commit d270bb5 (gh-133221). We reverted the original change due to failing android tests. The checks in _PyCode_CheckNoInternalState() were too strict, so we've relaxed them.
* Post 3.14.0b1Hugo van Kemenade2025-05-071-1/+1
|
* Python 3.14.0b1v3.14.0b1Hugo van Kemenade2025-05-061-3/+3
|
* Revert "gh-133395: add option for extension modules to specialize ↵Irit Katriel2025-05-065-22/+9
| | | | BINARY_OP/SUBSCR, apply to arrays (#133396)" (#133498)
* gh-132775: Revert "gh-132775: Add _PyCode_VerifyStateless() (gh-133221)" ↵Petr Viktorin2025-05-063-50/+0
| | | | (#133497)
* Remove duplicate includes: Include/internal/ (#132624)Adam Turner2025-05-062-4/+0
|
* gh-132775: Add _PyCode_GetXIData() (gh-133475)Eric Snow2025-05-051-0/+7
|
* gh-132775: Add _PyCode_VerifyStateless() (gh-133221)Eric Snow2025-05-053-0/+50
| | | | | | | | "Stateless" code is a function or code object which does not rely on external state or internal state. It may rely on arguments and builtins, but not globals or a closure. I've left a comment in pycore_code.h that provides more detail. We also add _PyFunction_VerifyStateless(). The new functions will be used in several later changes that facilitate "sharing" functions and code objects between interpreters.
* GH-131798: Split CALL_LEN into several uops (GH-133180)Diego Russo2025-05-053-197/+202
|
* gh-132917: Use RSS + swap for estimate of process memory usage (gh-133464)Neil Schemenauer2025-05-051-3/+3
|
* gh-132775: Unrevert "Add _PyCode_GetVarCounts()" (gh-133265)Eric Snow2025-05-052-0/+56
| | | | | | This reverts commit 811edcf (gh-133232), which itself reverted the original commit 811edcf (gh-133128). We reverted the original change due to failing s390 builds (a big-endian architecture). It ended up that I had not accommodated op caches.
* gh-133140: Add `PyUnstable_Object_IsUniquelyReferenced` for free-threading ↵Peter Bierma2025-05-051-0/+2
| | | | (#133144)
* gh-132917: Check resident set size (RSS) before GC trigger. (gh-133399)Neil Schemenauer2025-05-051-0/+10
| | | | | For the free-threaded build, check the process resident set size (RSS) increase before triggering a full automatic garbage collection. If the RSS has not increased 10% since the last collection then it is deferred.
* gh-133395: add option for extension modules to specialize BINARY_OP/SUBSCR, ↵Irit Katriel2025-05-055-9/+22
| | | | apply to arrays (#133396)
* gh-133371: Don't optimize `LOAD_FAST` instructions whose local is killed by ↵mpage2025-05-051-1/+2
| | | | | | | | `DELETE_FAST` (#133383) In certain cases it's possible for locals loaded by `LOAD_FAST` instructions to be on the stack when the local is killed by `DEL_FAST`. These `LOAD_FAST` instructions should not be optimized into `LOAD_FAST_BORROW` as the strong reference in the frame is killed while there is still a reference on the stack.
* gh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after ↵Yan Yanchii2025-05-041-3/+3
| | | | moving const folding to the peephole optimizier (#131830)
* bpo-44172: Keep reference to original window in curses subwindow objects ↵Michael Forney2025-05-041-1/+2
| | | | | | | | | | | | | | | | | | (GH-26226) The X/Open curses specification[0] and ncurses documentation[1] both state that subwindows must be deleted before the main window. Deleting the windows in the wrong order causes a double-free with NetBSD's curses implementation. To fix this, keep track of the original window object in the subwindow object, and keep a reference to the original for the lifetime of the subwindow. [0] https://pubs.opengroup.org/onlinepubs/7908799/xcurses/delwin.html [1] https://invisible-island.net/ncurses/man/curs_window.3x.html Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* GH-133231: Changes to executor management to support proposed `sys._jit` ↵Mark Shannon2025-05-045-4/+13
| | | | | | | | module (GH-133287) * Track the current executor, not the previous one, on the thread-state. * Batch executors for deallocation to avoid having to constantly incref executors; this is an ad-hoc form of deferred reference counting.
* gh-132983: Introduce `_zstd` bindings module (GH-133027)Emma Smith2025-05-044-0/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add _zstd module for https://peps.python.org/pep-0784/ This commit introduces the `_zstd` module, with bindings to libzstd from the pyzstd project. It also includes the unix build system configuration. Windows build system support will be integrated independently as it depends on integration with cpython-source-deps. * Add _zstd to modules * Fix path for compression.zstd module * Ignore _zstd module like _io * Expand module state macros to improve code quality Also removes module state references from the classes in the _zstd module and instead uses PyType_GetModuleState() * Remove backticks suggested in review Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> * Use critical sections to lock object state This should avoid races and deadlocks. * Remove compress/decompress and mark module as not reliant on the GIL The `compress`/`decompress` functions will be moved to Python code for simplicity. C implementations can always be re-added in the future. Also, mark _zstd as not requiring the GIL. * Lift critical section to avoid clang warning * Respond to comments by picnixz * Call out pyzstd explicitly in license description Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> * Use a much more robust implementation... ... for `get_zstd_state_from_type` Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> * Use PyList_GetItemRef for thread safety purposes * Use a macro for the minimum supported version * remove const from primivite types * Use PyMem_New in another spot * Simplify error handling in _get_frame_size * Another simplification of error handling in get_frame_info * Rename _module_state to mod_state * Rewrite comment explaining the context of the code * Add link to pyzstd * Add TODO about refactoring dict training code * Use PyModule_AddObjectRef over PyModule_AddObject PyModule_AddObject is soft-deprecated, so we should use PyModule_AddObjectRef * Check result of OutputBufferGrow * Simplify return logic in `add_constant_to_type` Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> * Ignore return value of _zstd_clear() Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> * Remove redundant comments * Remove __reduce__ from ZstdDict We should instead document that to pickle a dictionary a user should use the `.dict_content` attribute. * Use PyUnicode_FromFormat instead of a buffer * Don't use C constants/types in error messages * Make error messages easier to understand for Python users * Lower minimum required version 1.4.0 * Use casts and make slot function signatures correct * Be consistent with CPython on const usage * Make else clauses in line with PEP 7 * Fix over-indented blocks in argument clinic * Add critical section around ZSTD_DCtx_setParameter * Add a TODO about refactoring critical sections * Use Py_UNREACHABLE * Move bytes operations out of Py_BEGIN_ALLOW_THREADS * Add TODO about ensuring a lock is held * Remove asserts that may not be correct * Add TODO to make ZstdDict and others GC objects * Make objects GC tracked * Remove unused include * Fix some memory issues * Fix refleaks on module and in ZstdDict * Update configure to check for ZDICT_finalizeDictionary * Properly check version in configure * exit(1) if check fails * Use AC_RUN_IFELSE * Use a define() to re-use version check * Actually properly set _zstd module status based on version --------- Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* gh-132744: Check recursion limit in CALL_PY_GENERAL (GH-132746)Ken Jin2025-05-023-170/+175
|
* gh-128972: Add `_Py_ALIGN_AS` and revert `PyASCIIObject` memory layout. ↵Petr Viktorin2025-05-022-10/+61
| | | | | | | | | | | | | (GH-133085) Add `_Py_ALIGN_AS` as per C API WG vote: https://github.com/capi-workgroup/decisions/issues/61 This patch only adds it to free-threaded builds; the `#ifdef Py_GIL_DISABLED` can be removed in the future. Use this to revert `PyASCIIObject` memory layout for non-free-threaded builds. The long-term plan is to deprecate the entire struct; until that happens it's better to keep it unchanged, as courtesy to people that rely on it despite it not being stable ABI.
* gh-133164: Add `PyUnstable_Object_IsUniqueReferencedTemporary` C API (gh-133170)Sam Gross2025-05-021-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | After gh-130704, the interpreter replaces some uses of `LOAD_FAST` with `LOAD_FAST_BORROW` which avoid incref/decrefs by "borrowing" references on the interpreter stack when the bytecode compiler can determine that it's safe. This change broke some checks in C API extensions that relied on `Py_REFCNT()` of `1` to determine if it's safe to modify an object in-place. Objects may have a reference count of one, but still be referenced further up the interpreter stack due to borrowing of references. This provides a replacement function for those checks. `PyUnstable_Object_IsUniqueReferencedTemporary` is more conservative: it checks that the object has a reference count of one and that it exists as a unique strong reference in the interpreter's stack of temporary variables in the top most frame. See also: * https://github.com/numpy/numpy/issues/28681 Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com> Co-authored-by: T. Wouters <thomas@python.org> Co-authored-by: mpage <mpage@cs.stanford.edu> Co-authored-by: Mark Shannon <mark@hotpy.org> Co-authored-by: Victor Stinner <vstinner@python.org>
* gh-133258: Fix crash in test_index (GH-133262)Irit Katriel2025-05-012-2/+2
|
* gh-133261: Use __builtin_frame_address() on GCC 9 and older (#133269)Victor Stinner2025-05-011-2/+2
| | | | GCC 9 and older don't have __has_builtin(), but have __builtin_frame_address() function.
* gh-133256: Add _Py_NONSTRING macro (#133257)Victor Stinner2025-05-012-1/+16
| | | | | | | | | Fix GCC 15 compiler warnings such as: Modules/fcntlmodule.c:27:36: warning: initializer-string for array of 'char' truncates NUL terminator but destination lacks 'nonstring' attribute (9 chars into 8 available) [-Wunterminated-string-initialization] static const char guard[GUARDSZ] = "\x00\xfa\x69\xc4\x67\xa3\x6c\x58";
* gh-100239: specialize BINARY_OP/SUBSCR for list-slice (#132626)Irit Katriel2025-05-016-274/+295
|
* PyStats: Make sure that the `failure_kinds` array is big enough. (#133245)Mark Shannon2025-05-011-1/+1
|
* Revert "gh-132775: Add _PyCode_GetVarCounts() (gh-133128)" (gh-133232)Eric Snow2025-05-012-56/+0
| | | | | The change broke the s390 builds, so I'm reverting it while I investigate. This reverts commit 94b4fcd806e7b692955173d309ea3b70a193ad96.
* gh-132775: Add _PyPickle_GetXIData() (gh-133107)Eric Snow2025-04-301-0/+7
| | | There's some extra complexity due to making sure we we get things right when handling functions and classes defined in the __main__ module. This is also reflected in the tests, including the addition of extra functions in test.support.import_helper.
* gh-132775: Add _PyCode_GetVarCounts() (gh-133128)Eric Snow2025-04-302-0/+56
| | | | | | | This helper is useful in a variety of ways, including in demonstrating how the different counts relate to one another. It will be used in a later change to help identify if a function is "stateless", meaning it doesn't have any free vars or globals. Note that a majority of this change is tests.
* GH-124715: Move trashcan mechanism into `Py_Dealloc` (GH-132280)Mark Shannon2025-04-304-99/+45
|
* gh-132661: Implement PEP 750 (#132662)Lysandros Nikolaou2025-04-3014-127/+245
| | | | | | | | | | | | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Wingy <git@wingysam.xyz> Co-authored-by: Koudai Aono <koxudaxi@gmail.com> Co-authored-by: Dave Peck <davepeck@gmail.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Paul Everitt <pauleveritt@me.com> Co-authored-by: sobolevn <mail@sobolevn.me>
* GH-132508: Use tagged integers on the evaluation stack for the last ↵Mark Shannon2025-04-292-10/+69
| | | | instruction offset (GH-132545)
* gh-133079: Remove Py_C_RECURSION_LIMIT & PyThreadState.c_recursion_remaining ↵Petr Viktorin2025-04-291-4/+0
| | | | | | | | (GH-133080) Both were added in 3.13, are undocumented, and don't make sense in 3.14 due to changes in the stack overflow detection machinery (gh-112282). PEP 387 exception for skipping a deprecation period: https://github.com/python/steering-council/issues/288
generated by cgit v0.12 at 2026-03-20 14:36:19 (GMT)