summaryrefslogtreecommitdiffstats
path: root/Modules/_pickle.c
Commit message (Collapse)AuthorAgeFilesLines
* [3.13] gh-130163: Fix a leak in _pickle.c after backporting (GH-130568)Serhiy Storchaka2025-02-261-0/+7
|
* [3.13] gh-130163: Fix crashes related to PySys_GetObject() (GH-130503) ↵Serhiy Storchaka2025-02-251-4/+2
| | | | | | | | | | (GH-130556) The use of PySys_GetObject() and _PySys_GetAttr(), which return a borrowed reference, has been replaced by using one of the following functions, which return a strong reference and distinguish a missing attribute from an error: _PySys_GetOptionalAttr(), _PySys_GetOptionalAttrString(), _PySys_GetRequiredAttr(), and _PySys_GetRequiredAttrString(). (cherry picked from commit 0ef4ffeefd1737c18dc9326133c7894d58108c2e)
* [3.13] gh-130179: Fix `persistent_{id,load}_attr` reference leaks in ↵Miss Islington (bot)2025-02-161-0/+4
| | | | | | | | | `_pickle` (GH-130180) (#130190) gh-130179: Fix `persistent_{id,load}_attr` reference leaks in `_pickle` (GH-130180) (cherry picked from commit e7f00cd14f6a0c0dee6e733ac3e2c5d92e0809bb) Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* [3.13] gh-126991: Fix reference leak in loading pickle's opcode BUILD ↵Miss Islington (bot)2024-11-191-0/+1
| | | | | | | | (GH-126990) (GH-127018) If PyObject_SetItem() fails in the `load_build()` function of _pickle.c, no DECREF for the `dict` variable. (cherry picked from commit 29cbcbd73bbfd8c953c0b213fb33682c289934ff) Co-authored-by: Justin Applegate <70449145+Legoclones@users.noreply.github.com>
* [3.13] gh-125631: Enable setting persistent_id and persistent_load of ↵Miss Islington (bot)2024-11-071-0/+62
| | | | | | | | | pickler and unpickler (GH-125752) (GH-126528) pickle.Pickler.persistent_id and pickle.Unpickler.persistent_load can again be overridden as instance attributes. (cherry picked from commit 223d3dc554dde45f185f7f465753824c6f698b9b) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.13] gh-126461: Fix _Unpickler_ReadFromFile() error handling (GH-126485) ↵Miss Islington (bot)2024-11-061-0/+4
| | | | | | | | | | (#126495) gh-126461: Fix _Unpickler_ReadFromFile() error handling (GH-126485) Handle _Unpickler_SetStringInput() failure. (cherry picked from commit a1c57bcfd2bcbc55ff858407e09c1d8d8cee44e6) Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.13] gh-123431: Harmonize extension code checks in pickle (GH-123434) ↵Miss Islington (bot)2024-09-021-21/+11
| | | | | | | | | | | | | | | | (#123459) gh-123431: Harmonize extension code checks in pickle (GH-123434) This checks are redundant in normal circumstances and can only work if the extension registry was intentionally broken. * The Python implementation now raises exception for the extension code with false boolean value. * Simplify the C code. RuntimeError is now raised in explicit checks. * Add many tests. (cherry picked from commit 0c3ea3023878f5ad5ca4680d5510da1fe208cbfa) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.13] gh-122311: Fix a refleak in pickle (GH-122411) (GH-122415)Miss Islington (bot)2024-07-291-0/+1
| | | | | (cherry picked from commit 68840e91ac6689d3954b98a9ab136e194b5250b8) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.13] gh-122311: Fix some error messages in pickle (GH-122386) (GH-122387)Miss Islington (bot)2024-07-291-3/+3
| | | | | (cherry picked from commit 3b034d26eb8480f8d12ae11f42d038d24cf8498a) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.13] gh-82951: Fix serializing by name in pickle protocols < 4 (GH-122149) ↵Miss Islington (bot)2024-07-251-17/+36
| | | | | | | | | | (GH-122264) Serializing objects with complex __qualname__ (such as unbound methods and nested classes) by name no longer involves serializing parent objects by value in pickle protocols < 4. (cherry picked from commit dc07f65a53baf60d9857186294d3d7ba92d5606d) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.13] gh-121137: Add missing Py_DECREF calls for ADDITEMS opcode of ↵Miss Islington (bot)2024-06-281-0/+2
| | | | | | | | | | _pickle.c (GH-121136) (#121139) gh-121137: Add missing Py_DECREF calls for ADDITEMS opcode of _pickle.c (GH-121136) PyObject_GetAttr returns a new reference, but this reference is never decremented using Py_DECREF, so Py_DECREF calls to this referece are added (cherry picked from commit 92893fd8dc803ed7cdde55d29d25f84ccb5e3ef0) Co-authored-by: Justin Applegate <70449145+Legoclones@users.noreply.github.com>
* [3.13] gh-113993: Allow interned strings to be mortal, and fix related ↵Petr Viktorin2024-06-241-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | issues (GH-120520) (GH-120945) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. * Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* gh-116322: Add Py_mod_gil module slot (#116882)Brett Simmers2024-05-031-0/+1
| | | | | | | | | | | | | | This PR adds the ability to enable the GIL if it was disabled at interpreter startup, and modifies the multi-phase module initialization path to enable the GIL when loading a module, unless that module's spec includes a slot indicating it can run safely without the GIL. PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148. A warning will be issued up to once per interpreter for the first GIL-using module that is loaded. If `-v` is given, a shorter message will be printed to stderr every time a GIL-using module is loaded (including the first one that issues a warning).
* gh-112069: Add _PySet_NextEntryRef to be thread-safe. (gh-117990)Donghee Na2024-04-181-13/+20
|
* gh-111140: Adds PyLong_AsNativeBytes and PyLong_FromNative[Unsigned]Bytes ↵Steve Dower2024-02-121-1/+2
| | | | functions (GH-114886)
* gh-89850: Add default C implementations of persistent_id() and ↵Serhiy Storchaka2024-01-101-210/+107
| | | | | | | | | | | persistent_load() (GH-113579) Previously the C implementation of pickle.Pickler and pickle.Unpickler classes did not have such methods and they could only be used if they were overloaded in subclasses or set as instance attributes. Fixed calling super().persistent_id() and super().persistent_load() in subclasses of the C implementation of pickle.Pickler and pickle.Unpickler classes. It no longer causes an infinite recursion.
* bpo-41422: Visit the Pickler's and Unpickler's memo in tp_traverse (GH-21664)kale-smoothie2023-11-271-0/+15
| | | Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-108511: Add C API functions which do not silently ignore errors (GH-109025)Serhiy Storchaka2023-09-171-4/+3
| | | | | | | | | Add the following functions: * PyObject_HasAttrWithError() * PyObject_HasAttrStringWithError() * PyMapping_HasKeyWithError() * PyMapping_HasKeyStringWithError()
* gh-106320: Remove private _PySys functions (#108452)Victor Stinner2023-08-241-1/+2
| | | | | | | | | | Move private functions to the internal C API (pycore_sysmodule.h): * _PySys_GetAttr() * _PySys_GetSizeOf() No longer export most of these functions. Fix also a typo in Include/cpython/optimizer.h: add a missing space.
* gh-106320: Remove private PyLong C API functions (#108429)Victor Stinner2023-08-241-0/+1
| | | | | | | | | | | | | | | | Remove private PyLong C API functions: * _PyLong_AsByteArray() * _PyLong_DivmodNear() * _PyLong_Format() * _PyLong_Frexp() * _PyLong_FromByteArray() * _PyLong_FromBytes() * _PyLong_GCD() * _PyLong_Lshift() * _PyLong_Rshift() Move these functions to the internal C API. No longer export _PyLong_FromBytes() function.
* GH-84436: Skip refcounting for known immortals (GH-107605)Brandt Bucher2023-08-041-2/+1
|
* gh-106869: Use new PyMemberDef constant names (#106871)Victor Stinner2023-07-251-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove '#include "structmember.h"'. * If needed, add <stddef.h> to get offsetof() function. * Update Parser/asdl_c.py to regenerate Python/Python-ast.c. * Replace: * T_SHORT => Py_T_SHORT * T_INT => Py_T_INT * T_LONG => Py_T_LONG * T_FLOAT => Py_T_FLOAT * T_DOUBLE => Py_T_DOUBLE * T_STRING => Py_T_STRING * T_OBJECT => _Py_T_OBJECT * T_CHAR => Py_T_CHAR * T_BYTE => Py_T_BYTE * T_UBYTE => Py_T_UBYTE * T_USHORT => Py_T_USHORT * T_UINT => Py_T_UINT * T_ULONG => Py_T_ULONG * T_STRING_INPLACE => Py_T_STRING_INPLACE * T_BOOL => Py_T_BOOL * T_OBJECT_EX => Py_T_OBJECT_EX * T_LONGLONG => Py_T_LONGLONG * T_ULONGLONG => Py_T_ULONGLONG * T_PYSSIZET => Py_T_PYSSIZET * T_NONE => _Py_T_NONE * READONLY => Py_READONLY * PY_AUDIT_READ => Py_AUDIT_READ * READ_RESTRICTED => Py_AUDIT_READ * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
* gh-106320: Move private _PySet API to the internal API (#107041)Victor Stinner2023-07-221-0/+1
| | | | | | | | * Add pycore_setobject.h header file. * Move the following API to the internal C API: * _PySet_Dummy * _PySet_NextEntry() * _PySet_Update()
* gh-106320: Move _PyNone_Type to the internal C API (#107030)Victor Stinner2023-07-221-0/+1
| | | | Move private types _PyNone_Type and _PyNotImplemented_Type to internal C API.
* gh-106521: Remove _PyObject_LookupAttr() function (GH-106642)Serhiy Storchaka2023-07-121-19/+19
|
* gh-106307: C API: Add PyMapping_GetOptionalItem() function (GH-106308)Serhiy Storchaka2023-07-111-9/+6
| | | | Also add PyMapping_GetOptionalItemString() function.
* gh-106320: Remove _PyBytesWriter C API (#106399)Victor Stinner2023-07-041-1/+2
| | | | Remove the _PyBytesWriter C API: move it to the internal C API (pycore_bytesobject.h).
* gh-105375: Improve error handling in _Unpickler_SetInputStream() (#105667)Erlend E. Aasland2023-06-131-13/+18
| | | | Prevent exceptions from possibly being overwritten in case of multiple failures.
* gh-105375: Explicitly initialise all {Pickler,Unpickler}Object fields (#105686)Erlend E. Aasland2023-06-121-37/+53
| | | | | | All fields must be explicitly initialised to prevent manipulation of uninitialised fields in dealloc. Align initialisation order with the layout of the object structs.
* gh-105375: Improve _pickle error handling (#105475)Erlend E. Aasland2023-06-091-12/+27
| | | | Error handling was deferred in some cases, which could potentially lead to exceptions being overwritten.
* gh-92536: Remove PyUnicode_READY() calls (#105210)Victor Stinner2023-06-011-6/+0
| | | | Since Python 3.12, PyUnicode_READY() does nothing and always returns 0.
* gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)Eric Snow2023-05-051-0/+1
| | | Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules. We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
* gh-103092: Isolate `_pickle` module (#102982)Erlend E. Aasland2023-04-041-653/+609
| | | | Co-authored-by: Mohamed Koubaa <koubaa.m@gmail.com> Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
* gh-102255: Improve build support for Windows API partitions (GH-102256)Max Bachmann2023-03-091-0/+6
| | | | | Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs). CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes. `MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
* gh-99845: Use size_t type in __sizeof__() methods (#99846)Victor Stinner2022-11-301-15/+12
| | | | | | | | The implementation of __sizeof__() methods using _PyObject_SIZE() now use an unsigned type (size_t) to compute the size, rather than a signed type (Py_ssize_t). Cast explicitly signed (Py_ssize_t) values to unsigned type (Py_ssize_t).
* gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)Victor Stinner2022-11-231-2/+1
| | | Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
* gh-99537: Use Py_SETREF() function in C code (#99656)Victor Stinner2022-11-221-5/+2
| | | | | | | | | | | | | | | Fix potential race condition in code patterns: * Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);" * Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);" * Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);" Other changes: * Replace "old = var; var = new; Py_DECREF(var)" with "Py_SETREF(var, new);" * Replace "old = var; var = new; Py_XDECREF(var)" with "Py_XSETREF(var, new);" * And remove the "old" variable.
* gh-99300: Use Py_NewRef() in Modules/ directory (#99467)Victor Stinner2022-11-141-49/+26
| | | | Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in test C files of the Modules/ directory.
* remove new line in pickle exception message (GH-31782)Harshil2022-11-071-2/+2
|
* gh-83004: Clean up refleak in _pickle initialisation (#98841)Shantanu2022-11-061-7/+6
|
* GH-89988: Fix memory leak in pickle.Pickler dispatch_table lookup (GH-94298)Kumar Aditya2022-06-281-1/+3
|
* gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)Serhiy Storchaka2022-06-141-14/+2
| | | | | | It combines PyImport_ImportModule() and PyObject_GetAttrString() and saves 4-6 lines of code on every use. Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
* gh-92930: _pickle.c: Acquire strong references before calling save() (GH-92931)Dennis Sweeney2022-06-111-11/+36
|
* gh-89653: Use int type for Unicode kind (#92704)Victor Stinner2022-05-131-1/+1
| | | | Use the same type that PyUnicode_FromKindAndData() kind parameter type (public C API): int.
* Use static inline function Py_EnterRecursiveCall() (#91988)Victor Stinner2022-05-041-11/+12
| | | | | | | | | | | | | | | | Currently, calling Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() may use a function call or a static inline function call, depending if the internal pycore_ceval.h header file is included or not. Use a different name for the static inline function to ensure that the static inline function is always used in Python internals for best performance. Similar approach than PyThreadState_GET() (function call) and _PyThreadState_GET() (static inline function). * Rename _Py_EnterRecursiveCall() to _Py_EnterRecursiveCallTstate() * Rename _Py_LeaveRecursiveCall() to _Py_LeaveRecursiveCallTstate() * pycore_ceval.h: Rename Py_EnterRecursiveCall() to _Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() and _Py_LeaveRecursiveCall()
* gh-91731: Replace Py_BUILD_ASSERT() with static_assert() (#91730)Victor Stinner2022-04-201-1/+1
| | | | | | | | | | | | Python 3.11 now uses C11 standard which adds static_assert() to <assert.h>. * In pytime.c, replace Py_BUILD_ASSERT() with preprocessor checks on SIZEOF_TIME_T with #error. * On macOS, py_mach_timebase_info() now accepts timebase members with the same size than _PyTime_t. * py_get_monotonic_clock() now saturates GetTickCount64() to _PyTime_MAX: GetTickCount64() is unsigned, whereas _PyTime_t is signed.
* bpo-46712: share more global strings in deepfreeze (gh-32152)Kumar Aditya2022-04-191-1/+1
| | | (for gh-90868)
* bpo-46906: Add PyFloat_Pack8() to the C API (GH-31657)Victor Stinner2022-03-111-3/+2
| | | | | | | | | | | | | | | | | | | | | | Add new functions to pack and unpack C double (serialize and deserialize): * PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8() * PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8() Document these functions and add unit tests. Rename private functions and move them from the internal C API to the public C API: * _PyFloat_Pack2() => PyFloat_Pack2() * _PyFloat_Pack4() => PyFloat_Pack4() * _PyFloat_Pack8() => PyFloat_Pack8() * _PyFloat_Unpack2() => PyFloat_Unpack2() * _PyFloat_Unpack4() => PyFloat_Unpack4() * _PyFloat_Unpack8() => PyFloat_Unpack8() Replace the "unsigned char*" type with "char*" which is more common and easy to use.
* bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized ↵Eric Snow2022-02-081-79/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | global objects. (gh-30928) We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules. The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings). https://bugs.python.org/issue46541#msg411799 explains the rationale for this change. The core of the change is in: * (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros * Include/internal/pycore_runtime_init.h - added the static initializers for the global strings * Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState * Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config. The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *. The following are not changed (yet): * stop using _Py_IDENTIFIER() in the stdlib modules * (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API * (maybe) intern the strings during runtime init https://bugs.python.org/issue46541
* bpo-43974: Move Py_BUILD_CORE_MODULE into module code (GH-29157)Christian Heimes2021-10-221-2/+2
| | | | | | | | | | | | | | setup.py no longer defines Py_BUILD_CORE_MODULE. Instead every module defines the macro before #include "Python.h" unless Py_BUILD_CORE_BUILTIN is already defined. Py_BUILD_CORE_BUILTIN is defined for every module that is built by Modules/Setup. The PR also simplifies Modules/Setup. Makefile and makesetup already define Py_BUILD_CORE_BUILTIN and include Modules/internal for us. Signed-off-by: Christian Heimes <christian@python.org>
generated by cgit v0.12 at 2026-01-07 17:12:51 (GMT)