summaryrefslogtreecommitdiffstats
path: root/Objects/obmalloc.c
Commit message (Collapse)AuthorAgeFilesLines
* bpo-46670: Remove unused macros in the Objects directory (GH-31193)Victor Stinner2022-02-071-1/+0
|
* bpo-46072: Add some object layout and allocation stats (GH-31051)Mark Shannon2022-02-011-0/+4
|
* bpo-40170: Move _Py_GetAllocatedBlocks() to pycore_pymem.h (GH-30943)Victor Stinner2022-01-271-3/+0
| | | | | Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats() declarations to pycore_pymem.h. These functions are related to memory allocators, not to the PyObject structure.
* bpo-40170: Remove _Py_GetAllocatedBlocks() function (GH-30940)Victor Stinner2022-01-271-1/+5
| | | | Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats() private functions to the internal C API.
* bpo-45526: obmalloc radix use 64 addr bits (GH-29062)Neil Schemenauer2021-10-211-20/+35
| | | Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)Neil Schemenauer2021-10-211-1/+1
| | | | | MAP_BOT_LENGTH was incorrectly used to compute MAP_TOP_MASK instead of MAP_TOP_LENGTH. On 64-bit machines, the error causes the tree to hold 46-bits of virtual addresses, rather than the intended 48-bits.
* bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)Victor Stinner2021-10-131-0/+1
| | | | | Include <stdlib.h> explicitly in C files. Python.h includes <wchar.h>.
* Fix typos in the Objects directory (GH-28766)Christian Clauss2021-10-061-1/+1
|
* bpo-44032: Move data stack to thread from FrameObject. (GH-26076)Mark Shannon2021-05-211-1/+13
| | | | | | | | | | | | | | | | * Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames. * Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells. * Move locals, cells and freevars out of frame object into separate memory buffer. * Use per-threadstate allocated memory chunks for local variables. * Move globals and builtins from frame object to per-thread stack. * Move (slow) locals frame object to per-thread stack. * Move internal frame functions to internal header.
* When printing stats, move radix tree info to its own section. (GH-25125)Tim Peters2021-04-011-9/+14
| | | | | | When printing stats, move radix tree info to its own section. Restore that the breakdown of bytes in arenas exactly accounts for the total of arena bytes allocated. Add an assert so that invariant doesn't break again.
* bpo-37448: Use radix tree for pymalloc address_in_range(). (GH-14474)Neil Schemenauer2021-03-301-5/+336
| | | | | | | | The radix tree approach is a relatively simple and memory sanitary alternative to the old (slightly) unsanitary address_in_range(). To disable the radix tree map, set a preprocessor flag as follows: -DWITH_PYMALLOC_RADIX_TREE=0. Co-authored-by: Tim Peters <tim.peters@gmail.com>
* bpo-40943: Replace PY_FORMAT_SIZE_T with "z" (GH-20781)Victor Stinner2020-06-101-11/+8
| | | | | | | The PEP 353, written in 2005, introduced PY_FORMAT_SIZE_T. Python no longer supports macOS 10.4 and Visual Studio 2010, but requires more recent macOS and Visual Studio versions. In 2020 with Python 3.10, it is now safe to use directly "%zu" to format size_t and "%zi" to format Py_ssize_t.
* bpo-40268: Remove unused pycore_pymem.h includes (GH-19531)Victor Stinner2020-04-151-1/+1
|
* bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157)Victor Stinner2020-03-251-21/+11
|
* bpo-39882: Py_FatalError() logs the function name (GH-18819)Victor Stinner2020-03-061-12/+13
| | | | | | | | | | | | The Py_FatalError() function is replaced with a macro which logs automatically the name of the current function, unless the Py_LIMITED_API macro is defined. Changes: * Add _Py_FatalErrorFunc() function. * Remove the function name from the message of Py_FatalError() calls which included the function name. * Update tests.
* closes bpo-30364: Replace deprecated no_address_safety_analysis attribute. ↵Batuhan Taşkaya2019-12-271-7/+7
| | | | (GH-17702)
* bpo-38733: PyErr_Occurred() caller must hold the GIL (GH-17080)Victor Stinner2019-11-071-2/+3
| | | | | | | | | | | bpo-3605, bpo-38733: Optimize _PyErr_Occurred(): remove "tstate == NULL" test. Py_FatalError() no longer calls PyErr_Occurred() if called without holding the GIL. So PyErr_Occurred() no longer has to support tstate==NULL case. _Py_CheckFunctionResult(): use directly _PyErr_Occurred() to avoid explicit "!= NULL" test.
* bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)Victor Stinner2019-10-071-36/+23
| | | | Fix _PyBytesWriter API when Python is built in release mode with assertions.
* bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333)Victor Stinner2019-08-201-21/+16
| | | | | | | pymalloc_alloc() now returns directly the pointer, return NULL on memory allocation error. allocate_from_new_pool() already uses NULL as marker for "allocation failed".
* bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096)Hai Shi2019-08-141-2/+2
| | | | Fix the following warning with GCC 4.8.5: Objects/obmalloc.c: warning: ‘no_sanitize_thread’ attribute directive ignored
* bpo-37543: optimize pymalloc (#14674)Inada Naoki2019-07-171-218/+226
| | | | | | PyObject_Malloc() and PyObject_Free() inlines pymalloc_alloc and pymalloc_free partially. But when PGO is not used, compiler don't know where is the hot part in pymalloc_alloc and pymalloc_free.
* Fix compiler warning in new code. (#14690)Tim Peters2019-07-101-1/+1
| | | uintptr_t is an integer type, and can't be compared to NULL directly.
* bpo-37537: Compute allocated blocks in _Py_GetAllocatedBlocks() (#14680)Neil Schemenauer2019-07-101-7/+22
| | | | | | Keeping an account of allocated blocks slows down _PyObject_Malloc() and _PyObject_Free() by a measureable amount. Have _Py_GetAllocatedBlocks() iterate over the arenas to sum up the allocated blocks for pymalloc.
* bpo-37257: obmalloc: stop simple arena thrashing (#14039)Tim Peters2019-06-131-2/+7
| | | | | GH-14039: allow (no more than) one wholly empty arena on the usable_arenas list. This prevents thrashing in some easily-provoked simple cases that could end up creating and destroying an arena on each loop iteration in client code. Intuitively, if the only arena on the list becomes empty, it makes scant sense to give it back to the system unless we know we'll never need another free pool again before another arena frees a pool. If the latter obtains, then - yes - this will "waste" an arena.
* bpo-37029: keep usable_arenas in sorted order without searching (#13612)Tim Peters2019-06-011-32/+77
| | | | | This adds a vector of "search fingers" so that usable_arenas can be kept in sorted order (by number of free pools) via constant-time operations instead of linear search. This should reduce worst-case time for reclaiming a great many objects from O(A**2) to O(A), where A is the number of arenas. See bpo-37029.
* bpo-36763: Add PyMemAllocatorName (GH-13387)Victor Stinner2019-05-171-14/+59
| | | | | | | | | | | | | | * Add PyMemAllocatorName enum * _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of char* * Remove _PyPreConfig_Clear() * Add _PyMem_GetAllocatorName() * Rename _PyMem_GetAllocatorsName() to _PyMem_GetCurrentAllocatorName() * Remove _PyPreConfig_SetAllocator(): just call _PyMem_SetupAllocators() directly, we don't have do reallocate the configuration with the new allocator anymore! * _PyPreConfig_Write() parameter becomes const, as it should be in the first place!
* bpo-27987: pymalloc: align by 16bytes on 64bit platform (GH-12850)Inada Naoki2019-05-141-0/+6
|
* bpo-36594: Fix incorrect use of %p in format strings (GH-12769)Zackery Spytz2019-05-061-1/+1
| | | In addition, fix some other minor violations of C99.
* bpo-36611: Disable serialno field of debug memory allocators (#12796)Victor Stinner2019-04-121-19/+54
| | | | | | | | | | | Omit serialno field from debug hooks on Python memory allocators to reduce the memory footprint by 5%. Enable tracemalloc to get the traceback where a memory block has been allocated when a fatal memory error is logged to decide where to put a breakpoint. Compile Python with PYMEM_DEBUG_SERIALNO defined to get back the field.
* bpo-36389: Change PyMem_SetupDebugHooks() constants (GH-12782)Victor Stinner2019-04-111-3/+6
| | | | | Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD, 0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte patterns than Windows CRT debug malloc() and free().
* bpo-36389: _PyObject_IsFreed() now also detects uninitialized memory (GH-12770)Victor Stinner2019-04-111-17/+1
| | | | | | | | | Replace _PyMem_IsFreed() function with _PyMem_IsPtrFreed() inline function. The function is now way more efficient, it became a simple comparison on integers, rather than a short loop. It detects also uninitialized bytes and "forbidden bytes" filled by debug hooks on memory allocators. Add unit tests on _PyObject_IsFreed().
* bpo-36142: Add _PyPreConfig_SetAllocator() (GH-12187)Victor Stinner2019-03-061-14/+0
| | | | | | | | | | | * _PyPreConfig_Write() now reallocates the pre-configuration with the new memory allocator. * It is no longer needed to force the "default raw memory allocator" to clear pre-configuration and core configuration. Simplify the code. * _PyPreConfig_Write() now does nothing if called after Py_Initialize(): no longer check if the allocator is the same. * Remove _PyMem_GetDebugAllocatorsName(): dev mode sets again allocator to "debug".
* bpo-36142: Add _PyMem_GetDebugAllocatorsName() (GH-12185)Victor Stinner2019-03-051-0/+14
| | | | | The development mode now uses the effective name of the debug memory allocator ("pymalloc_debug" or "malloc_debug"). So the name doesn't change after setting the memory allocator.
* bpo-35081: Rename internal headers (GH-10275)Victor Stinner2018-11-121-1/+1
| | | | | | | | | | | | | | Rename Include/internal/ headers: * pycore_hash.h -> pycore_pyhash.h * pycore_lifecycle.h -> pycore_pylifecycle.h * pycore_mem.h -> pycore_pymem.h * pycore_state.h -> pycore_pystate.h Add missing headers to Makefile.pre.in and PCbuild: * pycore_condvar.h. * pycore_hamt.h * pycore_pyhash.h
* closes bpo-35204: Disable thread and memory sanitizers for ↵Alexey Izbyshev2018-11-111-12/+31
| | | | | | | | | address_in_range(). (GH-10442) This function may access memory which is mapped but is considered free by libc allocator. It behaves so by design, therefore we need to suppress sanitizer reports. GCC doesn't support MSan, so disable only TSan for it.
* bpo-35081: Add pycore_ prefix to internal header files (GH-10263)Victor Stinner2018-10-311-1/+1
| | | | | | | | | | | | | | | | | | | | * Rename Include/internal/ header files: * pyatomic.h -> pycore_atomic.h * ceval.h -> pycore_ceval.h * condvar.h -> pycore_condvar.h * context.h -> pycore_context.h * pygetopt.h -> pycore_getopt.h * gil.h -> pycore_gil.h * hamt.h -> pycore_hamt.h * hash.h -> pycore_hash.h * mem.h -> pycore_mem.h * pystate.h -> pycore_state.h * warnings.h -> pycore_warnings.h * PCbuild project, Makefile.pre.in, Modules/Setup: add the Include/internal/ directory to the search paths of header files. * Update includes. For example, replace #include "internal/mem.h" with #include "pycore_mem.h".
* bpo-35081: Move Py_BUILD_CORE code to internal/mem.h (GH-10249)Victor Stinner2018-10-311-0/+1
| | | | | * Add #include "internal/mem.h" to C files using _PyMem_SetDefaultAllocator(). * Include/internal/mem.h now requires Py_BUILD_CORE to be defined.
* bpo-35053: Enhance tracemalloc to trace free lists (GH-10063)Victor Stinner2018-10-251-0/+6
| | | | | | | | | | | | tracemalloc now tries to update the traceback when an object is reused from a "free list" (optimization for faster object creation, used by the builtin list type for example). Changes: * Add _PyTraceMalloc_NewReference() function which tries to update the Python traceback of a Python object. * _Py_NewReference() now calls _PyTraceMalloc_NewReference(). * Add an unit test.
* bpo-9263: _PyObject_Dump() detects freed memory (GH-10061)Victor Stinner2018-10-231-0/+16
| | | | | | | | | _PyObject_Dump() now uses an heuristic to check if the object memory has been freed: log "<freed object>" in that case. The heuristic rely on the debug hooks on Python memory allocators which fills the memory with DEADBYTE (0xDB) when memory is deallocated. Use PYTHONMALLOC=debug to always enable these debug hooks.
* remove vestigal locking from obmalloc (GH-5805)Benjamin Peterson2018-02-241-39/+0
| | | obmalloc has (empty) macros for locking in the allocator. These aren't needed in CPython; we rely on the GIL.
* bpo-32746: Fix multiple typos (GH-5144)Leo Arias2018-02-041-1/+1
| | | Fix typos found by codespell in docs, docstrings, and comments.
* bpo-32241: Add the const qualifire to declarations of umodifiable strings. ↵Serhiy Storchaka2017-12-121-1/+1
| | | | (#4748)
* bpo-32030: Add pymain_get_global_config() (#4735)Victor Stinner2017-12-061-2/+10
| | | | | | | | | | | | | * Py_Main() now starts by reading Py_xxx configuration variables to only work on its own private structure, and then later writes back the configuration into these variables. * Replace Py_GETENV() with pymain_get_env_var() which ignores empty variables. * Add _PyCoreConfig.dump_refs * Add _PyCoreConfig.malloc_stats * _PyObject_DebugMallocStats() is now responsible to check if debug hooks are installed. The function returns 1 if stats were written, or 0 if the hooks are disabled. Mark _PyMem_PymallocEnabled() as static.
* bpo-32030: Cleanup "path config" code (#4663)Victor Stinner2017-12-011-12/+12
| | | | | | | | | | * Rename PyPathConfig structure to _PyPathConfig and move it to Include/internal/pystate.h * Rename path_config to _Py_path_config * _PyPathConfig: Rename program_name field to program_full_path * Add assert(str != NULL); to _PyMem_RawWcsdup(), _PyMem_RawStrdup() and _PyMem_Strdup(). * Rename calculate_path() to pathconfig_global_init(). The function now does nothing if it's already initiallized.
* bpo-32030: Rework memory allocators (#4625)Victor Stinner2017-11-291-93/+199
| | | | | | | | | | | | | | | | | | | | * Fix _PyMem_SetupAllocators("debug"): always restore allocators to the defaults, rather than only caling _PyMem_SetupDebugHooks(). * Add _PyMem_SetDefaultAllocator() helper to set the "default" allocator. * Add _PyMem_GetAllocatorsName(): get the name of the allocators * main() now uses debug hooks on memory allocators if Py_DEBUG is defined, rather than calling directly malloc() * Document default memory allocators in C API documentation * _Py_InitializeCore() now fails with a fatal user error if PYTHONMALLOC value is an unknown memory allocator, instead of failing with a fatal internal error. * Add new tests on the PYTHONMALLOC environment variable * Add support.with_pymalloc() * Add the _testcapi.WITH_PYMALLOC constant and expose it as support.with_pymalloc(). * sysconfig.get_config_var('WITH_PYMALLOC') doesn't work on Windows, so replace it with support.with_pymalloc(). * pythoninfo: add _testcapi collector for pymem
* bpo-32030: Add _PyMainInterpreterConfig_ReadEnv() (#4542)Victor Stinner2017-11-241-0/+18
| | | | | | | | | | | | | | | | | | Py_GetPath() and Py_Main() now call _PyMainInterpreterConfig_ReadEnv() to share the same code to get environment variables. Changes: * Add _PyMainInterpreterConfig_ReadEnv() * Add _PyMainInterpreterConfig_Clear() * Add _PyMem_RawWcsdup() * _PyMainInterpreterConfig: rename pythonhome to home * Rename _Py_ReadMainInterpreterConfig() to _PyMainInterpreterConfig_Read() * Use _Py_INIT_USER_ERR(), instead of _Py_INIT_ERR(), for decoding errors: the user is able to fix the issue, it's not a bug in Python. Same change was made in _Py_INIT_NO_MEMORY(). * Remove _Py_GetPythonHomeWithConfig()
* bpo-32096: Remove obj and mem from _PyRuntime (#4532)Victor Stinner2017-11-241-186/+647
| | | | | | | | | | | | | | | | | bpo-32096, bpo-30860: Partially revert the commit 2ebc5ce42a8a9e047e790aefbf9a94811569b2b6: * Move structures back from Include/internal/mem.h to Objects/obmalloc.c * Remove _PyObject_Initialize() and _PyMem_Initialize() * Remove Include/internal/pymalloc.h * Add test_capi.test_pre_initialization_api(): Make sure that it's possible to call Py_DecodeLocale(), and then call Py_SetProgramName() with the decoded string, before Py_Initialize(). PyMem_RawMalloc() and Py_DecodeLocale() can be called again before _PyRuntimeState_Init(). Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
* bpo-32043: New "developer mode": "-X dev" option (#4413)Victor Stinner2017-11-161-10/+9
| | | | | | | | | | | | | Add a new "developer mode": new "-X dev" command line option to enable debug checks at runtime. Changes: * Add unit tests for -X dev * test_cmd_line: replace test.support with support. * Fix _PyRuntimeState_Fini(): Use the same memory allocator than _PyRuntimeState_Init(). * Fix _PyMem_GetDefaultRawAllocator()
* bpo-32030: Split Py_Main() into subfunctions (#4399)Victor Stinner2017-11-151-36/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Don't use "Python runtime" anymore to parse command line options or to get environment variables: pymain_init() is now a strict separation. * Use an error message rather than "crashing" directly with Py_FatalError(). Limit the number of calls to Py_FatalError(). It prepares the code to handle errors more nicely later. * Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now only added to the sys module once Python core is properly initialized. * _PyMain is now the well identified owner of some important strings like: warnings options, XOptions, and the "program name". The program name string is now properly freed at exit. pymain_free() is now responsible to free the "command" string. * Rename most methods in Modules/main.c to use a "pymain_" prefix to avoid conflits and ease debug. * Replace _Py_CommandLineDetails_INIT with memset(0) * Reorder a lot of code to fix the initialization ordering. For example, initializing standard streams now comes before parsing PYTHONWARNINGS. * Py_Main() now handles errors when adding warnings options and XOptions. * Add _PyMem_GetDefaultRawAllocator() private function. * Cleanup _PyMem_Initialize(): remove useless global constants: move them into _PyMem_Initialize(). * Call _PyRuntime_Initialize() as soon as possible: _PyRuntime_Initialize() now returns an error message on failure. * Add _PyInitError structure and following macros: * _Py_INIT_OK() * _Py_INIT_ERR(msg) * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case * _Py_INIT_FAILED(err)
* Add the const qualifier to "char *" variables that refer to literal strings. ↵Serhiy Storchaka2017-11-111-2/+2
| | | | (#4370)
generated by cgit v0.12 at 2024-12-04 11:40:24 (GMT)