summaryrefslogtreecommitdiffstats
path: root/Objects/obmalloc.c
Commit message (Collapse)AuthorAgeFilesLines
* Oops, this is supposed to be disabled by default.Neal Norwitz2006-02-161-1/+1
|
* Merge ssize_t branch.Martin v. Löwis2006-02-151-4/+7
|
* Do a better job of not inlining Py_ADDRESS_IN_RANGE() for newer gcc's.Neal Norwitz2005-11-131-2/+9
| | | | Perhaps Py_NO_INLINE should be moved to pyport.h or some other header?
* SF bug 1185883: PyObject_Realloc can't safely take over a block currentlyTim Peters2005-07-101-27/+23
| | | | | | | | | | managed by C, because it's possible for the block to be smaller than the new requested size, and at the end of allocated VM. Trying to copy over nbytes bytes to a Python small-object block can segfault then, and there's no portable way to avoid this (we would have to know how many bytes starting at p are addressable, and std C has no means to determine that). Bugfix candidate. Should be backported to 2.4, but I'm out of time.
* whoops, I wanted that commented out by default, will add doc to MiscNeal Norwitz2004-06-061-1/+1
|
* SF bug 881641, make it easier to use valgrindNeal Norwitz2004-06-061-4/+39
|
* Whitespace normalization.Walter Dörwald2003-06-171-6/+6
|
* Remove MALLOC_ZERO_RETURNS_NULL.Martin v. Löwis2002-11-231-2/+0
|
* Patch #627105: Document that SYSTEM_PAGE_SIZE really should not beMartin v. Löwis2002-10-261-1/+4
| | | | larger than the system page size.
* Fix warnings on 64-bit platforms about casts from pointers to ints.Guido van Rossum2002-09-121-1/+1
| | | | Two of these were real bugs.
* Remove extraneous semicolon.Jeremy Hylton2002-07-181-1/+1
| | | | (Silences compiler warning for Compaq C++ 6.5 on Tru64.)
* Documented PYMALLOC_DEBUG. This completes primary coverage of all theTim Peters2002-07-101-1/+1
| | | | | "special builds" I ever use. If you use others, document them here, or don't be surprised if I rip out the code for them <0.5 wink>.
* PyObject_Realloc(): If a small block is shrinking, bite the expense ofTim Peters2002-05-021-5/+16
| | | | copying it if at least 25% of the input block can be reclaimed.
* _PyObject_DebugCheckAddress(): If the leading pad bytes are corrupt,Tim Peters2002-04-281-18/+25
| | | | | display a msg warning that the count of bytes requested may be bogus, and that a segfault may happen next.
* _PyObject_DebugMallocStats(): Added some potentially expensive internalTim Peters2002-04-181-5/+41
| | | | | | consistency checks, enabled only in a debug (Py_DEBUG) build. Note that this never gets called automatically unless PYMALLOC_DEBUG is #define'd too, and the envar PYTHONMALLOCSTATS exists.
* PyObject_Malloc: make a tiny bit faster for platforms where malloc(0)Tim Peters2002-04-181-2/+9
| | | | | | doesn't return NULL. PyObject_Realloc: better comment for why we don't call PyObject_Malloc(0).
* Remove some long-disabled debugging boilerplate.Tim Peters2002-04-181-26/+0
|
* _PyObject_DebugDumpStats: renamed to _PyObject_DebugMallocStats.Tim Peters2002-04-131-4/+8
| | | | | | Added code to call this when PYMALLOC_DEBUG is enabled, and envar PYTHONMALLOCSTATS is set, whenever a new arena is obtained and once late in the Python shutdown process.
* Small anal correctness tweaks:Tim Peters2002-04-121-2/+2
| | | | | | | | | | _PyObject_DebugMalloc: explicitly cast PyObject_Malloc's result to the target pointer type. _PyObject_DebugDumpStats: change decl of arena_alignment from unsigned int to unsigned long. This is for the 2.3 release only (it's new code).
* _PyObject_DebugRealloc(): rewritten to let the underlying realloc doTim Peters2002-04-121-27/+30
| | | | | | | | | | most of the work. In particular, if the underlying realloc is able to grow the memory block in place, great (this routine used to do a fresh malloc + memcpy every time a block grew). BTW, I'm not so keen here on avoiding possible quadratic-time realloc patterns as I am on making the debug pymalloc more invisible (the more it uses memory "just like" the underlying allocator, the better the chance that a suspected memory corruption bug won't vanish when the debug malloc is turned on).
* _PyObject_DebugDumpAddress(): clarify an output message.Tim Peters2002-04-121-1/+1
|
* PYMALLOC_{CLEAN, DEAD, FORBIDDEN}BYTE symbols: remove the PYMALLOC_Tim Peters2002-04-121-34/+39
| | | | | prefix. These symbols are private to the file, and the PYMALLOC_ gets in the way (overly long code lines, comments, and error messages).
* Move PyObject_Malloc and PyObject_Free here from object.c. RemoveNeil Schemenauer2002-04-121-61/+33
| | | | | PyMalloc_ prefix and use PyObject_ instead. I'm not sure about the debugging functions. Perhaps they should stay as PyMalloc_.
* SF bug 542181: Realloc behaviorTim Peters2002-04-111-12/+16
| | | | | | | | | | | The bug report pointed out a bogosity in the comment block explaining thread safety for arena management. Repaired that comment, repaired a couple others while I was at it, and added an assert. _PyMalloc_DebugRealloc: If this needed to get more memory, but couldn't, it erroneously freed the original memory. Repaired that. This is for 2.3 only (unless we decide to backport the new pymalloc).
* Minor improvements to the stats output dump, including adding commas toTim Peters2002-04-061-10/+34
| | | | the big numbers.
* Repair an incomprehensible comment.Tim Peters2002-04-051-2/+2
|
* _PyMalloc_DebugDumpStats(): vastly improved the output, and it nowTim Peters2002-04-051-21/+58
| | | | accounts for every byte.
* Widespread, but mostly in _PyMalloc_Malloc: optimize away all expensiveTim Peters2002-04-051-40/+39
| | | | | | | | runtime multiplications and divisions, via the scheme developed with Vladimir Marangozov on Python-Dev. The pool_header struct loses its capacity member, but gains nextoffset and maxnextoffset members; this still leaves it at 32 bytes on a 32-bit box (it has to be padded to a multiple of 8 bytes).
* _PyMalloc_Realloc(): removed a now-pointless cast.Tim Peters2002-04-041-1/+1
|
* _PyMalloc_{Malloc, Realloc}: Strive to meet the doc's promises aboutTim Peters2002-04-041-42/+44
| | | | | | | | | | | | | | | | | | | what these do given a 0 size argument. This is so that when pymalloc is enabled, we don't need to wrap pymalloc calls in goofy little routines special-casing 0. Note that it's virtually impossible to meet the doc's promise that malloc(0) will never return NULL; this makes a best effort, but not an insane effort. The code does promise that realloc(not-NULL, 0) will never return NULL (malloc(0) is much harder). _PyMalloc_Realloc: Changed to take over all requests for 0 bytes, and rearranged to be a little quicker in expected cases. All over the place: when resorting to the platform allocator, call free/malloc/realloc directly, without indirecting thru macros. This should avoid needing a nightmarish pile of #ifdef-ery if PYMALLOC_DEBUG is changed so that pymalloc takes over all Py(Mem, Object} memory operations (which would add useful debugging info to PyMem_xyz allocations too).
* Fixed errors in two comments.Tim Peters2002-04-011-3/+4
|
* Restructured my pool-management overview in terms of the threeTim Peters2002-04-011-21/+61
| | | | | | | | | | | | | possible pool states. I think it's much clearer now. Added a new long overdue block-management overview comment block. I believe the comments are in good shape now. Added two comments about possible small optimizations (one getting rid of runtime multiplications at the cost of a new pool_header member; the other getting rid of runtime divisions and the pool_header capacity member, at the cost of a static const vector of 32 uints).
* New PYMALLOC_DEBUG function void _PyMalloc_DebugDumpStats(void).Tim Peters2002-04-011-1/+97
| | | | | | | | | | | | | This displays stats about the # of arenas, pools, blocks and bytes, to stderr, both used and reserved but unused. CAUTION: Because PYMALLOC_DEBUG is on, the debug malloc routine adds 16 bytes to each request. This makes each block appear two size classes higher than it would be if PYMALLOC_DEBUG weren't on. So far, playing with this confirms the obvious: there's a lot of activity in the "small dict" size class, but nothing in the core makes any use of the 8-byte or 16-byte classes.
* Add one more assert that indirectly interlocking conditions are consistentTim Peters2002-03-311-0/+1
| | | | with each other.
* Fixed an error in a new assert.Tim Peters2002-03-311-1/+1
|
* Fixed a typo in a new comment.Tim Peters2002-03-311-1/+1
|
* _PyMalloc_Free(): As was already done for _PyMalloc_Malloc, rearrangedTim Peters2002-03-311-37/+47
| | | | | | | | | | the code so that the most frequent cases come first. Added comments. Found a hidden assumption that a pool contains room for at least two blocks, and added an assert to catch a violation if it ever happens in a place where that matters. Gave the normal "I allocated this block" case a longer basic block to work with before it has to do its first branch (via breaking apart an embedded assignment in an "if", and hoisting common code out of both branches).
* Added a long-overdue comment block giving an overview of pool operationsTim Peters2002-03-311-3/+60
| | | | and terminology, plus explanation of some extreme obscurities.
* It's once again thought safe to call the pymalloc free/realloc with anTim Peters2002-03-301-14/+27
| | | | | | | | | | | | | | | | | | | address obtained from system malloc/realloc without holding the GIL. When the vector of arena base addresses has to grow, the old vector is deliberately leaked. This makes "stale" x-thread references safe. arenas and narenas are also declared volatile, and changed in an order that prevents a thread from picking up a value of narenas too large for the value of arenas it sees. Added more asserts. Fixed an old inaccurate comment. Added a comment explaining why it's safe to call pymalloc free/realloc with an address obtained from system malloc/realloc even when arenas is still NULL (this is obscure, since the ADDRESS_IN_RANGE macro appears <wink> to index into arenas).
* new_arena(): In error cases, reset the number of available pools to 0.Tim Peters2002-03-301-0/+1
| | | | Else the pymalloc malloc will go insane the next time it's called.
* Changed the #-of-arenas counters to uints -- no need to be insane aboutTim Peters2002-03-301-10/+11
| | | | | | | this. But added an overflow check just in case there is. Got rid of the ushort macro. It wasn't used anymore (it was only used in the no-longer-exists off_t macro), and there's no plausible use for it.
* Turns out the off_t macro isn't used anymore, so got rid of it.Tim Peters2002-03-301-3/+0
|
* Now that we're no longer linking arenas together, there's no need toTim Peters2002-03-301-25/+21
| | | | | | | | | | | | | | | | | | | | | | waste the first pool if malloc happens to return a pool-aligned address. This means the number of pools per arena can now vary by 1. Unfortunately, the code counted up from 0 to a presumed constant number of pools. So changed the increasing "watermark" counter to a decreasing "nfreepools" counter instead, and fiddled various stuff accordingly. This also allowed getting rid of two more macros. Also changed the code to align the first address to a pool boundary instead of a page boundary. These are two parallel sets of macro #defines that happen to be identical now, but the page macros are in theory more restrictive (bigger), and there's simply no reason I can see that it wasn't aligning to the less restrictive pool size all along (the code only relies on pool alignment). Hmm. The "page size" macros aren't used for anything *except* defining the pool size macros, and the comments claim the latter isn't necessary. So this has the feel of a layer of indirection that doesn't serve a purpose; should probably get rid of the page macros now.
* Retract the claim that this is always safe if PyMem_{Del, DEL, Free, FREE}Tim Peters2002-03-301-0/+4
| | | | | | | are called without the GIL. It's incredibly unlikely to fail, but I can't make this bulletproof without either adding a lock for exclusion, or giving up on growing the arena base-address vector (it would be safe if this were a static array).
* Lots of changes:Tim Peters2002-03-301-127/+225
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | + A new scheme for determining whether an address belongs to a pymalloc arena. This should be 100% reliable. The poolp->pooladdr and poolp->magic members are gone. A new poolp->arenaindex member takes their place. Note that the pool header overhead doesn't actually shrink, though, since the header is padded to a multiple of 8 bytes. + _PyMalloc_Free and _PyMalloc_Realloc should now be safe to call for any legit address, whether obtained from a _PyMalloc function or from the system malloc/realloc. It should even be safe to call _PyMalloc_Free when *not* holding the GIL, provided that the passed-in address was obtained from system malloc/realloc. Since this is accomplished without any locks, you better believe the code is subtle. I hope it's sufficiently commented. + The above implies we don't need the new PyMalloc_{New, NewVar, Del} API anymore, and could switch back to PyObject_XXX without breaking existing code mixing PyObject_XXX with PyMem_{Del, DEL, Free, FREE}. Nothing is done here about that yet, and I'd like to see this new code exercised more first. + The small object threshhold is boosted to 256 (the max). We should play with that some more, but the old 64 was way too small for 2.3. + Getting a new arena is now done via new function new_arena(). + Removed some unused macros, and squashed out some macros that were used only once to define other macros. + Arenas are no longer linked together. A new vector of arena base addresses had to be created anyway to make address classification bulletproof. + A lot of the patch size is an illusion: given the way address classification works now, it was more convenient to switch the sense of the prime "if" tests in the realloc and free functions, so the "if" and "else" blocks got swapped. + Assorted minor code, comment and whitespace cleanup. Back to the Windows installer <wink>.
* Add missing "void" to function.Neil Schemenauer2002-03-281-1/+1
|
* PYMALLOC_DEBUG routines: The "check API family" gimmick was going nowhereTim Peters2002-03-281-62/+34
| | | | | fast, and just cluttered the code. Get rid of it for now. If a compelling case can be made for it, easy to restore it later.
* _PyMalloc_DebugRealloc(): simplify decl of "fresh".Tim Peters2002-03-241-5/+15
| | | | | | Assorted: bump the serial number via a trivial new bumpserialno() function. The point is to give a single place to set a breakpoint when waiting for a specific serial number.
* Minor code cleanup -- no semantic changes.Tim Peters2002-03-231-22/+28
|
* Give Python a debug-mode pymalloc, much as sketched on Python-Dev.Tim Peters2002-03-231-17/+315
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug allocator. This can be done independent of build type (release or debug). A debug build automatically defines PYMALLOC_DEBUG when pymalloc is enabled. It's a detected error to define PYMALLOC_DEBUG when pymalloc isn't enabled. Two debugging entry points defined only under PYMALLOC_DEBUG: + _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb) to sanity-check a memory block obtained from pymalloc. It sprays info to stderr (see next) and dies via Py_FatalError if the block is detectably damaged. + _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info about a debug memory block to stderr. A tiny start at implementing "API family" checks isn't good for anything yet. _PyMalloc_DebugRealloc() has been optimized to do little when the new size is <= old size. However, if the new size is larger, it really can't call the underlying realloc() routine without either violating its contract, or knowing something non-trivial about how the underlying realloc() works. A memcpy is always done in this case. This was a disaster for (and only) one of the std tests: test_bufio creates single text file lines up to a million characters long. On Windows, fileobject.c's get_line() uses the horridly funky getline_via_fgets(), which keeps growing and growing a string object hoping to find a newline. It grew the string object 1000 bytes each time, so for a million-character string it took approximately forever (I gave up after a few minutes). So, also: fileobject.c, getline_via_fgets(): When a single line is outrageously long, grow the string object at a mildly exponential rate, instead of just 1000 bytes at a time. That's enough so that a debug-build test_bufio finishes in about 5 seconds on my Win98SE box. I'm curious to try this on Win2K, because it has very different memory behavior than Win9X, and test_bufio always took a factor of 10 longer to complete on Win2K. It *could* be that the endless reallocs were simply killing it on Win2K even in the release build.
generated by cgit v0.12 at 2024-11-16 20:51:55 (GMT)