summaryrefslogtreecommitdiffstats
path: root/jemalloc/src/jemalloc.c
Commit message (Collapse)AuthorAgeFilesLines
* Fix error detection for ipalloc() when profiling.Jason Evans2011-03-231-26/+35
| | | | | | | | | sa2u() returns 0 on overflow, but the profiling code was blindly calling sa2u() and allowing the error to silently propagate, ultimately ending in a later assertion failure. Refactor all ipalloc() callers to call sa2u(), check for overflow before calling ipalloc(), and pass usize rather than size. This allows ipalloc() to avoid calling sa2u() in the common case.
* Fix rallocm() rsize bug.Jason Evans2011-03-231-0/+2
| | | | Add code to set *rsize even when profiling is enabled.
* Fix bootstrapping order bug.Jason Evans2011-03-231-5/+5
| | | | | Initialize arenas_tsd earlier, so that the non-TLS case works when profiling is enabled.
* Use OSSpinLock*() for locking on OS X.Jason Evans2011-03-191-1/+7
| | | | | | pthread_mutex_lock() can call malloc() on OS X (!!!), which causes deadlock. Work around this by using spinlocks that are built of more primitive stuff.
* Add the "stats.cactive" mallctl.Jason Evans2011-03-191-1/+1
| | | | | | Add the "stats.cactive" mallctl, which can be used to efficiently and repeatedly query approximately how much active memory the application is utilizing.
* Improve thread-->arena assignment.Jason Evans2011-03-181-13/+58
| | | | | | | | Rather than blindly assigning threads to arenas in round-robin fashion, choose the lowest-numbered arena that currently has the smallest number of threads assigned to it. Add the "stats.arenas.<i>.nthreads" mallctl.
* Use bitmaps to track small regions.Jason Evans2011-03-171-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
* Add missing error checks.Jason Evans2011-03-151-1/+2
| | | | | Add missing error checks for pthread_mutex_init() calls. In practice, mutex initialization never fails, so this is merely good hygiene.
* Fix "thread.{de,}allocatedp" mallctl.Jason Evans2011-02-141-0/+22
| | | | | | | | | For the non-TLS case (as on OS X), if the "thread.{de,}allocatedp" mallctl was called before any allocation occurred for that thread, the TSD was still NULL, thus putting the application at risk of dereferencing NULL. Fix this by refactoring the initialization code, and making it part of the conditional logic for all per thread allocation counter accesses.
* Update various comments.Jason Evans2010-12-181-2/+2
|
* Add missing #ifdef JEMALLOC_PROF.Jason Evans2010-10-241-0/+2
| | | | Only call prof_boot0() if profiling is enabled.
* Replace JEMALLOC_OPTIONS with MALLOC_CONF.Jason Evans2010-10-241-317/+337
| | | | | | | | | | | Replace the single-character run-time flags with key/value pairs, which can be set via the malloc_conf global, /etc/malloc.conf, and the MALLOC_CONF environment variable. Replace the JEMALLOC_PROF_PREFIX environment variable with the "opt.prof_prefix" option. Replace umax2s() with u2s().
* Fix heap profiling bugs.Jason Evans2010-10-221-8/+29
| | | | | | | | | | | | | Fix a regression due to the recent heap profiling accuracy improvements: prof_{m,re}alloc() must set the object's profiling context regardless of whether it is sampled. Fix management of the CHUNK_MAP_CLASS chunk map bits, such that all large object (re-)allocation paths correctly initialize the bits. Prior to this fix, in-place realloc() cleared the bits, resulting in incorrect reported object size from arena_salloc_demote(). After this fix the non-demoted bit pattern is all zeros (instead of all ones), which makes it easier to assure that the bits are properly set.
* Add per thread allocation counters, and enhance heap sampling.Jason Evans2010-10-211-44/+206
| | | | | | | | | | | | | | | | | | | Add the "thread.allocated" and "thread.deallocated" mallctls, which can be used to query the total number of bytes ever allocated/deallocated by the calling thread. Add s2u() and sa2u(), which can be used to compute the usable size that will result from an allocation request of a particular size/alignment. Re-factor ipalloc() to use sa2u(). Enhance the heap profiler to trigger samples based on usable size, rather than request size. This has a subtle, but important, impact on the accuracy of heap sampling. For example, previous to this change, 16- and 17-byte objects were sampled at nearly the same rate, but 17-byte objects actually consume 32 bytes each. Therefore it was possible for the sample to be somewhat skewed compared to actual memory usage of the allocated objects.
* Make cumulative heap profile data optional.Jason Evans2010-10-031-0/+15
| | | | | | | | Add the R option to control whether cumulative heap profile data are maintained. Add the T option to control the size of per thread backtrace caches, primarily because when the R option is specified, backtraces that no longer have allocations associated with them are discarded as soon as no thread caches refer to them.
* Remove malloc_swap_enable().Jason Evans2010-10-021-17/+0
| | | | | | | Remove malloc_swap_enable(), which was obsoleted by the "swap.fds" mallctl. The prototype for malloc_swap_enable() was removed from jemalloc/jemalloc.h, but the function itself was accidentally left in place.
* Use offsetof() when sizing dynamic structures.Jason Evans2010-10-021-2/+2
| | | | | | Base dynamic structure size on offsetof(), rather than subtracting the size of the dynamic structure member. Results could differ on systems with strict data structure alignment requirements.
* Fix compiler warnings.Jason Evans2010-09-211-12/+54
| | | | | | Add --enable-cc-silence, which can be used to silence harmless warnings. Fix an aliasing bug in ckh_pointer_hash().
* Add memalign() and valloc() overrides.Jason Evans2010-09-201-0/+43
| | | | | If memalign() and/or valloc() are present on the system, override them in order to avoid mixed allocator usage.
* Wrap strerror_r().Jason Evans2010-09-201-0/+19
| | | | | Create the buferror() function, which wraps strerror_r(). This is necessary because glibc provides a non-standard strerror_r().
* Remove bad assertions in malloc_{pre,post}fork().Jason Evans2010-09-201-7/+1
| | | | | | | Remove assertions that malloc_{pre,post}fork() are only called if threading is enabled. This was true of these functions in the context of FreeBSD's libc, but now the functions are called unconditionally as a result of registering them with pthread_atfork().
* Add {,r,s,d}allocm().Jason Evans2010-09-171-8/+191
| | | | | | Add allocm(), rallocm(), sallocm(), and dallocm(), which are a functional superset of malloc(), calloc(), posix_memalign(), malloc_usable_size(), and free().
* Move size class table to man page.Jason Evans2010-09-121-82/+0
| | | | | | | Move the table of size classes from jemalloc.c to the manual page. When manually formatting the manual page, it is now necessary to use: nroff -man -t jemalloc.3
* Port to Mac OS X.Jason Evans2010-09-121-41/+47
| | | | | Add Mac OS X support, based in large part on the OS X support in Mozilla's version of jemalloc.
* Add a missing mutex unlock in malloc_init_hard().Jason Evans2010-07-221-0/+1
| | | | | | | | | | If multiple threads race to initialize malloc, the loser(s) busy-wait until initialization is complete. Add a missing mutex lock so that the loser(s) properly release the initialization mutex. Under some race conditions, this flaw could have caused one or more threads to become permanently blocked. Reported by Terrell Magee.
* Fix next_arena initialization.Jason Evans2010-05-111-1/+1
| | | | | | If there is more than one arena, initialize next_arena so that the first and second threads to allocate memory use arenas 0 and 1, rather than both using arena 0.
* Fix threads-related profiling bugs.Jason Evans2010-04-141-6/+7
| | | | | | | | Initialize bt2cnt_tsd so that cleanup at thread exit actually happens. Associate (prof_ctx_t *) with allocated objects, rather than (prof_thr_cnt_t *). Each thread must always operate on its own (prof_thr_cnt_t *), and an object may outlive the thread that allocated it.
* Add sampling activation/deactivation control.Jason Evans2010-04-011-0/+6
| | | | | | | Add the E/e options to control whether the application starts with sampling active/inactive (secondary control to F/f). Add the prof.active mallctl so that the application can activate/deactivate sampling on the fly.
* Make interval-triggered profile dumping optional.Jason Evans2010-04-011-1/+1
| | | | | | Make it possible to disable interval-triggered profile dumping, even if profiling is enabled. This is useful if the user only wants a single dump at exit, or if the application manually triggers profile dumps.
* Reduce statistical heap sampling memory overhead.Jason Evans2010-03-311-21/+68
| | | | | | | | | If the mean heap sampling interval is larger than one page, simulate sampled small objects with large objects. This allows profiling context pointers to be omitted for small objects. As a result, the memory overhead for sampling decreases as the sampling interval is increased. Fix a compilation error in the profiling code.
* Remove medium size classes.Jason Evans2010-03-171-51/+19
| | | | | | | | | | Remove medium size classes, because concurrent dirty page purging is no longer capable of purging inactive dirty pages inside active runs (due to recent arena/bin locking changes). Enhance tcache to support caching large objects, so that the same range of size classes is still cached, despite the removal of medium size class support.
* Simplify tcache object caching.Jason Evans2010-03-141-6/+3
| | | | | | | | | | | | | | | | | | | | Use chains of cached objects, rather than using arrays of pointers. Since tcache_bin_t is no longer dynamically sized, convert tcache_t's tbin to an array of structures, rather than an array of pointers. This implicitly removes tcache_bin_{create,destroy}(), which further simplifies the fast path for malloc/free. Use cacheline alignment for tcache_t allocations. Remove runtime configuration option for number of tcache bin slots, and replace it with a boolean option for enabling/disabling tcache. Limit the number of tcache objects to the lesser of TCACHE_NSLOTS_MAX and 2X the number of regions per run for the size class. For GC-triggered flush, discard 3/4 of the objects below the low water mark, rather than 1/2.
* Simplify malloc_message().Jason Evans2010-03-041-48/+33
| | | | | Rather than passing four strings to malloc_message(), malloc_write4(), and all the functions that use them, only pass one string.
* Implement sampling for heap profiling.Jason Evans2010-03-021-5/+14
|
* Restructure source tree.Jason Evans2010-02-111-1/+1
|
* Remove tracing (--enable-trace).Jason Evans2010-02-111-77/+9
| | | | | | | Remove all functionality related to tracing. This functionality was useful for understanding memory fragmentation during early algorithmic design of jemalloc, but it had little utility for non-trivial applications, due to the sheer volume of data written to disk.
* Implement interval-based heap profile dumping.Jason Evans2010-02-111-7/+3
| | | | | | Add mallctl interfaces for profiling parameters. Fix a file descriptor leak in heap profile dumping.
* Fix a profiling bootstrap bug.Jason Evans2010-02-111-8/+12
| | | | | Bootstrap profiling in three stages, so that it is usable by the time the first application allocation occurs.
* Implement allocation profiling and leack checking.Jason Evans2010-02-101-125/+188
| | | | | | | | | | | | | | Add the --enable-prof and --enable-prof-libunwind configure options. Add the B/b, F/f, I/i, L/l, and U/u JEMALLOC_OPTIONS. Interval-based profile dump triggering is not yet implemented. Add supporting generic code: * Add memory barriers. * Add prn (LCG PRNG). * Add hash (Murmur hash function). * Add ckh (cuckoo hash tables).
* Fix bootstrapping crash.Jason Evans2010-01-291-10/+20
| | | | | | | | | | | | | If a custom small_size2bin table was required due to non-default size class settings, memory allocation prior to initializing chunk parameters would cause a crash due to division by 0. The fix re-orders the various *_boot() function calls. Bootstrapping is simpler now than it was before the base allocator started just using the chunk allocator directly. This allows arena_boot[01]() to be combined. Add error detection for pthread_atfork() and atexit() function calls.
* Remove tcache bin sorting during flush.Jason Evans2010-01-291-8/+0
| | | | | This feature caused significant performance degradation, and the fragmentation reduction benefits were difficult to quantify.
* madvise(..., MADV_{RANODOM,NOSYNC}) swap files.Jason Evans2010-01-271-0/+9
| | | | Initialize malloc before calling into the ctl_*() functions.
* Implement mallctl{nametomib,bymib}().Jason Evans2010-01-271-21/+40
| | | | | | | | | | | | | Replace chunk stats code that was missing locking; this fixes a race condition that could corrupt chunk statistics. Converting malloc_stats_print() to use mallctl*(). Add a missing semicolon in th DSS code. Convert malloc_tcache_flush() to a mallctl. Convert malloc_swap_enable() to a set of mallctl's.
* Simplify malloc_{pre,post}fork().Jason Evans2010-01-251-37/+9
| | | | | | Revert to simpler lock acquistion/release code in malloc_{pre,post}fork(), since dynamic arena rebalancing is no longer implemented.
* Add the --enable-swap configure option.Jason Evans2010-01-241-0/+42
| | | | | | | | Add malloc_swap_enable(). Add the O/o JEMALLOC_OPTIONS flags, which control memory overcommit. Fix mapped memory stats reporting for arenas.
* Add the w4opaque argument to numerous functions.Jason Evans2010-01-191-5/+6
| | | | | Add the w4opaque argument malloc_message() and malloc_stats_print(), and propagate the change through all the internal APIs as necessary.
* Add the write4 parameter to malloc_stats_print().Jason Evans2010-01-171-1/+1
| | | | Add malloc_cprintf() and malloc_vcprintf().
* Add the --with-install-suffix configure option.Jason Evans2010-01-171-1/+1
| | | | Fix some bugs in the Makefile's install target.
* Refactor jemalloc.c into multiple source files.Jason Evans2010-01-161-5464/+100
| | | | | | | | | Fix a stats bug in large object curruns accounting. Replace tcache_bin_fill() with arena_tcache_fill(), and fix a bug in an OOM error path. Fix API name mangling to coexist with __attribute__((malloc)).
* Support malloc() even after tcache destruction has occurred, since other tsdJason Evans2010-01-131-12/+11
| | | | destructors may run after tcache_tsd's.
generated by cgit v0.12 at 2026-01-03 07:34:52 (GMT)