summaryrefslogtreecommitdiffstats
path: root/src/jemalloc.c
Commit message (Collapse)AuthorAgeFilesLines
* Fix a potential tsd cleanup leak.Jason Evans2016-02-281-0/+3
| | | | | | | | | | | | | | Prior to 767d85061a6fb88ec977bbcd9b429a43aff391e6 (Refactor arenas array (fixes deadlock).), it was possible under some circumstances for arena_get() to trigger recreation of the arenas cache during tsd cleanup, and the arenas cache would then be leaked. In principle a similar issue could still occur as a side effect of decay-based purging, which calls arena_tdata_get(). Fix arenas_tdata_cleanup() by setting tsd->arenas_tdata_bypass to true, so that arena_tdata_get() will gracefully fail (an expected behavior) rather than recreating tsd->arena_tdata. Reported by Christopher Ferris <cferris@google.com>.
* Add more HUGE_MAXCLASS overflow checks.Jason Evans2016-02-261-23/+34
| | | | | | | Add HUGE_MAXCLASS overflow checks that are specific to heap profiling code paths. This fixes test failures that were introduced by 0c516a00c4cb28cff55ce0995f756b5aae074c9e (Make *allocx() size class overflow behavior defined.).
* Make *allocx() size class overflow behavior defined.Jason Evans2016-02-251-24/+44
| | | | | | | Limit supported size and alignment to HUGE_MAXCLASS, which in turn is now limited to be less than PTRDIFF_MAX. This resolves #278 and #295.
* Refactor arenas array (fixes deadlock).Jason Evans2016-02-251-151/+90
| | | | | | | | | | | | Refactor the arenas array, which contains pointers to all extant arenas, such that it starts out as a sparse array of maximum size, and use double-checked atomics-based reads as the basis for fast and simple arena_get(). Additionally, reduce arenas_lock's role such that it only protects against arena initalization races. These changes remove the possibility for arena lookups to trigger locking, which resolves at least one known (fork-related) deadlock. This resolves #315.
* Silence miscellaneous 64-to-32-bit data loss warnings.Jason Evans2016-02-241-1/+1
|
* Use ssize_t for readlink() rather than int.Jason Evans2016-02-241-1/+1
|
* Make opt_narenas unsigned rather than size_t.Jason Evans2016-02-241-8/+12
|
* Refactor time_* into nstime_*.Jason Evans2016-02-221-1/+1
| | | | | | | Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
* Implement decay-based unused dirty page purging.Jason Evans2016-02-201-11/+42
| | | | | | | | | | | | | | | | This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
* Refactor arenas_cache tsd.Jason Evans2016-02-201-62/+87
| | | | | Refactor arenas_cache tsd into arenas_tdata, which is a structure of type arena_tdata_t.
* Add --with-malloc-conf.Jason Evans2016-02-201-3/+6
| | | | | Add --with-malloc-conf, which makes it possible to embed a default options string during configuration.
* Call malloc_test_boot0() from malloc_init_hard_recursible().Cosmin Paraschiv2016-01-111-5/+16
| | | | | | | | | | | When using LinuxThreads, malloc bootstrapping deadlocks, since malloc_tsd_boot0() ends up calling pthread_setspecific(), which causes recursive allocation. Fix it by moving the malloc_tsd_boot0() call to malloc_init_hard_recursible(). The deadlock was introduced by 8bb3198f72fc7587dc93527f9f19fb5be52fa553 (Refactor/fix arenas manipulation.), when tsd_boot() was split and the top half, tsd_boot0(), got an extra tsd_wrapper_set() call.
* Fast-path improvement: reduce # of branches and unnecessary operations.Qi Wang2015-11-101-53/+133
| | | | | | - Combine multiple runtime branches into a single malloc_slow check. - Avoid calling arena_choose / size2index / index2size on fast path. - A few micro optimizations.
* Add mallocx() OOM tests.Jason Evans2015-09-171-0/+2
|
* Simplify imallocx_prof_sample().Jason Evans2015-09-171-26/+13
| | | | | | | Simplify imallocx_prof_sample() to always operate on usize rather than sometimes using size. This avoids redundant usize computations and more closely fits the style adopted by i[rx]allocx_prof_sample() to fix sampling bugs.
* Fix irallocx_prof_sample().Jason Evans2015-09-171-5/+5
| | | | | Fix irallocx_prof_sample() to always allocate large regions, even when alignment is non-zero.
* Fix ixallocx_prof_sample().Jason Evans2015-09-171-17/+4
| | | | | | Fix ixallocx_prof_sample() to never modify nor create sampled small allocations. xallocx() is in general incapable of moving small allocations, so this fix removes buggy code without loss of generality.
* Centralize xallocx() size[+extra] overflow checks.Jason Evans2015-09-151-7/+11
|
* Fix ixallocx_prof() to check for size greater than HUGE_MAXCLASS.Jason Evans2015-09-151-1/+5
|
* Resolve an unsupported special case in arena_prof_tctx_set().Jason Evans2015-09-151-3/+3
| | | | | | | | | | | Add arena_prof_tctx_reset() and use it instead of arena_prof_tctx_set() when resetting the tctx pointer during reallocation, which happens whenever an originally sampled reallocated object is not sampled during reallocation. This regression was introduced by 594c759f37c301d0245dc2accf4d4aaf9d202819 (Optimize arena_prof_tctx_set().)
* Fix ixallocx_prof_sample() argument order reversal.Jason Evans2015-09-151-1/+1
| | | | | Fix ixallocx_prof() to pass usize_max and zero to ixallocx_prof_sample() in the correct order.
* s/max_usize/usize_max/gJason Evans2015-09-151-6/+6
|
* s/oldptr/old_ptr/gJason Evans2015-09-151-15/+15
|
* Make one call to prof_active_get_unlocked() per allocation event.Jason Evans2015-09-151-10/+19
| | | | | | | Make one call to prof_active_get_unlocked() per allocation event, and use the result throughout the relevant functions that handle an allocation event. Also add a missing check in prof_realloc(). These fixes protect allocation events against concurrent prof_active changes.
* Fix irealloc_prof() to prof_alloc_rollback() on OOM.Jason Evans2015-09-151-1/+3
|
* Optimize irallocx_prof() to optimistically update the sampler state.Jason Evans2015-09-151-3/+3
|
* Fix ixallocx_prof() size+extra overflow.Jason Evans2015-09-151-0/+3
| | | | | Fix ixallocx_prof() to clamp the extra parameter if size+extra would overflow HUGE_MAXCLASS.
* Force initialization of the init_lock in malloc_init_hard on Windows XPMike Hommey2015-09-041-1/+15
| | | | This resolves #269.
* Fix arenas_cache_cleanup() and arena_get_hard().Jason Evans2015-08-281-6/+5
| | | | | | | | | Fix arenas_cache_cleanup() and arena_get_hard() to handle allocation/deallocation within the application's thread-specific data cleanup functions even after arenas_cache is torn down. This is a more general fix that complements 45e9f66c280e1ba8bebf7bed387a43bc9e45536d (Fix arenas_cache_cleanup().).
* Fix arenas_cache_cleanup().Christopher Ferris2015-08-211-1/+5
| | | | | | Fix arenas_cache_cleanup() to handle allocation/deallocation within the application's thread-specific data cleanup functions even after arenas_cache is torn down.
* MSVC compatibility changesMatthijs2015-08-041-8/+16
| | | | | | - Decorate public function with __declspec(allocator) and __declspec(restrict), just like MSVC 1900 - Support JEMALLOC_HAS_RESTRICT by defining the restrict keyword - Move __declspec(nothrow) between 'void' and '*' so it compiles once more
* Move JEMALLOC_NOTHROW just after return type.Jason Evans2015-07-211-36/+27
| | | | | | Only use __declspec(nothrow) in C++ mode. This resolves #244.
* Remove JEMALLOC_ALLOC_SIZE annotations on functions not returning pointersMike Hommey2015-07-211-2/+2
| | | | | | | | As per gcc documentation: The alloc_size attribute is used to tell the compiler that the function return value points to memory (...) This resolves #245.
* Avoid function prototype incompatibilities.Jason Evans2015-07-101-20/+40
| | | | | | | | | Add various function attributes to the exported functions to give the compiler more information to work with during optimization, and also specify throw() when compiling with C++ on Linux, in order to adequately match what __THROW does in glibc. This resolves #237.
* Optimizations for WindowsMatthijs2015-06-251-1/+4
| | | | | | | - Set opt_lg_chunk based on run-time OS setting - Verify LG_PAGE is compatible with run-time OS setting - When targeting Windows Vista or newer, use SRWLOCK instead of CRITICAL_SECTION - When targeting Windows Vista or newer, statically initialize init_lock
* Fix size class overflow handling when profiling is enabled.Jason Evans2015-06-241-4/+12
| | | | | | | | | | Fix size class overflow handling for malloc(), posix_memalign(), memalign(), calloc(), and realloc() when profiling is enabled. Remove an assertion that erroneously caused arena_sdalloc() to fail when profiling was enabled. This resolves #232.
* Add alignment assertions to public aligned allocation functions.Jason Evans2015-06-231-28/+33
|
* Implement cache index randomization for large allocations.Jason Evans2015-05-061-1/+2
| | | | | | | | | | | | | | | | | | | | Extract szad size quantization into {extent,run}_quantize(), and . quantize szad run sizes to the union of valid small region run sizes and large run sizes. Refactor iteration in arena_run_first_fit() to use run_quantize{,_first,_next(), and add support for padded large runs. For large allocations that have no specified alignment constraints, compute a pseudo-random offset from the beginning of the first backing page that is a multiple of the cache line size. Under typical configurations with 4-KiB pages and 64-byte cache lines this results in a uniform distribution among 64 page boundary offsets. Add the --disable-cache-oblivious option, primarily intended for performance testing. This resolves #13.
* Concise JEMALLOC_HAVE_ISSETUGID case in secure_getenv().Igor Podlesny2015-04-301-11/+3
|
* Restore --enable-ivsalloc.Jason Evans2015-03-191-2/+2
| | | | | | | | | However, unlike before it was removed do not force --enable-ivsalloc when Darwin zone allocator integration is enabled, since the zone allocator code uses ivsalloc() regardless of whether malloc_usable_size() and sallocx() do. This resolves #211.
* Remove redundant tcache_boot() call.Jason Evans2015-02-151-2/+0
|
* Move centralized chunk management into arenas.Jason Evans2015-02-121-11/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Migrate all centralized data structures related to huge allocations and recyclable chunks into arena_t, so that each arena can manage huge allocations and recyclable virtual memory completely independently of other arenas. Add chunk node caching to arenas, in order to avoid contention on the base allocator. Use chunks_rtree to look up huge allocations rather than a red-black tree. Maintain a per arena unsorted list of huge allocations (which will be needed to enumerate huge allocations during arena reset). Remove the --enable-ivsalloc option, make ivsalloc() always available, and use it for size queries if --enable-debug is enabled. The only practical implications to this removal are that 1) ivsalloc() is now always available during live debugging (and the underlying radix tree is available during core-based debugging), and 2) size query validation can no longer be enabled independent of --enable-debug. Remove the stats.chunks.{current,total,high} mallctls, and replace their underlying statistics with simpler atomically updated counters used exclusively for gdump triggering. These statistics are no longer very useful because each arena manages chunks independently, and per arena statistics provide similar information. Simplify chunk synchronization code, now that base chunk allocation cannot cause recursive lock acquisition.
* Implement explicit tcache support.Jason Evans2015-02-101-82/+76
| | | | | | | | | Add the MALLOCX_TCACHE() and MALLOCX_TCACHE_NONE macros, which can be used in conjunction with the *allocx() API. Add the tcache.create, tcache.flush, and tcache.destroy mallctls. This resolves #145.
* Implement metadata statistics.Jason Evans2015-01-241-23/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | There are three categories of metadata: - Base allocations are used for bootstrap-sensitive internal allocator data structures. - Arena chunk headers comprise pages which track the states of the non-metadata pages. - Internal allocations differ from application-originated allocations in that they are for internal use, and that they are omitted from heap profiles. The metadata statistics comprise the metadata categories as follows: - stats.metadata: All metadata -- base + arena chunk headers + internal allocations. - stats.arenas.<i>.metadata.mapped: Arena chunk headers. - stats.arenas.<i>.metadata.allocated: Internal allocations. This is reported separately from the other metadata statistics because it overlaps with the allocated and active statistics, whereas the other metadata statistics do not. Base allocations are not reported separately, though their magnitude can be computed by subtracting the arena-specific metadata. This resolves #163.
* Refactor bootstrapping to delay tsd initialization.Jason Evans2015-01-221-108/+185
| | | | | | | | | | | | Refactor bootstrapping to delay tsd initialization, primarily to support integration with FreeBSD's libc. Refactor a0*() for internal-only use, and add the bootstrap_{malloc,calloc,free}() API for use by FreeBSD's libc. This separation limits use of the a0*() functions to metadata allocation, which doesn't require malloc/calloc/free API compatibility. This resolves #170.
* Fix arenas_cache_cleanup().Jason Evans2015-01-221-1/+1
| | | | | Fix arenas_cache_cleanup() to check whether arenas_cache is NULL before deallocation, rather than checking arenas.
* Fix OOM handling in memalign() and valloc().Jason Evans2015-01-171-2/+4
| | | | | | Fix memalign() and valloc() to heed imemalign()'s return value. Reported by Kurt Wampler.
* Introduce two new modes of junk filling: "alloc" and "free".Guilherme Goncalves2014-12-151-7/+47
| | | | | | | | In addition to true/false, opt.junk can now be either "alloc" or "free", giving applications the possibility of junking memory only on allocation or deallocation. This resolves #172.
* Ignore MALLOC_CONF in set{uid,gid,cap} binaries.Daniel Micay2014-12-141-1/+22
| | | | | | This eliminates the malloc tunables as tools for an attacker. Closes #173
* Style and spelling fixes.Jason Evans2014-12-091-1/+1
|
generated by cgit v0.12 at 2025-03-29 11:07:52 (GMT)