summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* Range-check mib[1] --> arena_ind casts.Jason Evans2017-01-071-7/+22
|
* Move static ctl_epoch variable into ctl_stats_t (as epoch).Jason Evans2017-01-071-4/+3
|
* Refactor ctl_stats_t.Jason Evans2017-01-071-58/+91
| | | | | | | Refactor ctl_stats_t to be a demand-zeroed non-growing data structure. To keep the size from being onerous (~60 MiB) on 32-bit systems, convert the arenas field to contain pointers rather than directly embedded ctl_arena_stats_t elements.
* Rename the arenas.extend mallctl to arenas.create.Jason Evans2017-01-072-4/+4
|
* Add MALLCTL_ARENAS_ALL.Jason Evans2017-01-072-77/+123
| | | | | | | Add the MALLCTL_ARENAS_ALL cpp macro as a fixed index for use in accessing the arena.<i>.{purge,decay,dss} and stats.arenas.<i>.* mallctls, and deprecate access via the arenas.narenas index (to be removed in 6.0.0).
* Fix locking in arena_dirty_count().Jason Evans2017-01-071-1/+3
| | | | This was a latent bug, since the function is (intentionally) not used.
* Fix allocated_large stats with respect to sampled small allocations.Jason Evans2017-01-071-6/+18
|
* Fix arena_large_reset_stats_cancel().Jason Evans2017-01-051-1/+1
| | | | | Decrement ndalloc_large rather than incrementing, in order to cancel out the increment in arena_large_dalloc_stats_update().
* Implement per arena base allocators.Jason Evans2016-12-279-202/+418
| | | | | | | | | | | | | Add/rename related mallctls: - Add stats.arenas.<i>.base . - Rename stats.arenas.<i>.metadata to stats.arenas.<i>.internal . - Add stats.arenas.<i>.resident . Modify the arenas.extend mallctl to take an optional (extent_hooks_t *) argument so that it is possible for all base allocations to be serviced by the specified extent hooks. This resolves #463.
* Refactor purging and splitting/merging.Jason Evans2016-12-274-48/+149
| | | | | | | | | | | | | | Split purging into lazy and forced variants. Use the forced variant for zeroing dss. Add support for NULL function pointers as an opt-out mechanism for the dalloc, commit, decommit, purge_lazy, purge_forced, split, and merge fields of extent_hooks_t. Add short-circuiting checks in large_ralloc_no_move_{shrink,expand}() so that no attempt is made if splitting/merging is not supported. This resolves #268.
* Rename arena_decay_t's ndirty to nunpurged.Jason Evans2016-12-271-4/+4
|
* Use exponential series to size extents.Jason Evans2016-12-272-31/+189
| | | | | | | | | | If virtual memory is retained, allocate extents such that their sizes form an exponentially growing series. This limits the number of disjoint virtual memory ranges so that extent merging can be effective even if multiple arenas' extent allocation requests are highly interleaved. This resolves #462.
* Add huge page configuration and pages_[no}huge().Jason Evans2016-12-271-1/+30
| | | | | | | | Add the --with-lg-hugepage configure option, but automatically configure LG_HUGEPAGE even if it isn't specified. Add the pages_[no]huge() functions, which toggle huge page state via madvise(..., MADV_[NO]HUGEPAGE) calls.
* Fix JSON-mode output for !config_stats and/or !config_prof cases.Jason Evans2016-12-231-10/+11
| | | | | | | These bugs were introduced by 0ba5b9b6189e16a983d8922d8c5cb6ab421906e8 (Add "J" (JSON) support to malloc_stats_print().), which was backported as b599b32280e1142856b0b96293a71e1684b1ccfb (with the same bugs except the inapplicable "metatata" misspelling) and first released in 4.3.0.
* Simplify arena_slab_regind().Jason Evans2016-12-231-59/+26
| | | | | | | | | Rewrite arena_slab_regind() to provide sufficient constant data for the compiler to perform division strength reduction. This replaces more general manual strength reduction that was implemented before arena_bin_info was compile-time-constant. It would be possible to slightly improve on the compiler-generated division code by taking advantage of range limits that the compiler doesn't know about.
* jemalloc cpp new/delete bindingsDave Watson2016-12-131-0/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds cpp bindings for jemalloc, along with necessary autoconf settings. This is mostly to add sized deallocation support, which can't be added from C directly. Sized deallocation is ~10% microbench improvement. * Import ax_cxx_compile_stdcxx.m4 from the autoconf repo, seems like the easiest way to get c++14 detection. * Adds various other changes, like CXXFLAGS, to configure.ac. * Adds new rules to Makefile.in for src/jemalloc-cpp.cpp, and a basic unittest. * Both new and delete are overridden, to ensure jemalloc is used for both. * TODO future enhancement of avoiding extra PLT thunks for new and delete - sdallocx and malloc are publicly exported jemalloc symbols, using an alias would link them directly. Unfortunately, was having trouble getting it to play nice with jemalloc's namespace support. Testing: Tested gcc 4.8, gcc 5, gcc 5.2, clang 4.0. Only gcc >= 5 has sized deallocation support, verified that the rest build correctly. Tested mac osx and Centos. Tested --with-jemalloc-prefix and --without-export. This resolves #202.
* Add --disable-syscall.Jason Evans2016-12-042-4/+4
| | | | This resolves #517.
* Add pthread_atfork(3) feature test.Jason Evans2016-11-171-2/+3
| | | | | | Some versions of Android provide a pthreads library without providing pthread_atfork(), so in practice a separate feature test is necessary for the latter.
* Refactor madvise(2) configuration.Jason Evans2016-11-171-5/+5
| | | | | | | | | Add feature tests for the MADV_FREE and MADV_DONTNEED flags to madvise(2), so that MADV_FREE is detected and used for Linux kernel versions 4.5 and newer. Refactor pages_purge() so that on systems which support both flags, MADV_FREE is preferred over MADV_DONTNEED. This resolves #387.
* Avoid gcc type-limits warnings.Jason Evans2016-11-171-12/+30
|
* Remove size_t -> unsigned -> size_t conversion.Maks Naumov2016-11-161-2/+1
|
* Uniformly cast mallctl[bymib]() oldp/newp arguments to (void *).Jason Evans2016-11-151-3/+4
| | | | | This avoids warnings in some cases, and is otherwise generally good hygiene.
* Add extent serial numbers.Jason Evans2016-11-154-28/+42
| | | | | | | | Add extent serial numbers and use them where appropriate as a sort key that is higher priority than address, so that the allocation policy prefers older extents. This resolves #147.
* Fix arena_reset() crashing bug.Jason Evans2016-11-151-41/+42
| | | | | This regression was caused by 498856f44a30b31fe713a18eb2fc7c6ecf3a9f63 (Move slabs out of chunks.).
* Rename atomic_*_{uint32,uint64,u}() to atomic_*_{u32,u64,zu}().Jason Evans2016-11-072-8/+8
| | | | This change conforms to naming conventions throughout the codebase.
* Refactor prng to not use 64-bit atomics on 32-bit platforms.Jason Evans2016-11-073-8/+8
| | | | This resolves #495.
* Fix/simplify extent_recycle() allocation size computations.Jason Evans2016-11-041-6/+5
| | | | | | | | | | | Do not call s2u() during alloc_size computation, since any necessary ceiling increase is taken care of later by extent_first_best_fit() --> extent_size_quantize_ceil(), and the s2u() call may erroneously cause a higher quantization result. Remove an overly strict overflow check that was added in 4a7852137d8b6598fdb90ea8e1fd3bc8a8b94a3a (Fix extent_recycle()'s cache-oblivious padding support.).
* Fix extent_recycle()'s cache-oblivious padding support.Jason Evans2016-11-041-5/+6
| | | | | | | Add padding *after* computing the size class, so that the optimal size class isn't skipped during search for a usable extent. This regression was caused by b46261d58b449cc4c099ed2384451a2499688f0e (Implement cache-oblivious support for huge size classes.).
* Fix psz/pind edge cases.Jason Evans2016-11-043-13/+13
| | | | | | | | | Add an "over-size" extent heap in which to store extents which exceed the maximum size class (plus cache-oblivious padding, if enabled). Remove psz2ind_clamp() and use psz2ind() instead so that trying to allocate the maximum size class can in principle succeed. In practice, this allows assertions to hold so that OOM errors can be successfully generated.
* Fix extent_alloc_cache[_locked]() to support decommitted allocation.Jason Evans2016-11-043-18/+17
| | | | | | | | | Fix extent_alloc_cache[_locked]() to support decommitted allocation, and use this ability in arena_stash_dirty(), so that decommitted extents are not needlessly committed during purging. In practice this does not happen on any currently supported systems, because both extent merging and decommit must be implemented; all supported systems implement one xor the other.
* Fix long spinning in rtree_node_initDave Watson2016-11-031-14/+9
| | | | | | | | | | | | | | | | | rtree_node_init spinlocks the node, allocates, and then sets the node. This is under heavy contention at the top of the tree if many threads start to allocate at the same time. Instead, take a per-rtree sleeping mutex to reduce spinning. Tested both pthreads and osx OSSpinLock, and both reduce spinning adequately Previous benchmark time: ./ttest1 500 100 ~15s New benchmark time: ./ttest1 500 100 .57s
* Check for existance of CPU_COUNT macro before using it.Dave Watson2016-11-031-1/+7
| | | | This resolves #485.
* Do not use syscall(2) on OS X 10.12 (deprecated).Jason Evans2016-11-032-4/+4
|
* Add os_unfair_lock support.Jason Evans2016-11-031-0/+2
| | | | | OS X 10.12 deprecated OSSpinLock; os_unfair_lock is the recommended replacement.
* Fix/refactor zone allocator integration code.Jason Evans2016-11-031-84/+107
| | | | | | | | | Fix zone_force_unlock() to reinitialize, rather than unlocking mutexes, since OS X 10.12 cannot tolerate a child unlocking mutexes that were locked by its parent. Refactor; this was a side effect of experimenting with zone {de,re}registration during fork(2).
* malloc_stats_print() fixes/cleanups.Jason Evans2016-11-011-18/+3
| | | | | | Fix and clean up various malloc_stats_print() issues caused by 0ba5b9b6189e16a983d8922d8c5cb6ab421906e8 (Add "J" (JSON) support to malloc_stats_print().).
* Add "J" (JSON) support to malloc_stats_print().Jason Evans2016-11-011-313/+716
| | | | This resolves #474.
* Fix extent_rtree acquire() to release element on error.Jason Evans2016-10-311-1/+3
| | | | This resolves #480.
* Use CLOCK_MONOTONIC_COARSE rather than COARSE_MONOTONIC_RAW.Jason Evans2016-10-301-2/+2
| | | | | | | | The raw clock variant is slow (even relative to plain CLOCK_MONOTONIC), whereas the coarse clock variant is faster than CLOCK_MONOTONIC, but still has resolution (~1ms) that is adequate for our purposes. This resolves #479.
* Use syscall(2) rather than {open,read,close}(2) during boot.Jason Evans2016-10-301-0/+19
| | | | | | | | | Some applications wrap various system calls, and if they call the allocator in their wrappers, unexpected reentry can result. This is not a general solution (many other syscalls are spread throughout the code), but this resolves a bootstrapping issue that is apparently common. This resolves #443.
* Do not mark malloc_conf as weak on Windows.Jason Evans2016-10-291-1/+1
| | | | | | | This works around malloc_conf not being properly initialized by at least the cygwin toolchain. Prior build system changes to use -Wl,--[no-]whole-archive may be necessary for malloc_conf resolution to work properly as a non-weak symbol (not tested).
* Do not mark malloc_conf as weak for unit tests.Jason Evans2016-10-291-1/+5
| | | | | | | This is generally correct (no need for weak symbols since no jemalloc library is involved in the link phase), and avoids linking problems (apparently unininitialized non-NULL malloc_conf) when using cygwin with gcc.
* Support static linking of jemalloc with glibcDave Watson2016-10-281-0/+31
| | | | | | | | | | | | | | | | | | | | | | | glibc defines its malloc implementation with several weak and strong symbols: strong_alias (__libc_calloc, __calloc) weak_alias (__libc_calloc, calloc) strong_alias (__libc_free, __cfree) weak_alias (__libc_free, cfree) strong_alias (__libc_free, __free) strong_alias (__libc_free, free) strong_alias (__libc_malloc, __malloc) strong_alias (__libc_malloc, malloc) The issue is not with the weak symbols, but that other parts of glibc depend on __libc_malloc explicitly. Defining them in terms of jemalloc API's allows the linker to drop glibc's malloc.o completely from the link, and static linking no longer results in symbol collisions. Another wrinkle: jemalloc during initialization calls sysconf to get the number of CPU's. GLIBC allocates for the first time before setting up isspace (and other related) tables, which are used by sysconf. Instead, use the pthread API to get the number of CPUs with GLIBC, which seems to work. This resolves #442.
* Fix over-sized allocation of rtree leaf nodes.Jason Evans2016-10-281-1/+1
| | | | | Use the correct level metadata when allocating child nodes so that leaf nodes don't end up over-sized (2^16 elements vs 2^4 elements).
* Uniformly cast mallctl[bymib]() oldp/newp arguments to (void *).Jason Evans2016-10-283-21/+29
| | | | | This avoids warnings in some cases, and is otherwise generally good hygiene.
* Do not (recursively) allocate within tsd_fetch().Jason Evans2016-10-215-75/+77
| | | | | | | Refactor tsd so that tsdn_fetch() does not trigger allocation, since allocation could cause infinite recursion. This resolves #458.
* Make dss operations lockless.Jason Evans2016-10-136-127/+121
| | | | | | | | | | | | | | Rather than protecting dss operations with a mutex, use atomic operations. This has negligible impact on synchronization overhead during typical dss allocation, but is a substantial improvement for extent_in_dss() and the newly added extent_dss_mergeable(), which can be called multiple times during extent deallocations. This change also has the advantage of avoiding tsd in deallocation paths associated with purging, which resolves potential deadlocks during thread exit due to attempted tsd resurrection. This resolves #425.
* Add/use adaptive spinning.Jason Evans2016-10-133-2/+10
| | | | | | | | Add spin_t and spin_{init,adaptive}(), which provide a simple abstraction for adaptive spinning. Adaptively spin during busy waits in bootstrapping and rtree node initialization.
* Remove all vestiges of chunks.Jason Evans2016-10-127-91/+8
| | | | | | | | Remove mallctls: - opt.lg_chunk - stats.cactive This resolves #464.
* Remove ratio-based purging.Jason Evans2016-10-124-279/+29
| | | | | | | | | | | | | Make decay-based purging the default (and only) mode. Remove associated mallctls: - opt.purge - opt.lg_dirty_mult - arena.<i>.lg_dirty_mult - arenas.lg_dirty_mult - stats.arenas.<i>.lg_dirty_mult This resolves #385.
generated by cgit v0.12 at 2025-04-22 06:24:57 (GMT)