summaryrefslogtreecommitdiffstats
path: root/src/jemalloc.c
Commit message (Collapse)AuthorAgeFilesLines
* Not re-enable background thread after fork.Qi Wang2017-06-121-1/+0
| | | | Avoid calling pthread_create in postfork handlers.
* Update a UTRACE() size argument.Jason Evans2017-06-081-1/+1
|
* Drop high rank locks when creating threads.Qi Wang2017-06-081-1/+35
| | | | | | Avoid holding arenas_lock and background_thread_lock when creating background threads, because pthread_create may take internal locks, and potentially cause deadlock with jemalloc internal locks.
* Make tsd no-cleanup during tsd reincarnation.Qi Wang2017-06-071-1/+2
| | | | | Since tsd cleanup isn't guaranteed when reincarnated, we set up tsd in a way that needs no cleanup, by making it going through slow path instead.
* Set reentrancy level to 1 during init.Qi Wang2017-06-021-15/+28
| | | | This makes sure we go down slow path w/ a0 in init.
* Refactor/fix background_thread/percpu_arena bootstrapping.Jason Evans2017-06-011-27/+36
| | | | | Refactor bootstrapping such that dlsym() is called during the bootstrapping phase that can tolerate reentrant allocation.
* Witness assertions: only assert locklessness when non-reentrant.David Goldblatt2017-06-011-49/+62
| | | | | Previously we could still hit these assertions down error paths or in the extended API.
* Header refactoring: Pull size helpers out of jemalloc module.David Goldblatt2017-05-311-125/+24
|
* Header refactoring: unify and de-catchall extent_mmap module.David Goldblatt2017-05-311-0/+1
|
* Header refactoring: unify and de-catchall extent_dss.David Goldblatt2017-05-311-0/+1
|
* Header refactoring: unify and de-catchall rtree module.David Goldblatt2017-05-311-0/+1
|
* Explicitly say so when aborting on opt_abort_conf.Qi Wang2017-05-311-2/+10
|
* Add opt.stats_print_opts.Qi Wang2017-05-291-1/+30
| | | | The value is passed to atexit(3)-triggered malloc_stats_print() calls.
* Added opt_abort_conf: abort on invalid config options.Qi Wang2017-05-271-0/+18
|
* Header refactoring: unify and de-catchall mutex moduleDavid Goldblatt2017-05-241-0/+1
|
* Header refactoring: unify and de-catchall witness code.David Goldblatt2017-05-241-38/+40
|
* Implementing opt.background_thread.Qi Wang2017-05-231-3/+50
| | | | | | | | | | | Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
* Allow mutexes to take a lock ordering enum at construction.David Goldblatt2017-05-191-2/+4
| | | | | | | This lets us specify whether and how mutexes of the same rank are allowed to be acquired. Currently, we only allow two polices (only a single mutex at a given rank at a time, and mutexes acquired in ascending order), but we can plausibly allow more (e.g. the "release uncontended mutexes before blocking").
* Refactor *decay_time into *decay_ms.Jason Evans2017-05-181-4/+8
| | | | | | | | Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
* Refactor (MALLOCX_ARENA_MAX + 1) to be MALLOCX_ARENA_LIMIT.Jason Evans2017-05-141-5/+5
| | | | This resolves #673.
* Automatically generate private symbol name mangling macros.Jason Evans2017-05-121-18/+29
| | | | | | | | Rather than using a manually maintained list of internal symbols to drive name mangling, add a compilation phase to automatically extract the list of internal symbols. This resolves #677.
* Header refactoring: tsd - cleanup and dependency breaking.David Goldblatt2017-05-011-2/+6
| | | | | | | | | | | | This removes the tsd macros (which are used only for tsd_t in real builds). We break up the circular dependencies involving tsd. We also move all tsd access through getters and setters. This allows us to assert that we only touch data when tsd is in a valid state. We simplify the usages of the x macro trick, removing all the customizability (get/set, init, cleanup), moving the lifetime logic to tsd_init and tsd_cleanup. This lets us make initialization order independent of order within tsd_t.
* Refactor !opt.munmap to opt.retain.Jason Evans2017-04-291-1/+1
|
* Header refactoring: ctl - unify and remove from catchall.David Goldblatt2017-04-251-0/+1
| | | | | In order to do this, we introduce the mutex_prof module, which breaks a circular dependency between ctl and prof.
* Replace --disable-munmap with opt.munmap.Jason Evans2017-04-251-22/+16
| | | | | | | | | Control use of munmap(2) via a run-time option rather than a compile-time option (with the same per platform default). The old behavior of --disable-munmap can be achieved with --with-malloc-conf=munmap:false. This partially resolves #580.
* Header refactoring: size_classes module - remove from the catchallDavid Goldblatt2017-04-241-0/+1
|
* Header refactoring: ticker module - remove from the catchall and unify.David Goldblatt2017-04-241-0/+1
|
* Get rid of most of the various inline macros.David Goldblatt2017-04-241-16/+16
|
* Enable -Wundef, when supported.David Goldblatt2017-04-221-9/+1
| | | | | | This can catch bugs in which one header defines a numeric constant, and another uses it without including the defining header. Undefined preprocessor symbols expand to '0', so that this will compile fine, silently doing the math wrong.
* Remove --enable-ivsalloc.Jason Evans2017-04-211-4/+11
| | | | | | | | | | | | Continue to use ivsalloc() when --enable-debug is specified (and add assertions to guard against 0 size), but stop providing a documented explicit semantics-changing band-aid to dodge undefined behavior in sallocx() and malloc_usable_size(). ivsalloc() remains compiled in, unlike when #211 restored --enable-ivsalloc, and if JEMALLOC_FORCE_IVSALLOC is defined during compilation, sallocx() and malloc_usable_size() will still use ivsalloc(). This partially resolves #580.
* Remove --disable-tcache.Jason Evans2017-04-211-8/+5
| | | | | | | | | | | Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
* Support --with-lg-page values larger than system page size.Jason Evans2017-04-191-1/+3
| | | | | | | | | All mappings continue to be PAGE-aligned, even if the system page size is smaller. This change is primarily intended to provide a mechanism for supporting multiple page sizes with the same binary; smaller page sizes work better in conjunction with jemalloc's design. This resolves #467.
* Header refactoring: unify spin.h and move it out of the catch-all.David Goldblatt2017-04-191-0/+1
|
* Header refactoring: move jemalloc_internal_types.h out of the catch-allDavid Goldblatt2017-04-191-0/+1
|
* Header refactoring: move assert.h out of the catch-allDavid Goldblatt2017-04-191-0/+1
|
* Header refactoring: move util.h out of the catchallDavid Goldblatt2017-04-191-0/+1
|
* Header refactoring: move malloc_io.h out of the catchallDavid Goldblatt2017-04-191-0/+1
|
* Switch to fine-grained reentrancy support.Qi Wang2017-04-151-71/+50
| | | | | | | Previously we had a general detection and support of reentrancy, at the cost of having branches and inc / dec operations on fast paths. To avoid taxing fast paths, we move the reentrancy operations onto tsd slow state, and only modify reentrancy level around external calls (that might trigger reentrancy).
* Bundle 3 branches on fast path into tsd_state.Qi Wang2017-04-141-36/+65
| | | | | | Added tsd_state_nominal_slow, which on fast path malloc() incorporates tcache_enabled check, and on fast path free() bundles both malloc_slow and tcache_enabled branches.
* Pass alloc_ctx down profiling path.Qi Wang2017-04-121-30/+54
| | | | | | With this change, when profiling is enabled, we avoid doing redundant rtree lookups. Also changed dalloc_atx_t to alloc_atx_t, as it's now used on allocation path as well (to speed up profiling).
* Pass dalloc_ctx down the sdalloc path.Qi Wang2017-04-121-2/+11
| | | | This avoids redundant rtree lookups.
* Header refactoring: move atomic.h out of the catch-allDavid Goldblatt2017-04-111-0/+2
|
* Header refactoring: Split up jemalloc_internal.hDavid Goldblatt2017-04-111-1/+2
| | | | | | | | | | | | | | This is a biggy. jemalloc_internal.h has been doing multiple jobs for a while now: - The source of system-wide definitions. - The catch-all include file. - The module header file for jemalloc.c This commit splits up this functionality. The system-wide definitions responsibility has moved to jemalloc_preamble.h. The catch-all include file is now jemalloc_internal_includes.h. The module headers for jemalloc.c are now in jemalloc_internal_[externs|inlines|types].h, just as they are for the other modules.
* Pass dealloc_ctx down free() fast path.Qi Wang2017-04-111-5/+13
| | | | This gets rid of the redundent rtree lookup down fast path.
* Move reentrancy_level to the beginning of TSD.Qi Wang2017-04-071-1/+1
|
* Add basic reentrancy-checking support, and allow arena_new to reenter.David Goldblatt2017-04-071-12/+82
| | | | | | | | | This checks whether or not we're reentrant using thread-local data, and, if we are, moves certain internal allocations to use arena 0 (which should be properly initialized after bootstrapping). The immediate thing this allows is spinning up threads in arena_new, which will enable spinning up background threads there.
* Integrate auto tcache into TSD.Qi Wang2017-04-071-8/+8
| | | | | | | | | The embedded tcache is initialized upon tsd initialization. The avail arrays for the tbins will be allocated / deallocated accordingly during init / cleanup. With this change, the pointer to the auto tcache will always be available, as long as we have access to the TSD. tcache_available() (called in tcache_get()) is provided to check if we should use tcache.
* Move arena-tracking atomics in jemalloc.c to C11-styleDavid Goldblatt2017-04-051-6/+8
|
* Do proper cleanup for tsd_state_reincarnated.Qi Wang2017-04-041-7/+2
| | | | | Also enable arena_bind under non-nominal state, as the cleanup will be handled correctly now.
* Force inline ifree to avoid function call costs on fast path.Qi Wang2017-03-251-2/+2
| | | | | Without ALWAYS_INLINE, sometimes ifree() gets compiled into its own function, which adds overhead on the fast path.
generated by cgit v0.12 at 2026-01-05 05:57:11 (GMT)