summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Combine background_thread started / paused into state.Qi Wang2017-06-122-29/+50
|
* Not re-enable background thread after fork.Qi Wang2017-06-122-36/+46
| | | | Avoid calling pthread_create in postfork handlers.
* Move background thread creation to background_thread_0.Qi Wang2017-06-122-144/+249
| | | | | To avoid complications, avoid invoking pthread_create "internally", instead rely on thread0 to launch new threads, and also terminating threads when asked.
* Normalize background thread configuration.Jason Evans2017-06-091-0/+2
| | | | Also fix a compilation error #ifndef JEMALLOC_PTHREAD_CREATE_WRAPPER.
* Update a UTRACE() size argument.Jason Evans2017-06-081-1/+1
|
* Add internal tsd for background_thread.Qi Wang2017-06-082-6/+14
|
* Drop high rank locks when creating threads.Qi Wang2017-06-084-13/+42
| | | | | | Avoid holding arenas_lock and background_thread_lock when creating background threads, because pthread_create may take internal locks, and potentially cause deadlock with jemalloc internal locks.
* Make tsd no-cleanup during tsd reincarnation.Qi Wang2017-06-072-21/+48
| | | | | Since tsd cleanup isn't guaranteed when reincarnated, we set up tsd in a way that needs no cleanup, by making it going through slow path instead.
* Remove assertions on extent_hooks being default.Qi Wang2017-06-051-16/+0
| | | | | It's possible to customize the extent_hooks while still using part of the default implementation.
* Take background thread lock when setting extent hooks.Qi Wang2017-06-052-3/+13
|
* Set reentrancy level to 1 during init.Qi Wang2017-06-021-15/+28
| | | | This makes sure we go down slow path w/ a0 in init.
* Set isthreaded when enabling background_thread.Qi Wang2017-06-022-23/+40
|
* Fix background thread creation.Qi Wang2017-06-011-13/+17
| | | | The state initialization should be done before pthread_create.
* Refactor/fix background_thread/percpu_arena bootstrapping.Jason Evans2017-06-015-93/+114
| | | | | Refactor bootstrapping such that dlsym() is called during the bootstrapping phase that can tolerate reentrant allocation.
* Witness assertions: only assert locklessness when non-reentrant.David Goldblatt2017-06-011-49/+62
| | | | | Previously we could still hit these assertions down error paths or in the extended API.
* Use real pthread_create for creating background threads.Qi Wang2017-05-312-3/+9
|
* Header refactoring: Pull size helpers out of jemalloc module.David Goldblatt2017-05-3111-189/+198
|
* Header refactoring: unify and de-catchall mutex_pool.David Goldblatt2017-05-312-0/+2
|
* Header refactoring: unify and de-catchall extent_mmap module.David Goldblatt2017-05-317-0/+7
|
* Header refactoring: unify and de-catchall extent_dss.David Goldblatt2017-05-315-0/+5
|
* Header refactoring: unify and de-catchall rtree module.David Goldblatt2017-05-315-0/+5
|
* Pass the O_CLOEXEC flag to open(2).Jason Evans2017-05-312-4/+5
| | | | This resolves #528.
* Track background thread status separately at fork.Qi Wang2017-05-311-3/+8
| | | | | Use a separate boolean to track the enabled status, instead of leaving the global background thread status inconsistent.
* Output total_wait_ns for bin mutexes.Qi Wang2017-05-311-19/+5
|
* Explicitly say so when aborting on opt_abort_conf.Qi Wang2017-05-311-2/+10
|
* Add the --disable-thp option to support cross compiling.Jason Evans2017-05-302-0/+4
| | | | This resolves #669.
* Fix npages during arena_decay_epoch_advance().Qi Wang2017-05-301-20/+14
| | | | | We do not lock extents while advancing epoch. This change makes sure that we only read npages from extents once in order to avoid any inconsistency.
* Fix extent_grow_next management.Jason Evans2017-05-302-150/+211
| | | | | | | | | | | | | Fix management of extent_grow_next to serialize operations that may grow retained memory. This assures that the sizes of the newly allocated extents correspond to the size classes in the intended growth sequence. Fix management of extent_grow_next to skip size classes if a request is too large to be satisfied by the next size in the growth sequence. This avoids the potential for an arbitrary number of requests to bypass triggering extent_grow_next increases. This resolves #858.
* Fix OOM paths in extent_grow_retained().Jason Evans2017-05-301-2/+8
|
* Add opt.stats_print_opts.Qi Wang2017-05-293-43/+52
| | | | The value is passed to atexit(3)-triggered malloc_stats_print() calls.
* Added opt_abort_conf: abort on invalid config options.Qi Wang2017-05-273-0/+22
|
* Cleanup smoothstep.sh / .h.Qi Wang2017-05-251-1/+1
| | | | h_step_sum was used to compute moving sum. Not in use anymore.
* Fix stats.mapped during deallocation.Qi Wang2017-05-241-1/+1
|
* Header refactoring: unify and de-catchall mutex moduleDavid Goldblatt2017-05-2412-0/+24
|
* Header refactoring: unify and de-catchall witness code.David Goldblatt2017-05-246-74/+100
|
* Do not assume dss never decreases.Jason Evans2017-05-231-38/+34
| | | | | | | | | An sbrk() caller outside jemalloc can decrease the dss, so add a separate atomic boolean to explicitly track whether jemalloc is concurrently calling sbrk(), rather than depending on state outside jemalloc's full control. This resolves #802.
* Do not hold the base mutex while calling extent hooks.Jason Evans2017-05-231-0/+6
| | | | | | | | | Drop the base mutex while allocating new base blocks, because extent allocation can enter code that prohibits holding non-core mutexes, e.g. the extent_[d]alloc() and extent_purge_forced_wrapper() calls in extent_alloc_dss(). This partially resolves #802.
* Fix # of unpurged pages in decay algorithm.Qi Wang2017-05-231-10/+26
| | | | | | | | | | When # of dirty pages move below npages_limit (e.g. they are reused), we should not lower number of unpurged pages because that would cause the reused pages to be double counted in the backlog (as a result, decay happen slower than it should). Instead, set number of unpurged to the greater of current npages and npages_limit. Added an assertion: the ceiling # of pages should be greater than npages_limit.
* Check for background thread inactivity on extents_dalloc.Qi Wang2017-05-232-19/+46
| | | | | | To avoid background threads sleeping forever with idle arenas, we eagerly check background threads' sleep time after extents_dalloc, and signal the thread if necessary.
* Add profiling for the background thread mutex.Qi Wang2017-05-232-0/+14
|
* Add background thread related stats.Qi Wang2017-05-234-21/+162
|
* Implementing opt.background_thread.Qi Wang2017-05-236-79/+814
| | | | | | | | | | | Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
* Protect the rtree/extent interactions with a mutex pool.David Goldblatt2017-05-193-214/+160
| | | | | | | | | | | | | | | | | | Instead of embedding a lock bit in rtree leaf elements, we associate extents with a small set of mutexes. This gets us two things: - We can use the system mutexes. This (hypothetically) protects us from priority inversion, and lets us stop doing a backoff/sleep loop, instead opting for precise wakeups from the mutex. - Cuts down on the number of mutex acquisitions we have to do (from 4 in the worst case to two). We end up simplifying most of the rtree code (which no longer has to deal with locking or concurrency at all), at the cost of additional complexity in the extent code: since the mutex protecting the rtree leaf elements is determined by reading the extent out of those elements, the initial read is racy, so that we may acquire an out of date mutex. We re-check the extent in the leaf after acquiring the mutex to protect us from this race.
* Allow mutexes to take a lock ordering enum at construction.David Goldblatt2017-05-199-27/+60
| | | | | | | This lets us specify whether and how mutexes of the same rank are allowed to be acquired. Currently, we only allow two polices (only a single mutex at a given rank at a time, and mutexes acquired in ascending order), but we can plausibly allow more (e.g. the "release uncontended mutexes before blocking").
* Refactor *decay_time into *decay_ms.Jason Evans2017-05-184-143/+136
| | | | | | | | Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
* Add stats: arena uptime.Qi Wang2017-05-183-0/+25
|
* Refactor (MALLOCX_ARENA_MAX + 1) to be MALLOCX_ARENA_LIMIT.Jason Evans2017-05-141-5/+5
| | | | This resolves #673.
* Automatically generate private symbol name mangling macros.Jason Evans2017-05-121-18/+29
| | | | | | | | Rather than using a manually maintained list of internal symbols to drive name mangling, add a compilation phase to automatically extract the list of internal symbols. This resolves #677.
* Stop depending on JEMALLOC_N() for function interception during testing.Jason Evans2017-05-126-167/+51
| | | | | | Instead, always define function pointers for interceptable functions, but mark them const unless testing, so that the compiler can optimize out the pointer dereferences.
* Revert "Use trylock in tcache_bin_flush when possible."Qi Wang2017-05-011-123/+48
| | | | | This reverts commit 8584adc451f31adfc4ab8693d9189cf3a7e5d858. Production results not favorable. Will investigate separately.
generated by cgit v0.12 at 2026-01-07 09:56:34 (GMT)