summaryrefslogtreecommitdiffstats
path: root/src/arena.c
Commit message (Collapse)AuthorAgeFilesLines
* Implement arena regind computation using div_info_t.David Goldblatt2017-12-211-17/+16
| | | | | This eliminates the need to generate an enormous switch statement in arena_slab_regind.
* Split up and standardize naming of stats code.David T. Goldblatt2017-12-191-141/+2
| | | | | | The arena-associated stats are now all prefixed with arena_stats_, and live in their own file. Likewise, malloc_bin_stats_t -> bin_stats_t, also in its own file.
* Move bin stats code from arena to bin module.David T. Goldblatt2017-12-191-14/+1
|
* Move bin forking code from arena to bin module.David T. Goldblatt2017-12-191-3/+3
|
* Move bin initialization from arena module to bin module.David T. Goldblatt2017-12-191-9/+2
|
* Pull out arena_bin_info_t and arena_bin_t into their own file.David T. Goldblatt2017-12-191-57/+39
| | | | | In the process, kill arena_bin_index, which is unused. To follow are several diffs continuing this separation.
* Over purge by 1 extent always.Qi Wang2017-12-181-2/+2
| | | | | | | | | When purging, large allocations are usually the ones that cross the npages_limit threshold, simply because they are "large". This means we often leave the large extent around for a while, which has the downsides of: 1) high RSS and 2) more chance of them getting fragmented. Given that they are not likely to be reused very soon (LRU), let's over purge by 1 extent (which is often large and not reused frequently).
* Fix unbounded increase in stash_decayed.Qi Wang2017-11-091-12/+18
| | | | | | Added an upper bound on how many pages we can decay during the current run. Without this, decay could have unbounded increase in stashed, since other threads could add new pages into the extents.
* Add arena.i.retain_grow_limitQi Wang2017-11-031-0/+28
| | | | | | | This option controls the max size when grow_retained. This is useful when we have customized extent hooks reserving physical memory (e.g. 1G huge pages). Without this feature, the default increasing sequence could result in fragmented and wasted physical memory.
* Clear cache bin ql postfork.Qi Wang2017-09-121-0/+7
| | | | | This fixes a regression in 9c05490, which introduced the new cache bin ql. The list needs to be cleaned up after fork, same as tcache_ql.
* Add stats for metadata_thp.Qi Wang2017-08-301-2/+3
| | | | Report number of THPs used in arena and aggregated stats.
* Make arena stats collection go through cache bins.David Goldblatt2017-08-171-4/+4
| | | | | | This eliminates the need for the arena stats code to "know" about tcaches; all that it needs is a cache_bin_array_descriptor_t to tell it where to find cache_bins whose stats it should aggregate.
* Pull out caching for a bin into its own file.David Goldblatt2017-08-171-3/+3
| | | | | | This is the first step towards breaking up the tcache and arena (since they interact primarily at the bin level). It should also make a future arena caching implementation more straightforward.
* Add extent_grow_mtx in pre_ / post_fork handlers.Qi Wang2017-06-301-4/+11
| | | | This fixed the issue that could cause the child process to stuck after fork.
* Check arena in current context in pre_reentrancy.Qi Wang2017-06-231-1/+1
|
* Set reentrancy when invoking customized extent hooks.Qi Wang2017-06-231-2/+2
| | | | | Customized extent hooks may malloc / free thus trigger reentry. Support this behavior by adding reentrancy on hook functions.
* Avoid inactivity_check within background threads.Qi Wang2017-06-221-17/+22
| | | | | | Passing is_background_thread down the decay path, so that background thread itself won't attempt inactivity_check. This fixes an issue with background thread doing trylock on a mutex it already owns.
* Clear tcache_ql after fork in child.Qi Wang2017-06-201-0/+17
|
* Log decay->nunpurged before purging.Qi Wang2017-06-151-2/+3
| | | | | During purging, we may unlock decay->mtx. Therefore we should finish logging decay related counters before attempt to purge.
* Drop high rank locks when creating threads.Qi Wang2017-06-081-11/+0
| | | | | | Avoid holding arenas_lock and background_thread_lock when creating background threads, because pthread_create may take internal locks, and potentially cause deadlock with jemalloc internal locks.
* Refactor/fix background_thread/percpu_arena bootstrapping.Jason Evans2017-06-011-2/+7
| | | | | Refactor bootstrapping such that dlsym() is called during the bootstrapping phase that can tolerate reentrant allocation.
* Header refactoring: Pull size helpers out of jemalloc module.David Goldblatt2017-05-311-27/+28
|
* Header refactoring: unify and de-catchall extent_mmap module.David Goldblatt2017-05-311-0/+1
|
* Header refactoring: unify and de-catchall extent_dss.David Goldblatt2017-05-311-0/+1
|
* Header refactoring: unify and de-catchall rtree module.David Goldblatt2017-05-311-0/+1
|
* Fix npages during arena_decay_epoch_advance().Qi Wang2017-05-301-20/+14
| | | | | We do not lock extents while advancing epoch. This change makes sure that we only read npages from extents once in order to avoid any inconsistency.
* Fix extent_grow_next management.Jason Evans2017-05-301-4/+5
| | | | | | | | | | | | | Fix management of extent_grow_next to serialize operations that may grow retained memory. This assures that the sizes of the newly allocated extents correspond to the size classes in the intended growth sequence. Fix management of extent_grow_next to skip size classes if a request is too large to be satisfied by the next size in the growth sequence. This avoids the potential for an arbitrary number of requests to bypass triggering extent_grow_next increases. This resolves #858.
* Cleanup smoothstep.sh / .h.Qi Wang2017-05-251-1/+1
| | | | h_step_sum was used to compute moving sum. Not in use anymore.
* Fix stats.mapped during deallocation.Qi Wang2017-05-241-1/+1
|
* Header refactoring: unify and de-catchall mutex moduleDavid Goldblatt2017-05-241-0/+1
|
* Header refactoring: unify and de-catchall witness code.David Goldblatt2017-05-241-6/+12
|
* Fix # of unpurged pages in decay algorithm.Qi Wang2017-05-231-10/+26
| | | | | | | | | | When # of dirty pages move below npages_limit (e.g. they are reused), we should not lower number of unpurged pages because that would cause the reused pages to be double counted in the backlog (as a result, decay happen slower than it should). Instead, set number of unpurged to the greater of current npages and npages_limit. Added an assertion: the ceiling # of pages should be greater than npages_limit.
* Check for background thread inactivity on extents_dalloc.Qi Wang2017-05-231-0/+4
| | | | | | To avoid background threads sleeping forever with idle arenas, we eagerly check background threads' sleep time after extents_dalloc, and signal the thread if necessary.
* Implementing opt.background_thread.Qi Wang2017-05-231-55/+101
| | | | | | | | | | | Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
* Allow mutexes to take a lock ordering enum at construction.David Goldblatt2017-05-191-6/+7
| | | | | | | This lets us specify whether and how mutexes of the same rank are allowed to be acquired. Currently, we only allow two polices (only a single mutex at a given rank at a time, and mutexes acquired in ascending order), but we can plausibly allow more (e.g. the "release uncontended mutexes before blocking").
* Refactor *decay_time into *decay_ms.Jason Evans2017-05-181-64/+66
| | | | | | | | Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
* Add stats: arena uptime.Qi Wang2017-05-181-0/+7
|
* Stop depending on JEMALLOC_N() for function interception during testing.Jason Evans2017-05-121-12/+4
| | | | | | Instead, always define function pointers for interceptable functions, but mark them const unless testing, so that the compiler can optimize out the pointer dereferences.
* Add extent_destroy_t and use it during arena destruction.Jason Evans2017-04-291-11/+9
| | | | | | | | | | Add the extent_destroy_t extent destruction hook to extent_hooks_t, and use it during arena destruction. This hook explicitly communicates to the callee that the extent must be destroyed or tracked for later reuse, lest it be permanently leaked. Prior to this change, retained extents could unintentionally be leaked if extent retention was enabled. This resolves #560.
* Refactor !opt.munmap to opt.retain.Jason Evans2017-04-291-3/+3
|
* Replace --disable-munmap with opt.munmap.Jason Evans2017-04-251-2/+2
| | | | | | | | | Control use of munmap(2) via a run-time option rather than a compile-time option (with the same per platform default). The old behavior of --disable-munmap can be achieved with --with-malloc-conf=munmap:false. This partially resolves #580.
* Use trylock in arena_decay_impl().Qi Wang2017-04-241-8/+16
| | | | If another thread is working on decay, we don't have to wait for the mutex.
* Header refactoring: size_classes module - remove from the catchallDavid Goldblatt2017-04-241-0/+1
|
* Get rid of most of the various inline macros.David Goldblatt2017-04-241-3/+3
|
* Remove --disable-tcache.Jason Evans2017-04-211-32/+24
| | | | | | | | | | | Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
* Bypass extent tracking for auto arenas.Qi Wang2017-04-211-11/+22
| | | | | | | | Tracking extents is required by arena_reset. To support this, the extent linkage was used for tracking 1) large allocations, and 2) full slabs. However modifying the extent linkage could be an expensive operation as it likely incurs cache misses. Since we forbid arena_reset on auto arenas, let's bypass the linkage operations for auto arenas.
* Header refactoring: move assert.h out of the catch-allDavid Goldblatt2017-04-191-0/+1
|
* Header refactoring: move util.h out of the catchallDavid Goldblatt2017-04-191-0/+2
|
* Prefer old/low extent_t structures during reuse.Jason Evans2017-04-171-7/+7
| | | | | | Rather than using a LIFO queue to track available extent_t structures, use a red-black tree, and always choose the oldest/lowest available during reuse.
* Switch to fine-grained reentrancy support.Qi Wang2017-04-151-4/+2
| | | | | | | Previously we had a general detection and support of reentrancy, at the cost of having branches and inc / dec operations on fast paths. To avoid taxing fast paths, we move the reentrancy operations onto tsd slow state, and only modify reentrancy level around external calls (that might trigger reentrancy).
generated by cgit v0.12 at 2025-04-05 04:25:34 (GMT)