summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* Replace spin_init() with SPIN_INITIALIZER.Jason Evans2017-02-095-12/+4
|
* Remove rtree support for 0 (NULL) keys.Jason Evans2017-02-093-45/+43
| | | | | NULL can never actually be inserted in practice, and removing support allows a branch to be removed from the fast path.
* Determine rtree levels at compile time.Jason Evans2017-02-099-272/+248
| | | | | | | Rather than dynamically building a table to aid per level computations, define a constant table at compile time. Omit both high and low insignificant bits. Use one to three tree levels, depending on the number of significant bits.
* Remove rtree leading 0 bit optimization.Jason Evans2017-02-092-84/+16
| | | | A subsequent change instead ignores insignificant high bits.
* Make non-essential inline rtree functions static functions.Jason Evans2017-02-094-119/+85
|
* Split rtree_elm_lookup_hard() out of rtree_elm_lookup().Jason Evans2017-02-094-101/+111
| | | | | Anything but a hit in the first element of the lookup cache is expensive enough to negate the benefits of inlining.
* Replace rtree path cache with LRU cache.Jason Evans2017-02-094-124/+108
| | | | | | | Rework rtree_ctx_t to encapsulate an rtree leaf LRU lookup cache rather than a single-path element lookup cache. The replacement is logically much simpler, as well as slightly faster in the fast path case and less prone to degraded performance during non-trivial sequences of lookups.
* Optimize a branch out of rtree_read() if !dependent.Jason Evans2017-02-091-1/+1
|
* Conditianalize lg_tcache_max use on JEMALLOC_TCACHE.Jason Evans2017-02-071-1/+5
|
* Fix extent_record().Jason Evans2017-02-071-18/+33
| | | | | | | | | | | | | | Read adjacent rtree elements while holding element locks, since the extents mutex only protects against relevant like-state extent mutation. Fix management of the 'coalesced' loop state variable to merge forward/backward results, rather than overwriting the result of forward coalescing if attempting to coalesce backward. In practice this caused no correctness issues, but could cause extra iterations in rare cases. These regressions were introduced by d27f29b468ae3e9d2b1da4a9880351d76e5a1662 (Disentangle arena and extent locking.).
* Fix a race in extent_grow_retained().Jason Evans2017-02-041-9/+14
| | | | | | | | | | | | | Set extent as active prior to registration so that other threads can't modify it in the absence of locking. This regression was introduced by d27f29b468ae3e9d2b1da4a9880351d76e5a1662 (Disentangle arena and extent locking.), via non-obvious means. Removal of extents_mtx protection during extent_grow_retained() execution opened up the race, but in the presence of that locking, the code was safe. This resolves #599.
* Optimize compute_size_with_overflow().Jason Evans2017-02-041-5/+16
| | | | Do not check for overflow unless it is actually a possibility.
* Fix compute_size_with_overflow().Jason Evans2017-02-041-1/+1
| | | | | | | Fix compute_size_with_overflow() to use a high_bits mask that has the high bits set, rather than the low bits. This regression was introduced by 5154ff32ee8c37bacb6afd8a07b923eb33228357 (Unify the allocation paths).
* Disentangle arena and extent locking.Jason Evans2017-02-0219-645/+767
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor arena and extent locking protocols such that arena and extent locks are never held when calling into the extent_*_wrapper() API. This requires extra care during purging since the arena lock no longer protects the inner purging logic. It also requires extra care to protect extents from being merged with adjacent extents. Convert extent_t's 'active' flag to an enumerated 'state', so that retained extents are explicitly marked as such, rather than depending on ring linkage state. Refactor the extent collections (and their synchronization) for cached and retained extents into extents_t. Incorporate LRU functionality to support purging. Incorporate page count accounting, which replaces arena->ndirty and arena->stats.retained. Assert that no core locks are held when entering any internal [de]allocation functions. This is in addition to existing assertions that no locks are held when entering external [de]allocation functions. Audit and document synchronization protocols for all arena_t fields. This fixes a potential deadlock due to recursive allocation during gdump, in a similar fashion to b49c649bc18fff4bd10a1c8adbaf1f25f6453cb6 (Fix lock order reversal during gdump.), but with a necessarily much broader code impact.
* Fix/refactor tcaches synchronization.Jason Evans2017-02-026-29/+101
| | | | | | | Synchronize tcaches with tcaches_mtx rather than ctl_mtx. Add missing synchronization for tcache flushing. This bug was introduced by 1cb181ed632e7573fb4eab194e4d216867222d27 (Implement explicit tcache support.), which was first released in 4.0.0.
* Add witness_assert_depth[_to_rank]().Jason Evans2017-02-026-26/+84
| | | | | This makes it possible to make lock state assertions about precisely which locks are held.
* Synchronize extent_grow_next accesses.Jason Evans2017-02-021-3/+15
| | | | | | This should have been part of 411697adcda2fd75e135cdcdafb95f2bd295dc7f (Use exponential series to size extents.), which introduced extent_grow_next.
* Call prof_gctx_create() without owing bt2gctx_mtx.Jason Evans2017-02-021-12/+29
| | | | | | | This reduces the probability of allocating (and thereby indirectly making a system call) while owning bt2gctx_mtx. Unfortunately it is an incomplete solution, because ckh insertion/deletion can also allocate/deallocate, which requires more extensive changes to address.
* Conditionalize prof fork handling on config_prof.Jason Evans2017-02-021-4/+4
| | | | This allows the compiler to completely remove dead code.
* Handle race in stats_arena_bins_printQi Wang2017-02-011-2/+11
| | | | | | | When multiple threads calling stats_print, race could happen as we read the counters in separate mallctl calls; and the removed assertion could fail when other operations happened in between the mallctl calls. For simplicity, output "race" in the utilization field in this case.
* Silence harmless warnings discovered via run_tests.sh.Jason Evans2017-02-011-2/+5
|
* CI: Run --enable-debug builds on windowsDavid Goldblatt2017-02-011-1/+15
| | | | This will hopefully catch some windows-specific bugs.
* Introduce scripts to run all possible testsDavid Goldblatt2017-01-312-0/+45
| | | | | | | | | | | | | | In 6e7d0890 we added better travis continuous integration tests. This is nice, but has two problems: - We run only a subset of interesting tests. - The travis builds can take hours to give us back results (especially on OS X). This adds scripts/gen_run_tests.py, and its output, run_tests.sh, which builds and runs a larger portion of possible configurations on the local machine. While a travis run takes several hours to complete , I can run these scripts on my (OS X) latop and (Linux) devserve, and get a more exhaustive set of results back in around 10 minutes.
* Beef up travis CI integration testingDavid Goldblatt2017-01-272-11/+159
| | | | | | | | | | | | | Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
* Fix a bug in which a potentially invalid usize replaced sizeDavid Goldblatt2017-01-251-3/+3
| | | | | | | | | In the refactoring that unified the allocation paths, usize was substituted for size. This worked fine under the default test configuration, but triggered asserts when we started beefing up our CI testing. This change fixes the issue, and clarifies the comment describing the argument selection that it got wrong.
* Avoid redeclaring glibc's secure_getenvTamir Duberstein2017-01-251-4/+6
| | | | | | Avoid the name secure_getenv to avoid redeclaring secure_getenv when secure_getenv is present but its use is manually disabled via ac_cv_func_secure_getenv=no.
* Replace tabs following #define with spaces.Jason Evans2017-01-21110-713/+712
| | | | This resolves #564.
* Remove extraneous parens around return arguments.Jason Evans2017-01-21104-1168/+1161
| | | | This resolves #540.
* Update brace style.Jason Evans2017-01-21119-3572/+2971
| | | | | | | Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
* Unify the allocation pathsDavid Goldblatt2017-01-201-392/+505
| | | | | | | | | This unifies the allocation paths for malloc, posix_memalign, aligned_alloc, calloc, memalign, valloc, and mallocx, so that they all share common code where they can. There's more work that could be done here, but I think this is the smallest discrete change in this direction.
* Fix --disable-stats support.Jason Evans2017-01-205-195/+249
| | | | | Fix numerous regressions that were exposed by --disable-stats, both in the core library and in the tests.
* Test JSON output of malloc_stats_print() and fix bugs.Jason Evans2017-01-193-28/+1044
| | | | | | | | Implement and test a JSON validation parser. Use the parser to validate JSON output from malloc_stats_print(), with a significant subset of supported output options. This resolves #551.
* Remove -Werror=declaration-after-statement.Jason Evans2017-01-191-1/+0
| | | | This partially resolves #536.
* Added stats about number of bytes cached in tcache currently.Qi Wang2017-01-184-0/+40
|
* Add dummy implementations for most remaining OSX zone allocator functionsMike Hommey2017-01-181-10/+108
| | | | | | | | | | | | | | | | | | | | | | | | | Some system libraries are using malloc_default_zone() and then using some of the malloc_zone_* API. Under normal conditions, those functions check the malloc_zone_t/malloc_introspection_t struct for the values that are allowed to be NULL, so that a NULL deref doesn't happen. As of OSX 10.12, malloc_default_zone() doesn't return the actual default zone anymore, but returns a fake, wrapper zone. The wrapper zone defines all the possible functions in the malloc_zone_t/malloc_introspection_t struct (almost), and calls the function from the registered default zone (jemalloc in our case) on its own. Without checking whether the pointers are NULL. This means that a system library that calls e.g. malloc_zone_batch_malloc(malloc_default_zone(), ...) ends up trying to call jemalloc_zone.batch_malloc, which is NULL, and crash follows. So as of OSX 10.12, the default zone is required to have all the functions available (really, the same as the wrapper zone), even if they do nothing. This is arguably a bug in libsystem_malloc in OSX 10.12, but jemalloc still needs to work in that case.
* Don't rely on OSX SDK malloc/malloc.h for malloc_zone struct definitionsMike Hommey2017-01-184-69/+86
| | | | | | | | | | The SDK jemalloc is built against might be not be the latest for various reasons, but the resulting binary ought to work on newer versions of OSX. In order to ensure this, we need the fullest definitions possible, so copy what we need from the latest version of malloc/malloc.h available on opensource.apple.com.
* Fix prof_realloc() regression.Jason Evans2017-01-176-54/+206
| | | | | | | | | | | | | | Mostly revert the prof_realloc() changes in 498856f44a30b31fe713a18eb2fc7c6ecf3a9f63 (Move slabs out of chunks.) so that prof_free_sampled_object() is called when appropriate. Leave the prof_tctx_[re]set() optimization in place, but add an assertion to verify that all eight cases are correctly handled. Add a comment to make clear the code ordering, so that the regression originally fixed by ea8d97b8978a0c0423f0ed64332463a25b787c3d (Fix prof_{malloc,free}_sample_object() call order in prof_realloc().) is not repeated. This resolves #499.
* Formatting/comment fixes.Jason Evans2017-01-172-3/+2
|
* Add nullptr support to sized delete operators.Jason Evans2017-01-172-0/+16
|
* Fix style nits.Jason Evans2017-01-171-19/+18
|
* Remove redundent stats-merging logic when destroying tcache.Qi Wang2017-01-171-11/+4
| | | | The removed stats merging logic is already taken care of by tcache_flush.
* Remove leading blank lines from function bodies.Jason Evans2017-01-13103-611/+0
| | | | This resolves #535.
* Fix indentation.Jason Evans2017-01-132-4/+4
|
* Break up headers into constituent partsDavid Goldblatt2017-01-1294-3614/+3455
| | | | | | | | | | This is part of a broader change to make header files better represent the dependencies between one another (see https://github.com/jemalloc/jemalloc/issues/533). It breaks up component headers into smaller parts that can be made to have a simpler dependency graph. For the autogenerated headers (smoothstep.h and size_classes.h), no splitting was necessary, so I didn't add support to emit multiple headers.
* Remove mb.h, which is unusedDavid Goldblatt2017-01-112-119/+0
|
* Use better pre-processor defines for sparc64John Paul Adrian Glaubitz2017-01-111-1/+1
| | | | | | | | Currently, jemalloc detects sparc64 targets by checking whether __sparc64__ is defined. However, this definition is used on BSD targets only. Linux targets define both __sparc__ and __arch64__ for sparc64. Since this also works on BSD, rather use __sparc__ and __arch64__ instead of __sparc64__ to detect sparc64 targets.
* Implement arena.<i>.destroy .Jason Evans2017-01-0716-140/+620
| | | | | | | Add MALLCTL_ARENAS_DESTROYED for accessing destroyed arena stats as an analogue to MALLCTL_ARENAS_ALL. This resolves #382.
* Refactor test extent hook code to be reusable.Jason Evans2017-01-073-348/+366
| | | | | | Move test extent hook code from the extent integration test into a header, and normalize the out-of-band controls and introspection. Also refactor the base unit test to use the header.
* Replace the arenas.initialized mallctl with arena.<i>.initialized .Jason Evans2017-01-074-64/+78
|
* Range-check mib[1] --> arena_ind casts.Jason Evans2017-01-072-8/+23
|
generated by cgit v0.12 at 2025-03-17 10:02:16 (GMT)