Refactor/fix arenas manipulation.

Abstract arenas access to use arena_get() (or a0get() where appropriate)
rather than directly reading e.g. arenas[ind].  Prior to the addition of
the arenas.extend mallctl, the worst possible outcome of directly
accessing arenas was a stale read, but arenas.extend may allocate and
assign a new array to arenas.

Add a tsd-based arenas_cache, which amortizes arenas reads.  This
introduces some subtle bootstrapping issues, with tsd_boot() now being
split into tsd_boot[01]() to support tsd wrapper allocation
bootstrapping, as well as an arenas_cache_bypass tsd variable which
dynamically terminates allocation of arenas_cache itself.

Promote a0malloc(), a0calloc(), and a0free() to be generally useful for
internal allocation, and use them in several places (more may be
appropriate).

Abstract arena->nthreads management and fix a missing decrement during
thread destruction (recent tsd refactoring left arenas_cleanup()
unused).

Change arena_choose() to propagate OOM, and handle OOM in all callers.
This is important for providing consistent allocation behavior when the
MALLOCX_ARENA() flag is being used.  Prior to this fix, it was possible
for an OOM to result in allocation silently allocating from a different
arena than the one specified.

1 files changed, 13 insertions, 6 deletions

diff --git a/src/tsd.c b/src/tsd.c
index cbc64e4..59253fe 100644
--- a/src/tsd.c
+++ b/src/tsd.c
@@ -15,16 +15,14 @@ void *
 malloc_tsd_malloc(size_t size)
 {
 
-	/* Avoid choose_arena() in order to dodge bootstrapping issues. */
-	return (arena_malloc(NULL, arenas[0], CACHELINE_CEILING(size), false,
-	    false));
+	return (a0malloc(CACHELINE_CEILING(size)));
 }
 
 void
 malloc_tsd_dalloc(void *wrapper)
 {
 
-	idalloct(NULL, wrapper, false);
+	a0free(wrapper);
 }
 
 void
@@ -106,15 +104,24 @@ MALLOC_TSD
 }
 
 bool
-malloc_tsd_boot(void)
+malloc_tsd_boot0(void)
 {
 
 	ncleanups = 0;
-	if (tsd_boot())
+	if (tsd_boot0())
 		return (true);
+	*tsd_arenas_cache_bypassp_get(tsd_fetch()) = true;
 	return (false);
 }
 
+void
+malloc_tsd_boot1(void)
+{
+
+	tsd_boot1();
+	*tsd_arenas_cache_bypassp_get(tsd_fetch()) = false;
+}
+
 #ifdef _WIN32
 static BOOL WINAPI
 _tls_callback(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)