diff options
Diffstat (limited to 'doc/html/TechNotes/ThreadSafeLibrary.html')
| -rw-r--r-- | doc/html/TechNotes/ThreadSafeLibrary.html | 794 |
1 files changed, 0 insertions, 794 deletions
diff --git a/doc/html/TechNotes/ThreadSafeLibrary.html b/doc/html/TechNotes/ThreadSafeLibrary.html deleted file mode 100644 index e7fdf11..0000000 --- a/doc/html/TechNotes/ThreadSafeLibrary.html +++ /dev/null @@ -1,794 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" - "http://www.w3.org/TR/REC-html40/loose.dtd"> -<html lang="en-US"> -<head> - <title>Thread Safe Library</title> -</head> - -<body bgcolor=#ffffff> - -<center><h1>HDF5 Thread Safe library</h1></center> - -<p> - -<h1>1. Library header files and conditional compilation</h1> - -<p> -The following code is placed at the beginning of H5private.h: -</p> - -<blockquote> - <pre> - #ifdef H5_HAVE_THREADSAFE - #include <pthread.h> - #endif - </pre> -</blockquote> - -<p> -<code>H5_HAVE_THREADSAFE</code> is defined when the HDF-5 library is -compiled with the --enable-threadsafe configuration option. In general, -code for the non-threadsafe version of HDF-5 library are placed within -the <code>#else</code> part of the conditional compilation. The exception -to this rule are the changes to the <code>FUNC_ENTER</code> (in -H5private.h), <code>HRETURN</code> and <code>HRETURN_ERROR</code> (in -H5Eprivate.h) macros (see section 3.2). -</p> - - -<h1>2. Global variables/structures</h1> - -<h2>2.1 Global library initialization variable</h2> - -<p> -In the threadsafe implementation, the global library initialization -variable <code>H5_libinit_g</code> is changed to a global structure -consisting of the variable with its associated lock (locks are explained -in section 4.1): -</p> - -<blockquote> - <pre> - hbool_t H5_libinit_g = FALSE; - </pre> -</blockquote> - -<p> -becomes -</p> - -<blockquote> - <pre> - H5_api_t H5_g; - </pre> -</blockquote> - -<p> -where <code>H5_api_t</code> is -</p> - -<blockquote> - <pre> - typedef struct H5_api_struct { - H5_mutex_t init_lock; /* API entrance mutex */ - hbool_t H5_libinit_g; - } H5_api_t; - </pre> -</blockquote> - -<p> -All former references to <code>H5_libinit_g</code> in the library are now -made using the macro <code>H5_INIT_GLOBAL</code>. If the threadsafe -library is to be used, the macro is set to <code>H5_g.H5_libinit_g</code> -instead. -</p> - -<h2>2.2 Global serialization variable</h2> - -<p> -A new global boolean variable <code>H5_allow_concurrent_g</code> is used -to determine if multiple threads are allowed to an API call -simultaneously. This is set to <code>FALSE</code>. -</p> - -<p> -All APIs that are allowed to do so have their own local variable that -shadows the global variable and is set to <code>TRUE</code>. In phase 1, -no such APIs exist. -</p> - -<p> -It is defined in <code>H5.c</code> as follows: -</p> - -<blockquote> - <pre> - hbool_t H5_allow_concurrent_g = FALSE; - </pre> -</blockquote> - -<h2>2.3 Global thread initialization variable</h2> - -<p> -The global variable <code>H5_first_init_g</code> of type -<code>pthread_once_t</code> is used to allow only the first thread in the -application process to call an initialization function using -<code>pthread_once</code>. All subsequent calls to -<code>pthread_once</code> by any thread are disregarded. -</p> - -<p> -The call sets up the mutex in the global structure <code>H5_g</code> (see -section 3.1) via an initialization function -<code>H5_first_thread_init</code>. The first thread initialization -function is described in section 4.2. -</p> - -<p> -<code>H5_first_init_g</code> is defined in <code>H5.c</code> as follows: -</p> - -<blockquote> - <pre> - pthread_once_t H5_first_init_g = PTHREAD_ONCE_INIT; - </pre> -</blockquote> - -<h2>2.4 Global key for per-thread error stacks</h2> - -<p> -A global pthread-managed key <code>H5_errstk_key_g</code> is used to -allow pthreads to maintain a separate error stack (of type -<code>H5E_t</code>) for each thread. This is defined in <code>H5.c</code> -as: -</p> - -<blockquote> - <pre> - pthread_key_t H5_errstk_key_g; - </pre> -</blockquote> - -<p> -Error stack management is described in section 4.3. -</p> - -<h2>2.5 Global structure and key for thread cancellation prevention</h2> - -<p> -We need to preserve the thread cancellation status of each thread -individually by using a key <code>H5_cancel_key_g</code>. The status is -preserved using a structure (of type <code>H5_cancel_t</code>) which -maintains the cancellability state of the thread before it entered the -library and a count (which works very much like the recursive lock -counter) which keeps track of the number of API calls the thread makes -within the library. -</p> - -<p> -The structure is defined in <code>H5private.h</code> as: -</p> - -<blockquote> - <pre> - /* cancelability structure */ - typedef struct H5_cancel_struct { - int previous_state; - unsigned int cancel_count; - } H5_cancel_t; - </pre> -</blockquote> - -<p> -Thread cancellation is described in section 4.4. -</p> - - -<h1>3. Changes to Macro expansions</h1> - -<h2>3.1 Changes to FUNC_ENTER</h2> - -<p> -The <code>FUNC_ENTER</code> macro is now extended to include macro calls -to initialize first threads, disable cancellability and wraps a lock -operation around the checking of the global initialization flag. It -should be noted that the cancellability should be disabled before -acquiring the lock on the library. Doing so otherwise would allow the -possibility that the thread be cancelled just after it has acquired the -lock on the library and in that scenario, if the cleanup routines are not -properly set, the library would be permanently locked out. -</p> - -<p> -The additional macro code and new macro definitions can be found in -Appendix E.1 to E.5. The changes are made in <code>H5private.h</code>. -</p> - -<h2>3.2 Changes to HRETURN and HRETURN_ERROR</h2> - -<p> -The <code>HRETURN</code> and <code>HRETURN_ERROR</code> macros are the -counterparts to the <code>FUNC_ENTER</code> macro described in section -3.1. <code>FUNC_LEAVE</code> makes a macro call to <code>HRETURN</code>, -so it is also covered here. -</p> - -<p> -The basic changes to these two macros involve adding macro calls to call -an unlock operation and re-enable cancellability if necessary. It should -be noted that the cancellability should be re-enabled only after the -thread has released the lock to the library. The consequence of doing -otherwise would be similar to that described in section 3.1. -</p> - -<p> -The additional macro code and new macro definitions can be found in -Appendix E.9 to E.9. The changes are made in <code>H5Eprivate.h</code>. -</p> - -<h1>4. Implementation of threadsafe functionality</h1> - -<h2>4.1 Recursive Locks</h2> - -<p> -A recursive mutex lock m allows a thread t1 to successfully lock m more -than once without blocking t1. Another thread t2 will block if t2 tries -to lock m while t1 holds the lock to m. If t1 makes k lock calls on m, -then it also needs to make k unlock calls on m before it releases the -lock. -</p> - -<p> -Our implementation of recursive locks is built on top of a pthread mutex -lock (which is not recursive). It makes use of a pthread condition -variable to have unsuccessful threads wait on the mutex. Waiting threads -are awaken by a signal from the final unlock call made by the thread -holding the lock. -</p> - -<p> -Recursive locks are defined to be the following type -(<code>H5private.h</code>): -</p> - -<blockquote> - <pre> - typedef struct H5_mutex_struct { - pthread_t owner_thread; /* current lock owner */ - pthread_mutex_t atomic_lock; /* lock for atomicity of new mechanism */ - pthread_cond_t cond_var; /* condition variable */ - unsigned int lock_count; - } H5_mutex_t; - </pre> -</blockquote> - -<p> -Detailed implementation code can be found in Appendix A. The -implementation changes are made in <code>H5TS.c</code>. -</p> - -<h2>4.2 First thread initialization</h2> - -<p> -Because the mutex lock associated with a recursive lock cannot be -statically initialized, a mechanism is required to initialize the -recursive lock associated with <code>H5_g</code> so that it can be used -for the first time. -</p> - -<p> -The pthreads library allows this through the pthread_once call which as -described in section 3.3 allows only the first thread accessing the -library in an application to initialize <code>H5_g</code>. -</p> - -<p> -In addition to initializing <code>H5_g</code>, it also initializes the -key (see section 3.4) for use with per-thread error stacks (see section -4.3). -</p> - -<p> -The first thread initialization mechanism is implemented as the function -call <code>H5_first_thread_init()</code> in <code>H5TS.c</code>. This is -described in appendix B. -</p> - -<h2>4.3 Per-thread error stack management</h2> - -<p> -Pthreads allows individual threads to access dynamic and persistent -per-thread data through the use of keys. Each key is associated with -a table that maps threads to data items. Keys can be initialized by -<code>pthread_key_create()</code> in pthreads (see sections 3.4 and 4.2). -Per-thread data items are accessed using a key through the -<code>pthread_getspecific()</code> and <code>pthread_setspecific()</code> -calls to read and write to the association table respectively. -</p> - -<p> -Per-thread error stacks are accessed through the key -<code>H5_errstk_key_g</code> which is initialized by the first thread -initialization call (see section 4.2). -</p> - -<p> -In the non-threadsafe version of the library, there is a global stack -variable <code>H5E_stack_g[1]</code> which is no longer defined in the -threadsafe version. At the same time, the macro call to gain access to -the error stack <code>H5E_get_my_stack</code> is changed from: -</p> - -<blockquote> - <pre> - #define H5E_get_my_stack() (H5E_stack_g+0) - </pre> -</blockquote> - -<p> -to: -</p> - -<blockquote> - <pre> - #define H5E_get_my_stack() H5E_get_stack() - </pre> -</blockquote> - -<p> -where <code>H5E_get_stack()</code> is a surrogate function that does the -following operations: -</p> - -<ol> - <li>if a thread is attempting to get an error stack for the first - time, the error stack is dynamically allocated for the thread and - associated with <code>H5_errstk_key_g</code> using - <code>pthread_setspecific()</code>. The way we detect if it is the - first time is through <code>pthread_getspecific()</code> which - returns <code>NULL</code> if no previous value is associated with - the thread using the key.</li> - - <li>if <code>pthread_getspecific()</code> returns a non-null value, - then that is the pointer to the error stack associated with the - thread and the stack can be used as usual.</li> -</ol> - -<p> -A final change to the error reporting routines is as follows; the current -implementation reports errors to always be detected at thread 0. In the -threadsafe implementation, this is changed to report the number returned -by a call to <code>pthread_self()</code>. -</p> - -<p> -The change in code (reflected in <code>H5Eprint</code> of file -<code>H5E.c</code>) is as follows: -</p> - -<blockquote> - <pre> - #ifdef H5_HAVE_THREADSAFE - fprintf (stream, "HDF5-DIAG: Error detected in thread %d." - ,pthread_self()); - #else - fprintf (stream, "HDF5-DIAG: Error detected in thread 0."); - #endif - </pre> -</blockquote> - -<p> -Code for <code>H5E_get_stack()</code> can be found in Appendix C. All the -above changes were made in <code>H5E.c</code>. -</p> - -<h2>4.4 Thread Cancellation safety</h2> - -<p> -To prevent thread cancellations from killing a thread while it is in the -library, we maintain per-thread information about the cancellability -status of the thread before it entered the library so that we can restore -that same status when the thread leaves the library. -</p> - -<p> -By <i>enter</i> and <i>leave</i> the library, we mean the points when a -thread makes an API call from a user application and the time that API -call returns. Other API or callback function calls made from within that -API call are considered <i>within</i> the library. -</p> - -<p> -Because other API calls may be made from within the first API call, we -need to maintain a counter to determine which was the first and -correspondingly the last return. -</p> - -<p> -When a thread makes an API call, the macro <code>H5_API_SET_CANCEL</code> -calls the worker function <code>H5_cancel_count_inc()</code> which does -the following: -</p> - -<ol> - <li>if this is the first time the thread has entered the library, - a new cancellability structure needs to be assigned to it.</li> - <li>if the thread is already within the library when the API call is - made, then cancel_count is simply incremented. Otherwise, we set - the cancellability state to <code>PTHREAD_CANCEL_DISABLE</code> - while storing the previous state into the cancellability structure. - <code>cancel_count</code> is also incremented in this case.</li> -</ol> - -<p> -When a thread leaves an API call, the macro -<code>H5_API_UNSET_CANCEL</code> calls the worker function -<code>H5_cancel_count_dec()</code> which does the following: -</p> - -<ol> - <li>if <code>cancel_count</code> is greater than 1, indicating that the - thread is not yet about to leave the library, then - <code>cancel_count</code> is simply decremented.</li> - <li>otherwise, we reset the cancellability state back to its original - state before it entered the library and decrement the count (back - to zero).</li> -</ol> - -<p> -<code>H5_cancel_count_inc</code> and <code>H5_cancel_count_dec</code> are -described in Appendix D and may be found in <code>H5TS.c</code>. -</p> - -<h1>5. Test programs</h1> - -<p> -Except where stated, all tests involve 16 simultaneous threads that make -use of HDF-5 API calls without any explicit synchronization typically -required in a non-threadsafe environment. -</p> - -<h2>5.1 Data set create and write</h2> - -<p> -The test program sets up 16 threads to simultaneously create 16 -different datasets named from <i>zero</i> to <i>fifteen</i> for a single -file and then writing an integer value into that dataset equal to the -dataset's named value. -</p> - -<p> -The main thread would join with all 16 threads and attempt to match the -resulting HDF-5 file with expected results - that each dataset contains -the correct value (0 for <i>zero</i>, 1 for <i>one</i> etc ...) and all -datasets were correctly created. -</p> - -<p> -The test is implemented in the file <code>ttsafe_dcreate.c</code>. -</p> - -<h2>5.2 Test on error stack</h2> - -<p> -The error stack test is one in which 16 threads simultaneously try to -create datasets with the same name. The result, when properly serialized, -should be equivalent to 16 attempts to create the dataset with the same -name. -</p> - -<p> -The error stack implementation runs correctly if it reports 15 instances -of the dataset name conflict error and finally generates a correct HDF-5 -containing that single dataset. Each thread should report its own stack -of errors with a thread number associated with it. -</p> - -<p> -The test is implemented in the file <code>ttsafe_error.c</code>. -</p> - -<h2>5.3 Test on cancellation safety</h2> - -<p> -The main idea in thread cancellation safety is as follows; a child thread -is spawned to create and write to a dataset. Following that, it makes a -<code>H5Diterate</code> call on that dataset which activates a callback -function. -</p> - -<p> -A deliberate barrier is invoked at the callback function which waits for -both the main and child thread to arrive at that point. After that -happens, the main thread proceeds to make a thread cancel call on the -child thread while the latter sleeps for 3 seconds before proceeding to -write a new value to the dataset. -</p> - -<p> -After the iterate call, the child thread logically proceeds to wait -another 3 seconds before writing another newer value to the dataset. -</p> - -<p> -The test is correct if the main thread manages to read the second value -at the end of the test. This means that cancellation did not take place -until the end of the iteration call despite of the 3 second wait within -the iteration callback and the extra dataset write operation. -Furthermore, the cancellation should occur before the child can proceed -to write the last value into the dataset. -</p> - -<h2>5.4 Test on attribute creation</h2> - -<p> -A main thread makes 16 threaded calls to <code>H5Acreate</code> with a -generated name for each attribute. Sixteen attributes should be created -for the single dataset in random (chronological) order and receive values -depending on its generated attribute name (e.g. <i>attrib010</i> would -receive the value 10). -</p> - -<p> -After joining with all child threads, the main thread proceeds to read -each attribute by generated name to see if the value tallies. Failure is -detected if the attribute name does not exist (meaning they were never -created) or if the wrong values were read back. -</p> - -<h1>A. Recursive Lock implementation code</h1> - -<blockquote> - <pre> - void H5_mutex_init(H5_mutex_t *H5_mutex) - { - H5_mutex->owner_thread = NULL; - pthread_mutex_init(&H5_mutex->atomic_lock, NULL); - pthread_cond_init(&H5_mutex->cond_var, NULL); - H5_mutex->lock_count = 0; - } - - void H5_mutex_lock(H5_mutex_t *H5_mutex) - { - pthread_mutex_lock(&H5_mutex->atomic_lock); - - if (pthread_equal(pthread_self(), H5_mutex->owner_thread)) { - /* already owned by self - increment count */ - H5_mutex->lock_count++; - } else { - if (H5_mutex->owner_thread == NULL) { - /* no one else has locked it - set owner and grab lock */ - H5_mutex->owner_thread = pthread_self(); - H5_mutex->lock_count = 1; - } else { - /* if already locked by someone else */ - while (1) { - pthread_cond_wait(&H5_mutex->cond_var, &H5_mutex->atomic_lock); - - if (H5_mutex->owner_thread == NULL) { - H5_mutex->owner_thread = pthread_self(); - H5_mutex->lock_count = 1; - break; - } /* else do nothing and loop back to wait on condition*/ - } - } - } - - pthread_mutex_unlock(&H5_mutex->atomic_lock); - } - - void H5_mutex_unlock(H5_mutex_t *H5_mutex) - { - pthread_mutex_lock(&H5_mutex->atomic_lock); - H5_mutex->lock_count--; - - if (H5_mutex->lock_count == 0) { - H5_mutex->owner_thread = NULL; - pthread_cond_signal(&H5_mutex->cond_var); - } - pthread_mutex_unlock(&H5_mutex->atomic_lock); - } - </pre> -</blockquote> - -<h1>B. First thread initialization</h1> - -<blockquote> - <pre> - void H5_first_thread_init(void) - { - /* initialize global API mutex lock */ - H5_g.H5_libinit_g = FALSE; - H5_g.init_lock.owner_thread = NULL; - pthread_mutex_init(&H5_g.init_lock.atomic_lock, NULL); - pthread_cond_init(&H5_g.init_lock.cond_var, NULL); - H5_g.init_lock.lock_count = 0; - - /* initialize key for thread-specific error stacks */ - pthread_key_create(&H5_errstk_key_g, NULL); - - /* initialize key for thread cancellability mechanism */ - pthread_key_create(&H5_cancel_key_g, NULL); - } - </pre> -</blockquote> - - -<h1>C. Per-thread error stack acquisition</h1> - -<blockquote> - <pre> - H5E_t *H5E_get_stack(void) - { - H5E_t *estack; - - if (estack = pthread_getspecific(H5_errstk_key_g)) { - return estack; - } else { - /* no associated value with current thread - create one */ - estack = (H5E_t *)malloc(sizeof(H5E_t)); - pthread_setspecific(H5_errstk_key_g, (void *)estack); - return estack; - } - } - </pre> -</blockquote> - -<h1>D. Thread cancellation mechanisms</h1> - -<blockquote> - <pre> - void H5_cancel_count_inc(void) - { - H5_cancel_t *cancel_counter; - - if (cancel_counter = pthread_getspecific(H5_cancel_key_g)) { - /* do nothing here */ - } else { - /* - * first time thread calls library - create new counter and - * associate with key - */ - cancel_counter = (H5_cancel_t *)malloc(sizeof(H5_cancel_t)); - cancel_counter->cancel_count = 0; - pthread_setspecific(H5_cancel_key_g, (void *)cancel_counter); - } - - if (cancel_counter->cancel_count == 0) { - /* thread entering library */ - pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, - &(cancel_counter->previous_state)); - } - - cancel_counter->cancel_count++; - } - - void H5_cancel_count_dec(void) - { - H5_cancel_t *cancel_counter = pthread_getspecific(H5_cancel_key_g); - - if (cancel_counter->cancel_count == 1) - pthread_setcancelstate(cancel_counter->previous_state, NULL); - - cancel_counter->cancel_count--; - } - </pre> -</blockquote> - -<h1>E. Macro expansion codes</h1> - -<h2>E.1 <code>FUNC_ENTER</code></h2> - -<blockquote> - <pre> - /* Initialize the library */ \ - H5_FIRST_THREAD_INIT \ - H5_API_UNSET_CANCEL \ - H5_API_LOCK_BEGIN \ - if (!(H5_INIT_GLOBAL)) { \ - H5_INIT_GLOBAL = TRUE; \ - if (H5_init_library() < 0) { \ - HRETURN_ERROR (H5E_FUNC, H5E_CANTINIT, err, \ - "library initialization failed"); \ - } \ - } \ - H5_API_LOCK_END \ - : - : - : - </pre> -</blockquote> - -<h2>E.2 <code>H5_FIRST_THREAD_INIT</code></h2> - -<blockquote> - <pre> - /* Macro for first thread initialization */ - #define H5_FIRST_THREAD_INIT \ - pthread_once(&H5_first_init_g, H5_first_thread_init); - </pre> -</blockquote> - - -<h2>E.3 <code>H5_API_UNSET_CANCEL</code></h2> - -<blockquote> - <pre> - #define H5_API_UNSET_CANCEL \ - if (H5_IS_API(FUNC)) { \ - H5_cancel_count_inc(); \ - } - </pre> -</blockquote> - - -<h2>E.4 <code>H5_API_LOCK_BEGIN</code></h2> - -<blockquote> - <pre> - #define H5_API_LOCK_BEGIN \ - if (H5_IS_API(FUNC)) { \ - H5_mutex_lock(&H5_g.init_lock); - </pre> -</blockquote> - - -<h2>E.5 <code>H5_API_LOCK_END</code></h2> - -<blockquote> - <pre> - #define H5_API_LOCK_END } - </pre> -</blockquote> - - -<h2>E.6 <code>HRETURN</code> and <code>HRETURN_ERROR</code></h2> - -<blockquote> - <pre> - : - : - H5_API_UNLOCK_BEGIN \ - H5_API_UNLOCK_END \ - H5_API_SET_CANCEL \ - return ret_val; \ - } - </pre> -</blockquote> - -<h2>E.7 <code>H5_API_UNLOCK_BEGIN</code></h2> - -<blockquote> - <pre> - #define H5_API_UNLOCK_BEGIN \ - if (H5_IS_API(FUNC)) { \ - H5_mutex_unlock(&H5_g.init_lock); - </pre> -</blockquote> - -<h2>E.8 <code>H5_API_UNLOCK_END</code></h2> - -<blockquote> - <pre> - #define H5_API_UNLOCK_END } - </pre> -</blockquote> - - -<h2>E.9 <code>H5_API_SET_CANCEL</code></h2> - -<blockquote> - <pre> - #define H5_API_SET_CANCEL \ - if (H5_IS_API(FUNC)) { \ - H5_cancel_count_dec(); \ - } - </pre> -</blockquote> - -<h2>By Chee Wai Lee</h2> -<h4>By Bill Wendling</h4> -<h4>27. October 2000</h4> - -</body> -</html> |