[svn-r2508] Purpose:

    Bug fixes
Description:
    All tests were core=dumping in IRIX64.  The bug is in Generic
    property list creation in which malloc asked for 2*64-1 bytes
    due to coding bug.  The object creation failed but the return
    code was not checked.  Program eventually crashed.
Solution:
H5F.c:
    Check the return code from new file object creation and flag
    error accordingly.
H5FL.c:
    H5FL_arr_free is a replacement for H5MM_xfree which accepts
    null value as a legal argument value.  H5FL_arr_free assert
    on it.  Since other parts of the code have been passing null
    value to H5MM_xfree, H5FL_arr_free must accept it too until
    all the calling routines are changed to not pass Null.
H5P.c:
    some routine passes in 0 as the hashsize value which is uintn.
    The expression (hashsize-1) underflows to the largest unsigned
    int for some machines.  Thus the calloc failed.  Cast hashsize
    to unsigned int first (this assumes hashsize stays within the
    signed int data range.
H5Smpio.c:
    Added the extra parameter because the H5FD_write has been redefined.

Platforms tested:
    IRIX64 -64 and -n32

1 files changed, 4 insertions, 2 deletions

diff --git a/src/H5P.c b/src/H5P.c
index c60ad6c..1e985d6 100644
--- a/src/H5P.c
+++ b/src/H5P.c
@@ -3385,7 +3385,9 @@ H5P_create_class(H5P_genclass_t *par_class, const char *name, uintn hashsize, ui
     }
 
     /* Allocate room for the class & it's hash table of properties */
-    if (NULL==(pclass = H5MM_calloc (sizeof(H5P_genclass_t)+((hashsize-1)*sizeof(H5P_genprop_t *)))))
+    /* Need to (intn) hashsize so that (hashsize-1) can be negative since hashsize can be 0. */
+    if (NULL==(pclass = H5MM_calloc (sizeof(H5P_genclass_t)+
+		((((intn)hashsize)-1)*sizeof(H5P_genprop_t *)))))
         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL,"memory allocation failed");
 
     /* Set class state */
@@ -3540,7 +3542,7 @@ H5P_genplist_t *H5P_create_list(H5P_genclass_t *class)
      */
 
     /* Allocate room for the property list & it's hash table of properties */
-    if (NULL==(plist = H5MM_calloc (sizeof(H5P_genplist_t)+((class->hashsize-1)*sizeof(H5P_genprop_t *)))))
+    if (NULL==(plist = H5MM_calloc (sizeof(H5P_genplist_t)+(((intn)(class->hashsize)-1)*sizeof(H5P_genprop_t *)))))
         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL,"memory allocation failed");
 
     /* Set class state */