HDFFV-10578 and HDFFV-10676

Description:
    HDFFV-10578 - CVE-2018-17234
        The file has some issue, however, there was a bug in h5dump that caused
        memory leaks after the problem in the file was encountered. The bug
        was that an if statement was missing in the function table_list_add()
        resulting in the memory not being freed at a later time.
        After the fix had been applied, there were no more leaks after h5dump
        detected the issue in the file and reported the error.

        In H5O__chunk_deserialize, replaced an assert with an if statement
        and reporting error, per Neil's recommendation

    HDFFV-10676 - CVE-2018-13873
        Also in H5O__chunk_deserialize, added an assertion to detect
        out of bound ids

1 files changed, 4 insertions, 3 deletions

diff --git a/tools/src/h5dump/h5dump.c b/tools/src/h5dump/h5dump.c
index b9e37e8..5267188 100644
--- a/tools/src/h5dump/h5dump.c
+++ b/tools/src/h5dump/h5dump.c
@@ -407,9 +407,10 @@ table_list_add(hid_t oid, unsigned long file_no)
     }
     if(init_objs(oid, &info, &table_list.tables[idx].group_table,
                  &table_list.tables[idx].dset_table, &table_list.tables[idx].type_table) < 0) {
-        H5Idec_ref(oid);
-        table_list.nused--;
-        return -1;
+        if (H5Idec_ref(oid) < 0) {
+            table_list.nused--;
+            return -1;
+        }
     }
 
 #ifdef H5DUMP_DEBUG