Fix for a segfault when H5Pset_fapl_log is passed an invalid fapl ID (#607)

* Committing clang-format changes

* Fixes an issue where H5Pset_fapl_log sefaults when passed an invalid
fapl ID

This was due to a pointer-containing struct being memset after the first
internal API call. If the first call failed, the error condition would
check if the pointer was not NULL and then attempt to free it if not.
This would lead to the freeing of a wild pointer if an invalid fapl ID
were passed in.

This was fixed by reordering the memset and adding a test to ensure the
problem stays fixed.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

1 files changed, 12 insertions, 1 deletions

diff --git a/test/vfd.c b/test/vfd.c
index 6b851a2..8484bfd 100644
--- a/test/vfd.c
+++ b/test/vfd.c
@@ -1867,12 +1867,23 @@ test_log(void)
     hsize_t       file_size = 0;
     unsigned int  flags     = H5FD_LOG_ALL;
     size_t        buf_size  = 4 * KB;
+    herr_t        ret       = SUCCEED;
 
     TESTING("LOG file driver");
 
-    /* Set property list and file name for log driver. */
     if ((fapl = H5Pcreate(H5P_FILE_ACCESS)) < 0)
         TEST_ERROR;
+
+    /* Make sure calling with an invalid fapl doesn't crash */
+    H5E_BEGIN_TRY
+    {
+        ret = H5Pset_fapl_log(H5I_INVALID_HID, LOG_FILENAME, 0, 0);
+    }
+    H5E_END_TRY;
+    if (SUCCEED == ret)
+        TEST_ERROR;
+
+    /* Set property list and file name for log driver. */
     if (H5Pset_fapl_log(fapl, LOG_FILENAME, flags, buf_size) < 0)
         TEST_ERROR;
     h5_fixname(FILENAME[6], fapl, filename, sizeof filename);