Merge in VFD SWMR changes by John Mainzer.

Commit log message from John:
    Returned VFD SWMR to using the VFD SWMR reader VFD only in the reader case.
    In passing, added a private VFD SWMR reader VFD fapl entry that is pushed
    and popped off the FAPL during file open, and removed the code that set
    the VFD SWMR reader VFD as the driver in the FAPL when VFD SWMR is
    configured.  This was necessary, as there is no mechanism to prevent the
    user from overwriting this entry on the FAPL before file open.  While
    we don't use it now, it also gives us a mechanism for allowing the
    user to specify an underlying VFD for VFD SWMR.

    Modified code to compare file opens to compare the terminal VFDs, not
    the top level VFDs.  Failure to do this allowed multiple opens of
    the same file with the same VFD but with different pass through
    VFDs to appear to be treated as different files -- with the obvious
    file corruption issues.  To support this, added a new VFD ctl op code
    to return a pointer to the instance of H5FD_t associated with the
    terminal VFD.

    Note that this change does not address the case of the same file being
    opened twice with different terminal VFDs -- that will have to be addressed
    another day.

Overview of major changes from John:
1) Reworked file open so that the VFD SWMR reader VFD is only
   used when a file is opened VFD SWMR reader.  This required
   the following changes:
   a) Removed code to set the driver in H5Pset_vfd_swmr_config()
   b) Added a private fapl entry for the VFD SWMR reader VFD
   c) Modified H5F_open to test for VFD SWMR reader opens, and push the
      vfd swmr reader vfd FAPL entry on the VFD stack if so.  In this case
      the entry is popped off the VFD stack on exit so as to avoid any net
      modification from the supplied FAPL.
2) Removed dedup code, and augmented H5FD_cmp() to provide the
   necessary functionality.  This required the following changes:
   a) Added the get terminal VFD op code to the H5FD ctl call.
      This allows duplicate file opens with the same VFD but different overlying
      pass through VFDs to be recognized.  Updated ctl callback in VFDs as required
      to support the new op code.
   b) Modified H5FD_cmp to use the above ctl op code to allow it to recognize
      duplicate file opens with the same VFD but with different overlying
      passthoguh VFDs.  This is necessary to recognize duplicate VFD SWMR reader
      and regular opens.  Note that this does not allow us to recognize duplicate
      opens with different terminal VFDs.

1 files changed, 59 insertions, 1 deletions

diff --git a/src/H5FDdirect.c b/src/H5FDdirect.c
index a6139da..66843e9 100644
--- a/src/H5FDdirect.c
+++ b/src/H5FDdirect.c
@@ -140,6 +140,8 @@ static herr_t  H5FD__direct_truncate(H5FD_t *_file, hid_t dxpl_id, hbool_t closi
 static herr_t  H5FD__direct_lock(H5FD_t *_file, hbool_t rw);
 static herr_t  H5FD__direct_unlock(H5FD_t *_file);
 static herr_t  H5FD__direct_delete(const char *filename, hid_t fapl_id);
+static herr_t  H5FD__direct_ctl(H5FD_t *_file, uint64_t op_code, uint64_t flags, const void *input,
+                                void **output);
 
 static const H5FD_class_t H5FD_direct_g = {
     H5FD_DIRECT_VALUE,          /* value                */
@@ -175,7 +177,7 @@ static const H5FD_class_t H5FD_direct_g = {
     H5FD__direct_lock,          /* lock                 */
     H5FD__direct_unlock,        /* unlock               */
     H5FD__direct_delete,        /* del                  */
-    NULL,                       /* ctl                  */
+    H5FD__direct_ctl,           /* ctl                  */
     H5FD_FLMAP_DICHOTOMY        /* fl_map               */
 };
 
@@ -1421,4 +1423,60 @@ done:
     FUNC_LEAVE_NOAPI(ret_value)
 } /* end H5FD__direct_delete() */
 
+/*-------------------------------------------------------------------------
+ * Function:    H5FD__direct_ctl
+ *
+ * Purpose:     Direct VFD version of the ctl callback.
+ *
+ *              The desired operation is specified by the op_code
+ *              parameter.
+ *
+ *              The flags parameter controls management of op_codes that
+ *              are unknown to the callback
+ *
+ *              The input and output parameters allow op_code specific
+ *              input and output
+ *
+ *              At present, the only op code supported is
+ *              H5FD_CTL__GET_TERMINAL_VFD, which is used to obtain the
+ *              instance of H5FD_t associated with the terminal
+ *              VFD.  This allows comparison of files whose terminal
+ *              VFD may have overlying pass through VFDs.
+ *
+ * Return:      Non-negative on success/Negative on failure
+ *
+ * Changes:     None.
+ *
+ *-------------------------------------------------------------------------
+ */
+static herr_t
+H5FD__direct_ctl(H5FD_t *_file, uint64_t op_code, uint64_t flags, const void H5_ATTR_UNUSED *input,
+                 void **output)
+{
+    H5FD_direct_t *file      = (H5FD_direct_t *)_file;
+    herr_t         ret_value = SUCCEED;
+
+    FUNC_ENTER_PACKAGE
+
+    /* Sanity checks */
+    HDassert(file);
+
+    switch (op_code) {
+
+        case H5FD_CTL__GET_TERMINAL_VFD:
+            HDassert(output);
+            *output = (void *)(file);
+            break;
+
+        /* Unknown op code */
+        default:
+            if (flags & H5FD_CTL__FAIL_IF_UNKNOWN_FLAG)
+                HGOTO_ERROR(H5E_VFL, H5E_FCNTL, FAIL, "unknown op_code and fail if unknown flag is set")
+            break;
+    }
+
+done:
+    FUNC_LEAVE_NOAPI(ret_value)
+} /* end H5FD__direct_ctl() */
+
 #endif /* H5_HAVE_DIRECT */