VFD ctl feature (#981)

* Added "ctl" callback to the VFD interface, and the associated
H5FDctl() and H5FD_ctl() calls.

Modified the MPIO VFD accordingly -- specifically:

Added ctl() call with op-code support to expose rank, size,
and communicator.

Modified H5FD_mpi_get_rank(), H5FD_mpi_get_size(),
and H5FD_mpi_get_comm() to use the new ctl() callback.  In passing
removed the const qualifier from the file parameter of these
functions, as the file parameter of the ctl callback is not
const.

Deleted the old  H5FD__mpio_mpi_rank(), H5FD__mpio_mpi_size(), and
H5FD__mpio_communicator() calls from the MPIO VFD.

Deleted H5FD_class_mpi_t from H5FDprivate.h, and modified the
MPIO VFD accordingly.  Note that all VFDs now use H5FD_class_t,
with no special class for VFDs that that support MPI.

Some minor touch ups to the Neil's selection I/O mods in passing.

Tested serial and parallel, debug and production on charis and
jelly.

* Reserve a range of VFD "ctl" opcodes for library and experimental usage

* Add "ctl" callbacks to passthrough VFDs

* Add RELEASE.txt entry for "ctl" callback

* Use H5FDopen with H5F_ACC_RDWR flag instead of H5F_ACC_TRUNC in vfd test

* Remove handling of passthrough "ctl" flag from multi VFD

* Move logic for testing H5FD_CTL__TEST_OPCODE into a testing VFD

Revise description of "ctl" callback in RELEASE.txt

Remove unused H5FD_CTL__NUM_OPCODES definition

Fix some warnings in multi VFD

Co-authored-by: mainzer <mainzer#hdfgroup.org>

1 files changed, 105 insertions, 0 deletions

diff --git a/src/H5FD.c b/src/H5FD.c
index abda921..d3f72e8 100644
--- a/src/H5FD.c
+++ b/src/H5FD.c
@@ -1688,6 +1688,111 @@ done:
 } /* end H5FD_unlock() */
 
 /*-------------------------------------------------------------------------
+ * Function:    H5FDctl
+ *
+ * Purpose:     Perform a CTL operation.
+ *
+ *              The desired operation is specified by the op_code
+ *              parameter.
+ *
+ *              The flags parameter controls management of op_codes that
+ *              are unknown to the callback
+ *
+ *              The input and output parameters allow op_code specific
+ *              input and output
+ *
+ * Return:      Non-negative on success/Negative on failure
+ *
+ * Programmer:  JRM -- 8/3/21
+ *
+ *-------------------------------------------------------------------------
+ */
+herr_t
+H5FDctl(H5FD_t *file, uint64_t op_code, uint64_t flags, const void *input, void **output)
+{
+    herr_t ret_value = SUCCEED; /* Return value */
+
+    FUNC_ENTER_API(FAIL)
+    H5TRACE5("e", "*#ULUL*x**x", file, op_code, flags, input, output);
+
+    /* Check arguments */
+    if (!file)
+        HGOTO_ERROR(H5E_ARGS, H5E_BADVALUE, FAIL, "file pointer cannot be NULL")
+
+    if (!file->cls)
+        HGOTO_ERROR(H5E_ARGS, H5E_BADVALUE, FAIL, "file class pointer cannot be NULL")
+
+    /* Don't attempt to validate the op code.  If appropriate, that will
+     * be done by the underlying VFD callback, along with the input and
+     * output parameters.
+     */
+
+    /* Call private function */
+    if (H5FD_ctl(file, op_code, flags, input, output) < 0)
+        HGOTO_ERROR(H5E_VFL, H5E_FCNTL, FAIL, "VFD ctl request failed")
+
+done:
+
+    FUNC_LEAVE_API(ret_value)
+
+} /* end H5FDctl() */
+
+/*-------------------------------------------------------------------------
+ * Function:    H5FD_ctl
+ *
+ * Purpose:     Private version of H5FDctl()
+ *
+ *              The desired operation is specified by the op_code
+ *              parameter.
+ *
+ *              The flags parameter controls management of op_codes that
+ *              are unknown to the callback
+ *
+ *              The input and output parameters allow op_code specific
+ *              input and output
+ *
+ * Return:      Non-negative on success/Negative on failure
+ *
+ * Programmer:  JRM -- 8/3/21
+ *
+ *-------------------------------------------------------------------------
+ */
+herr_t
+H5FD_ctl(H5FD_t *file, uint64_t op_code, uint64_t flags, const void *input, void **output)
+{
+    herr_t ret_value = SUCCEED; /* Return value */
+
+    FUNC_ENTER_NOAPI(FAIL)
+
+    /* Sanity checks */
+    HDassert(file);
+    HDassert(file->cls);
+
+    /* Dispatch to driver if the ctl function exists.
+     *
+     * If it doesn't, fail if the H5FD_CTL__FAIL_IF_UNKNOWN_FLAG is set.
+     *
+     * Otherwise, report success.
+     */
+    if (file->cls->ctl) {
+
+        if ((file->cls->ctl)(file, op_code, flags, input, output) < 0)
+
+            HGOTO_ERROR(H5E_VFL, H5E_FCNTL, FAIL, "VFD ctl request failed")
+    }
+    else if (flags & H5FD_CTL__FAIL_IF_UNKNOWN_FLAG) {
+
+        HGOTO_ERROR(H5E_VFL, H5E_FCNTL, FAIL,
+                    "VFD ctl request failed (no ctl callback and fail if unknown flag is set)")
+    }
+
+done:
+
+    FUNC_LEAVE_NOAPI(ret_value)
+
+} /* end H5FD_ctl() */
+
+/*-------------------------------------------------------------------------
  * Function:    H5FD_get_fileno
  *
  * Purpose:     Quick and dirty routine to retrieve the file's 'fileno' value