VFD ctl feature (#981)

* Added "ctl" callback to the VFD interface, and the associated
H5FDctl() and H5FD_ctl() calls.

Modified the MPIO VFD accordingly -- specifically:

Added ctl() call with op-code support to expose rank, size,
and communicator.

Modified H5FD_mpi_get_rank(), H5FD_mpi_get_size(),
and H5FD_mpi_get_comm() to use the new ctl() callback.  In passing
removed the const qualifier from the file parameter of these
functions, as the file parameter of the ctl callback is not
const.

Deleted the old  H5FD__mpio_mpi_rank(), H5FD__mpio_mpi_size(), and
H5FD__mpio_communicator() calls from the MPIO VFD.

Deleted H5FD_class_mpi_t from H5FDprivate.h, and modified the
MPIO VFD accordingly.  Note that all VFDs now use H5FD_class_t,
with no special class for VFDs that that support MPI.

Some minor touch ups to the Neil's selection I/O mods in passing.

Tested serial and parallel, debug and production on charis and
jelly.

* Reserve a range of VFD "ctl" opcodes for library and experimental usage

* Add "ctl" callbacks to passthrough VFDs

* Add RELEASE.txt entry for "ctl" callback

* Use H5FDopen with H5F_ACC_RDWR flag instead of H5F_ACC_TRUNC in vfd test

* Remove handling of passthrough "ctl" flag from multi VFD

* Move logic for testing H5FD_CTL__TEST_OPCODE into a testing VFD

Revise description of "ctl" callback in RELEASE.txt

Remove unused H5FD_CTL__NUM_OPCODES definition

Fix some warnings in multi VFD

Co-authored-by: mainzer <mainzer#hdfgroup.org>

1 files changed, 25 insertions, 0 deletions

diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt
index ed12c5e..6043145 100644
--- a/release_docs/RELEASE.txt
+++ b/release_docs/RELEASE.txt
@@ -471,6 +471,31 @@ New Features
 
     Library:
     --------
+    - Adds new "ctl" callback to VFD H5FD_class_t structure
+      with the following prototype:
+
+        herr_t (*ctl)(H5FD_t *file, uint64_t op_code,
+                      uint64_t flags, const void *input,
+                      void **output);
+
+      This newly-added "ctl" callback allows Virtual File
+      Drivers to intercept and handle arbitary operations
+      identified by an operation code. Its parameters are
+      as follows:
+
+        `file` [in] - A pointer to the file to be operated on
+        `op_code` [in] - The operation code identifying the
+                         operation to be performed
+        `flags` [in] - Flags governing the behavior of the
+                       operation performed (see H5FDpublic.h
+                       for a list of valid flags)
+        `input` [in] - A pointer to arguments passed to the
+                       VFD performing the operation
+        `output` [out] - A pointer for the receiving VFD to
+                         use for output from the operation
+
+        (JRM - 2021/08/16)
+
     - Change how the release part of version, in major.minor.release is checked
       for compatibility