Onion VFD (#1953)

* Onion VFD feature

* Fixes onion VFD errors with non-sec2 backing store VFDs

* Disables the onion VFD tests w/ ph5diff

* Disables non-sec2 VFDs as onion VFD backing stores

* Committing clang-format changes

* Formatted source

* Typo

* Adds onion VFD tools tests to CMake

* Fixes for v16 API compatibility

* Memset structs to avoid bad frees on errors

* H5Dwrite() calls now use H5T_NATIVE_INT as the memory type vs LE

* Properly decodes checksums on BE machines

* Be more careful about uint64_t to haddr_t/hsize_t conversions

* Another fix for BE data comparison

* Removed double underscores from onion constants

* Replace hard-coded onion header string w/ constant

* Fixes cleanup paths in H5FD__onion_ingest_history()

* Fixed use of size_t revision numbers

* Fix h5dump revision count format string

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

10 files changed, 54 insertions, 0 deletions

diff --git a/tools/testfiles/tst_onion_dset_1d.ddl b/tools/testfiles/tst_onion_dset_1d.ddl
new file mode 100644
index 0000000..adca75f
--- /dev/null
+++ b/tools/testfiles/tst_onion_dset_1d.ddl
@@ -0,0 +1,12 @@
+Using revision 1
+HDF5 "tst_onion_dset_1d.h5" {
+GROUP "/" {
+   DATASET "DS1" {
+      DATATYPE  H5T_STD_I32LE
+      DATASPACE  SIMPLE { ( 1, 16 ) / ( 1, 16 ) }
+      DATA {
+      (0,0): 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+      }
+   }
+}
+}
diff --git a/tools/testfiles/tst_onion_dset_1d.h5 b/tools/testfiles/tst_onion_dset_1d.h5
new file mode 100644
index 0000000..5042ebf
--- /dev/null
+++ b/tools/testfiles/tst_onion_dset_1d.h5
diff --git a/tools/testfiles/tst_onion_dset_1d.h5.onion b/tools/testfiles/tst_onion_dset_1d.h5.onion
new file mode 100644
index 0000000..a86670e
--- /dev/null
+++ b/tools/testfiles/tst_onion_dset_1d.h5.onion
diff --git a/tools/testfiles/tst_onion_dset_ext.ddl b/tools/testfiles/tst_onion_dset_ext.ddl
new file mode 100644
index 0000000..2405774
--- /dev/null
+++ b/tools/testfiles/tst_onion_dset_ext.ddl
@@ -0,0 +1,19 @@
+Using revision 1
+HDF5 "tst_onion_dset_ext.h5" {
+GROUP "/" {
+   DATASET "DS1" {
+      DATATYPE  H5T_STD_I32LE
+      DATASPACE  SIMPLE { ( 8, 4 ) / ( H5S_UNLIMITED, H5S_UNLIMITED ) }
+      DATA {
+      (0,0): 0, 1, 2, 3,
+      (1,0): 1, 2, 3, 4,
+      (2,0): 2, 3, 4, 5,
+      (3,0): 3, 4, 5, 6,
+      (4,0): 0, 1, 2, 3,
+      (5,0): 1, 2, 3, 4,
+      (6,0): 2, 3, 4, 5,
+      (7,0): 3, 4, 5, 6
+      }
+   }
+}
+}
diff --git a/tools/testfiles/tst_onion_dset_ext.h5 b/tools/testfiles/tst_onion_dset_ext.h5
new file mode 100644
index 0000000..479aaa8
--- /dev/null
+++ b/tools/testfiles/tst_onion_dset_ext.h5
diff --git a/tools/testfiles/tst_onion_dset_ext.h5.onion b/tools/testfiles/tst_onion_dset_ext.h5.onion
new file mode 100644
index 0000000..b5cbdb4
--- /dev/null
+++ b/tools/testfiles/tst_onion_dset_ext.h5.onion
diff --git a/tools/testfiles/tst_onion_objs.ddl b/tools/testfiles/tst_onion_objs.ddl
new file mode 100644
index 0000000..c66275b
--- /dev/null
+++ b/tools/testfiles/tst_onion_objs.ddl
@@ -0,0 +1,22 @@
+Using revision 3
+HDF5 "tst_onion_objs.h5" {
+GROUP "/" {
+   ATTRIBUTE "file_attribute" {
+      DATATYPE  H5T_STD_I32LE
+      DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }
+      DATA {
+      (0): 0, 0, 0, 0
+      }
+   }
+   DATASET "DS1" {
+      DATATYPE  H5T_STD_I32LE
+      DATASPACE  SIMPLE { ( 4, 4 ) / ( H5S_UNLIMITED, H5S_UNLIMITED ) }
+      DATA {
+      (0,0): 0, 1, 2, 3,
+      (1,0): 1, 2, 3, 4,
+      (2,0): 2, 3, 4, 5,
+      (3,0): 3, 4, 5, 6
+      }
+   }
+}
+}
diff --git a/tools/testfiles/tst_onion_objs.h5 b/tools/testfiles/tst_onion_objs.h5
new file mode 100644
index 0000000..479aaa8
--- /dev/null
+++ b/tools/testfiles/tst_onion_objs.h5
diff --git a/tools/testfiles/tst_onion_objs.h5.onion b/tools/testfiles/tst_onion_objs.h5.onion
new file mode 100644
index 0000000..4abc174
--- /dev/null
+++ b/tools/testfiles/tst_onion_objs.h5.onion
diff --git a/tools/testfiles/tst_onion_revision_count.ddl b/tools/testfiles/tst_onion_revision_count.ddl
new file mode 100644
index 0000000..4f2e1db
--- /dev/null
+++ b/tools/testfiles/tst_onion_revision_count.ddl
@@ -0,0 +1 @@
+The number of revisions for the onion file is 6