diff options
| author | Larry Knox <lrknox@hdfgroup.org> | 2021-02-08 16:56:16 (GMT) |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-02-08 16:56:16 (GMT) |
| commit | 22c67ff6570f16050f0a5654cf1c84df917bb598 (patch) | |
| tree | 38fb63aa498b250c2543fe7e22a511706de96a42 /test/ros3.c | |
| parent | 551f15f8ae02ca9c995619b216121081eb07633e (diff) | |
| download | hdf5-22c67ff6570f16050f0a5654cf1c84df917bb598.zip hdf5-22c67ff6570f16050f0a5654cf1c84df917bb598.tar.gz hdf5-22c67ff6570f16050f0a5654cf1c84df917bb598.tar.bz2 | |
1.8/master (#317)hdf5-1_8_22
* Revert incorrect bool type
* Correct grep text for filter test
* Check sizeof type to use correct ref_type
* h5dump reg ref chnages has no error file
* Fix comment
* Remove unneeded files
* Verify the decoded version for "all" and "none" selection.
* Remove double entry
* Add missing include
* Snapshot version 1.8 release 22 (snap4)
Update version to 1.8-snap5
* Fixes for test failures with --disable-tests --disable-tools options.
Don't test ph5diff if tests are disabled.
* Correct conditions for parallel builds with and without tests.
* Snapshot version 1.8 release 22 (snap5)
Update version to 1.8.22-snap6
* TRILAB-192 merging warnings changes from develop
* Autotools reconfigure update
* Restore page ejects.
* Restore page eject chars
* Fix orphaned token
* Update LT files
* Correct extra flags and merge TRILAB-24
* Update release note for CMake warnings
* H5repack bug fixes from develop
* TRILAB-192 - merge changes from develop
Single source, config files, for warnings for both autotools and CMake.
Update CMake libraries, tools, tests to use correct flags.
* Add missing m4 file
* Remove 128bit config check fo 1.8
* TRILAB-192 update LTparse files
* A fix in the cleaning up code for datatype when datatype initialization via H5D__init_type() fails.
The code to fix the problem is the same as what is done in H5D__open_oid().
* Add missing " and update function calls incorrectly
named load_gcc_arguments.
Commit changes to gnu-cxxflags to remove unmatched " and to gnu-fflags
to not add C warnings flags to H5_FCFLAGS.
* TRILAB-192 - cleanup edits and match CMake CXX to autotools
* Fix shadowed type
* TRILAB-192 correct fortran commands
* TRILAB-192 version of std=f2008 special check
* Fix shadow variable
* Add prefix_relto_examplesdir function to set relative path to h5c++.
* TRILAB-192 - parser files cannot handle this warning
* Parallel Fortran tests now use the MPI module instead of including
mpif.h.
* Don't add general warnings flags for unsupported old versions of gcc and
g++ (older than gcc/g++ 4.2).
Correct gnu-cxxflags to determine warnings flags to be added based on
C++ compiler version instead of C compiler version.
* Snapshot version 1.8 release 22 (snap6)
Update version to 1.8.22-snap7
* TRILAB-244 separate CXX warnings and errors from C
* Fix NoFilter build
* Move ADD_H5_FLAGS macro from *CompilerFlags.cmake files to
config/cmake_ext_mod/HDFMacros.cmake for common use.
* Restrict errors to gcc 4.8 and above.
* OESS-65 replace szip with aec szip
* Correct CMake version
* Correct -Werror=unused-variable
* Add -Werror= flags to lists displayed in libhdf5.settings.
* Fix -Werror=unused-variable
* More fixes for unused variable
* Resolve warning errors from -Werror= flags:
-Werror=bad-function-cast
-Werror=bad-function-cast
-Werror=implicit-function-declaration
-Werror=nested-externs
* Remove verbose assignments in smoke_check 3 and 5 functions.
* Update to use optimization flags, C++ error flags, and correct
libhdf5.settings.in to remove unworkable changes.
* Update config/gnu-cxxflags.
* Add missing headers
* Fix ifort warning flag.
Restore AC_DEFUN([PAC_PROG_FC_DEFAULT_REALisDBLE] in m4/aclocal_fc.m4.
Add --enable-symbols, --enable-profiling, and --enable-optimization
options to autotools configure; all configure flags for
--enable-production and --disable-production should match HDF5 1.10 and
1.12 production and debug default flags.
* Add github actions file
* Correct path
* OESS-65 Fix packaging
* Update reference name
* Correct filename
* disable pr action and enable fail fast
* Snapshot 1.8.22-snap7
Update version to 1.8.22-snap8.
* Correct mingw path
* Correct mingw paths
* Revise gnu-*flags and cmake/HDF*CompilerFlags.cmake files to add warning
flags for GCC compilers version 4.8 and above.
Removed files from gnu-warnings that only apply to versions < 4.8.
Consolidated warnings from versions < 4.8 that apply to versions >= 4.8
into the 4.8 warnings files.
* Update MANIFEST for removal of older warnings files.
* Yanked all MPI-1 calls
Added batch scripts in bin/batch.
* Snapshot 1.8.22-snap8.
Update version to 1.8.22-snap9.
* Squashed commit of the following:
commit 45b0cbc71929930a874e1c09e0770aa64d0ae697
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Thu Apr 23 13:14:56 2020 -0500
Add C++ warnings treated as error for autotools builds.
commit 6debcddcc766601799d5bd661bd60946f951a07e
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Fri Jun 19 16:50:03 2020 -0500
Remove tests for develop branch from testh5cc.sh.in.
Add @H5_ECXXFLAGS@ to AM_CXXFLAGS.
commit fed497e7f207313b2a133d3b68c942b7d7104b90
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Fri Jun 12 15:06:39 2020 -0500
Merge pull request #2646 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit '998c6470d78e14228695419c3b514c35ecf3d69e':
Remove unnecessary version conditions for Clang compilers.
commit 92e52ce5c211bd1d3991a3b8bb67287ac7b652aa
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Wed Jun 10 18:53:45 2020 -0500
Merge pull request #2639 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit 'a33667faf226f5d8c9633bf537893e8fce1bf1f6':
Add c++ to --enable-sanitize-checks option.
commit 2e802534fb997b81fa98fdd1c7d97d4310898e0d
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Wed Jun 10 15:18:36 2020 -0500
Merge pull request #2633 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit '20eab963899841c9a003baebead8e3bc07d9e127':
Remove duplicate entries in MANIFEST.
Make changes to CMake CompilerFlags.cmake files so extra flags are loaded for non-GNU compilers.
Update new clang files to not pick up clang as vendor for pgCC. Add new files to MANIFEST Temporary demotion of 2 -Werror warning flags that fail on macos 10.12 Remove Production flag unknown to Apple clang.
commit 96ef60a58a23847a7da89a891f2415055ec2ab60
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Mon Jun 8 16:24:49 2020 -0500
Merge pull request #2631 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit 'b942a4d8a3e27317cac50ce37ff5302d477664d8':
Clean up code to get clang version in config/linux-gnulibc1
commit 8a7c687da568e8b50b38fa53da1ca63759be6ec4
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Mon Jun 8 11:42:37 2020 -0500
Merge pull request #2623 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit 'c7626f79fdee4eee13925e240ef41e61f3367eab':
Add flags from config/clang-warnings/*general files to H5 C and CXX flags for all versions of Clang and Clang++ compilers. Switched from cut to awk in testcheck_version.sh.in to avoid dependence on tab vs. " " in version definitions in H5public.h.
Add files for adding warning flags for clang compilers in autotools configure.
commit db3ef7ff3c1ed79167cecef831501411cff8291f
Merge: 5a0f8d7 f9f1310
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Fri Jun 19 14:46:22 2020 -0500
Merge branch 'hdf5_1_12' of https://bitbucket.hdfgroup.org/scm/~lrknox/hdf5_lrk into hdf5_1_12
commit 5a0f8d7940ae57b445f545a0abd7e303ce6924ee
Author: Larry Knox <lrknox@hdfgroup.org>
Date: Wed Jun 10 20:15:41 2020 -0500
Merge pull request #2636 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:HDFFV-11000-update-testh5cc.sh.in-for-version to develop
* commit '5c0bd670d6e9919c463fbc05de99e2ba55b0761e':
Add tests for all version to H5_NO_DEPRECATED_SYMBOLS section and to section for current version, with and without default API version flags.
HDFFV-11000: update-testh5cc.sh.in to test sample versioned functions in HDF5 1.10, 1.12 and develop.
* Update configure and Makefile.ins after bin/reconfigure.
* Fix MANIFEST
* Fix HDFFV-10591
Description:
h52gif produced a segfault when a buffer overflow occurred because
the data size was corrupted and became very large. This commit added
a check on the data size against the buffer size to prevent the segfault.
It also added error reporting to h52gif to display an error message
instead of silently exiting when the failure occurred.
Platforms tested:
Linux/64 (jelly)
* Snapshot 1.8.22-snap9. Update version to 1.8.22-snap10.
* Merge pull request #2686 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit '800f93f7c994f88dfa716746153ded4b1e690e3a':
Remove path to szip header file from AM_CPPFLAGS when configure check of libsz fails. Fix for HDFFV-10830.
* Regenerate configure.
* Merge from 1.10-1.12-dev - LT parse needs update
* Incorporate pull request #2693 from develop for the fix to HDFFV-11080:
(1) Patch up the file pointer when reading attribute of variable length datatype
(2) Test to verify the fix when doing multiple threads
(3) Update MANIFEST
(4) Add new test to Cmake
* Switch bison and flex update of LT files from bin/genltanalyse to
bin/genparser, and from automatically updating the LT files whenever
reconfigure is run to running only on man<F12>ual command.
* Update hl/sr/H5LT files.
Add bin/genparser to MANIFEST; remove bin/genltanalyze.
* Fix to the merge of PR #2708 for HDFFV-11080 to the 1.8 branch.
Need to checkin test/Makefile.in for the new file added.
* Fix HDFFV-11120 and HDFFV-11121 (CVE-2018-13870 and CVE-2018-13869)
Description:
When a buffer overflow occurred because a name length was corrupted
and became very large, h5dump produced a segfault on one file and a
memcpy parameter overlap on another file. This commit added checks
that detect a read pass the end of the buffer to prevent these error
conditions.
Platforms tested:
Linux/64 (jelly)
* Fixed typo
* Check for header szlib.h only when libsz passes AC_CHECK_LIB, so that
H5_HAVE_SALIB_H is not defined when szip is not enabled, to prevent
compile failures for incompatible szlib.
* HDFFV-11127 - force RTLD_LOCAL in dlopen
* spelling
* Remove extra parens
* Remove unnecessary assignment freom configure.ac.
* Add hypen to subrelease regex
* Add special case when H5_VERS_SUBRELEASE is 0
* Revert H5_VERS_SUBRELEASE check
* The version string has a second use for packaging
* MinGW uses "MinGW Makefiles"
* Snapshot 1.8.22-snap10. Update version to 1.8.22-snap11.
* Merge clang-format changes from develop
* remove stray file
* Source formatted
* Add options to github actions.
* Add back file, remove py file
* HDFFV-11096 Correct h5dump usage text
* Update formating
* remove autogen for 1.8 branch
* Fix comment whitespace
* Change '==' to 'eq' in test script
* Merge from develop
github updates
bin scripts
parser updates
* Snapshot 1.8.22-snap11. Update version to 1.8.22-snap12.
* Merge from 1.10
Comments, whitespace
Simple init and if block brackets.
Minimal code changes limited to return value and spelling
* Update autotools files
* Update FD source
* More merges from 1.10 comments and inits
* Comment fixes
* Fix macro argument
* Remove autogen file and cache dir
* Revert autotools files to previous version
* Correct boolean value and missing define
* URL and options corrections
* revert autoconf files
* Fix compile issues with vfd drivers.
* Fix HDFS test
* Remove extra #endif
* Move init above execution
* Multiple changes to bring branch up-to-date (#87)
Correct TARGET variable and CMake config file location.
Add option to allow filter plugins to be built inline.
Update CMake tools macros.
* hdf5 1 8 release text (#113)
* Multiple changes to bring branch up-to-date
Correct TARGET variable and CMake config file location.
Add option to allow filter plugins to be built inline.
Update CMake tools macros.
* release text updates - mostly whitespace
* Fixed HDFFV-10480 and HDFFV-11159 (#145)
* Fixed HDFFV-10480 and HDFFV-11159
Description
Checked against buffer size to prevent segfault, in case of data corruption.
+ HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode
+ HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode and
A user's patch was applied to this previously, but it is redone
for a more correct fix, that is the check now accounted for the
previous advance of the buffer pointer.
Platforms tested:
Linux/64 (jelly)
* Fixed typo
* Update HDF5 1.8 copyright headers with updated copyright URL (#148)
* Update URL for COPYING file in copyright headers.
* Make fortran copyright headers uniformly spaced.
* Update url for Copyright file in c++/src/footer.html.
* OESS-98 Update plugin build option (#171)
* 1.8 has extra fortran CMake option (#176)
* 1.8 has extra fortran CMake option
* Update license URL
* OESS-98 fix tools test for plugins (#180)
* Hdf5 1 8 (#169)
* Fixed HDFFV-10480 and HDFFV-11159
Description
Checked against buffer size to prevent segfault, in case of data corruption.
+ HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode
+ HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode and
A user's patch was applied to this previously, but it is redone
for a more correct fix, that is the check now accounted for the
previous advance of the buffer pointer.
Platforms tested:
Linux/64 (jelly)
* Fixed typo
* Fixed HDFFV-11150
Description
Replaced an HDassert with a check for null pointer in H5O_dec_rc() to
catch null pointer in corrupted data situation.
Reversed the fix in svn-r24463 in which a check for null pointer prior
to calling H5O_dec_rc()
Platforms tested:
Linux/64 (jelly)
* HDF5 1 8 fix fortran build on macs (#186)
* Correct fortran and shared libs option
* Fix for no shared fortran build
* OESS-98 fix hdf5 link target (#191)
* Partial changes to RELEASE.txt for release. (#185)
* Partial changes to RELEASE.txt for release.
* Update supported and tested platforms.
* Update version to 1.8.22-12.
* close #195. (#196)
* Update HDF5PluginMacros.cmake
* Update HDF5PluginMacros.cmake
* Update directory for SZ filter in HDF5PluginMacros.cmake.
Updates for release:
Switch configure default to production mode.
Set HDF5_GENERATE_HEADERS to OFF.
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Restores maintainer mode in the autotools (#200) (#203)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
* Hdf5 1 8 22 (#212)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Brings ttsafe_attr_vlen.c changes from develop (#214)
Fixes exposed pthread problem on Windows.
* Update SO numbers for Hdf5 1 8 22 (#215)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Hdf5 1 8 22 (#224)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
* Update CMake/HDF5Examples version in bin/release
* Update CMake/HDF5Examples version number in bin/release (#225)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
* Fixed typo in an error message. (#228)
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* Bring 3 small changes from Hdf5 1.8 to 1.8.22 (#241)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Stat st blocks fix 1822 (#251)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* Fixes Autotools detection of the st_blocks field in stat (#246)
* Fixes Autotools detection of the st_blocks field in stat
The Autotools and CMake will now both correctly determine if the
stat struct has the st_blocks field and set H5_HAVE_STAT_ST_BLOCKS
appropriately.
* Fixes a typo in configure.ac
* Restore lines in RELEASE.txt.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Stat st blocks fix 1822 (#256)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* Fixes Autotools detection of the st_blocks field in stat (#246)
* Fixes Autotools detection of the st_blocks field in stat
The Autotools and CMake will now both correctly determine if the
stat struct has the st_blocks field and set H5_HAVE_STAT_ST_BLOCKS
appropriately.
* Fixes a typo in configure.ac
* Restore lines in RELEASE.txt.
* Updated configure with reconfigure.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* RELEASE.txt cleanup.
* Hdf5 1 8 22 (#261)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* RELEASE.txt cleanup.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Add macOS Big Sur to tested machines, also missing entries for macOS 10.13 and 10.14.
* )Update version.
* Hdf5 1 8 22 (#266)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* RELEASE.txt cleanup.
* Add macOS Big Sur to tested machines, also missing entries for macOS 10.13 and 10.14.
* )Update version.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Reverts lock/unlock callback signature to 1.8.21 version (#254)
* Reverts lock/unlock callback signature to 1.8.21 version
This callback is unused in 1.8. The ros3 and hdfs VFDs are the only VFDs
that have the lock callback implemented and that is just as no-op stubs.
These stubs were removed so the callbacks are now NULL pointers, like
the other VFDs in 1.8.
* Trivial whitespace fix
* Update version to 1.8.22-14.
* Hdf5 1 8 22 - Reverts lock/unlock callback signature to 1.8.21 version (#267)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* RELEASE.txt cleanup.
* Add macOS Big Sur to tested machines, also missing entries for macOS 10.13 and 10.14.
* )Update version.
* Reverts lock/unlock callback signature to 1.8.21 version (#254)
* Reverts lock/unlock callback signature to 1.8.21 version
This callback is unused in 1.8. The ros3 and hdfs VFDs are the only VFDs
that have the lock callback implemented and that is just as no-op stubs.
These stubs were removed so the callbacks are now NULL pointers, like
the other VFDs in 1.8.
* Trivial whitespace fix
* Update version to 1.8.22-14.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Update version in H5public.h
* Hdf5 1 8 22 (#269)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* RELEASE.txt cleanup.
* Add macOS Big Sur to tested machines, also missing entries for macOS 10.13 and 10.14.
* )Update version.
* Reverts lock/unlock callback signature to 1.8.21 version (#254)
* Reverts lock/unlock callback signature to 1.8.21 version
This callback is unused in 1.8. The ros3 and hdfs VFDs are the only VFDs
that have the lock callback implemented and that is just as no-op stubs.
These stubs were removed so the callbacks are now NULL pointers, like
the other VFDs in 1.8.
* Trivial whitespace fix
* Update version to 1.8.22-14.
* Update version in H5public.h
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Set version 1.8.22 for release.
* dd RELEASE.txt entry for HDFFV-10741.
* Hdf5 1 8 22 (#279)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* RELEASE.txt cleanup.
* Add macOS Big Sur to tested machines, also missing entries for macOS 10.13 and 10.14.
* )Update version.
* Reverts lock/unlock callback signature to 1.8.21 version (#254)
* Reverts lock/unlock callback signature to 1.8.21 version
This callback is unused in 1.8. The ros3 and hdfs VFDs are the only VFDs
that have the lock callback implemented and that is just as no-op stubs.
These stubs were removed so the callbacks are now NULL pointers, like
the other VFDs in 1.8.
* Trivial whitespace fix
* Update version to 1.8.22-14.
* Update version in H5public.h
* Set version 1.8.22 for release.
* dd RELEASE.txt entry for HDFFV-10741.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
* Improve performance of multiple calls to H5Sget_select_elem_pointlist (#270) (#277)
* Cache the pointer to the next point to process after the last call to
H5S__get_select_elem_pointlist. This allows the normal process of
iterating over the points in batches to be much more efficient, as the
library does not need to traverse the entirety of the preceding points
every time the funciton is re-entered.
* Update RELEASE.txt for point selection iteration performance fix.
* Hdf5 1 8 22 (#281)
* Restores maintainer mode in the autotools (#200)
Maintainer mode should be enabled in development branches.
Also adds helpful commenting.
Add bin/switch_maint_mode
Disable maintainer mode for release.
Fix incomplete merge for stub functions in H5Fdhdfs.c
* Update configure for Restores maintainer mode in the autotools (#200).
* Update MANIFEST for switch_maint_mode script.
* Update so numbers for 1.8.22 release.
* Add so numbers changes in Makefile.ins for 1.8.22 release.
* Update pkgconfig settings with version - #218 (#223)
* Add notice of final HDFF5 1.8 release.
Add solaris 64bit alignment issue to "Known Problems".
* Update 1.8 final release notice.
* Update CMake/HDF5Examples version in bin/release
* Fixed typo in an error message. (#227)
* Remove duplicate setting (#239)
* RELEASE.txt cleanup.
* Add macOS Big Sur to tested machines, also missing entries for macOS 10.13 and 10.14.
* )Update version.
* Reverts lock/unlock callback signature to 1.8.21 version (#254)
* Reverts lock/unlock callback signature to 1.8.21 version
This callback is unused in 1.8. The ros3 and hdfs VFDs are the only VFDs
that have the lock callback implemented and that is just as no-op stubs.
These stubs were removed so the callbacks are now NULL pointers, like
the other VFDs in 1.8.
* Trivial whitespace fix
* Update version to 1.8.22-14.
* Update version in H5public.h
* Set version 1.8.22 for release.
* dd RELEASE.txt entry for HDFFV-10741.
* Improve performance of multiple calls to H5Sget_select_elem_pointlist (#270) (#277)
* Cache the pointer to the next point to process after the last call to
H5S__get_select_elem_pointlist. This allows the normal process of
iterating over the points in batches to be much more efficient, as the
library does not need to traverse the entirety of the preceding points
every time the funciton is re-entered.
* Update RELEASE.txt for point selection iteration performance fix.
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
Co-authored-by: Neil Fortner <nfortne2@hdfgroup.org>
* Hdf5 1 8 22 (#284)
* Fixed typo in an error message.
* Updated for HDFFV-11150, HDFFV-10480, and HDFFV-11159
* Update "Support for New Platforms and Compilers" section in RELEASE.txt;
add check_version workaround for binary compatibility to "Known
Problems".
* Add SUSE Linux to tested platforms.
* Update numbers in config/lt_vers.am and run bin/reconfigure for so
numbers.
* Update version in 3 files missed by merge.
Co-authored-by: Allen Byrne <byrn@hdfgroup.org>
Co-authored-by: Vailin Choi <vchoi@hdfgroup.org>
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: hdftest <hdftest@hdfgroup.org>
Co-authored-by: Jordan Henderson <jhenderson@hdfgroup.org>
Co-authored-by: Dana Robinson <derobins@hdfgroup.org>
Co-authored-by: Binh-Minh Ribler <bmribler@hdfgroup.org>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: Neil Fortner <nfortne2@hdfgroup.org>
Diffstat (limited to 'test/ros3.c')
| -rw-r--r-- | test/ros3.c | 1818 |
1 files changed, 1818 insertions, 0 deletions
diff --git a/test/ros3.c b/test/ros3.c new file mode 100644 index 0000000..cd723c2 --- /dev/null +++ b/test/ros3.c @@ -0,0 +1,1818 @@ +/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * + * Copyright by The HDF Group. * + * All rights reserved. * + * * + * This file is part of HDF5. The full HDF5 copyright notice, including * + * terms governing use, modification, and redistribution, is contained in * + * the COPYING file, which can be found at the root of the source code * + * distribution tree, or in https://www.hdfgroup.org/licenses. * + * If you do not have access to either file, you may request a copy from * + * help@hdfgroup.org. * + * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ + +/* + * Read-Only S3 Virtual File Driver (VFD) + * + * Purpose: + * + * Verify behavior for Read-Only S3 VFD + * at the VFL (virtual file layer) level. + * + * Demonstrates basic use cases and fapl/dxpl interaction. + * + * Programmer: Jacob Smith + * 2017-10-11 + */ + +#include "h5test.h" + +#include "H5FDprivate.h" /* Virtual File Driver utilities */ +#include "H5FDros3.h" /* this file driver's utilities */ +#include "H5FDs3comms.h" /* for loading of credentials */ + +#ifdef H5_HAVE_ROS3_VFD + +/* only include the testing macros if needed */ + +/***************************************************************************** + * + * FILE-LOCAL TESTING MACROS + * + * Purpose: + * + * 1) Upon test failure, goto-jump to single-location teardown in test + * function. E.g., `error:` (consistency with HDF corpus) or + * `failed:` (reflects purpose). + * >>> using "error", in part because `H5E_BEGIN_TRY` expects it. + * 2) Increase clarity and reduce overhead found with `TEST_ERROR`. + * e.g., "if(somefunction(arg, arg2) < 0) TEST_ERROR:" + * requires reading of entire line to know whether this if/call is + * part of the test setup, test operation, or a test unto itself. + * 3) Provide testing macros with optional user-supplied failure message; + * if not supplied (NULL), generate comparison output in the spirit of + * test-driven development. E.g., "expected 5 but was -3" + * User messages clarify test's purpose in code, encouraging description + * without relying on comments. + * 4) Configurable expected-actual order in generated comparison strings. + * Some prefer `VERIFY(expected, actual)`, others + * `VERIFY(actual, expected)`. Provide preprocessor ifdef switch + * to satifsy both parties, assuming one paradigm per test file. + * (One could #undef and redefine the flag through the file as desired, + * but _why_.) + * + * Provided as courtesy, per consideration for inclusion in the library + * proper. + * + * Macros: + * + * JSVERIFY_EXP_ACT - ifdef flag, configures comparison order + * FAIL_IF() - check condition + * FAIL_UNLESS() - check _not_ condition + * JSVERIFY() - long-int equality check; prints reason/comparison + * JSVERIFY_NOT() - long-int inequality check; prints + * JSVERIFY_STR() - string equality check; prints + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *****************************************************************************/ + +/*---------------------------------------------------------------------------- + * + * ifdef flag: JSVERIFY_EXP_ACT + * + * JSVERIFY macros accept arguments as (EXPECTED, ACTUAL[, reason]) + * default, if this is undefined, is (ACTUAL, EXPECTED[, reason]) + * + *---------------------------------------------------------------------------- + */ +#define JSVERIFY_EXP_ACT 1L + +/*---------------------------------------------------------------------------- + * + * Macro: JSFAILED_AT() + * + * Purpose: + * + * Preface a test failure by printing "*FAILED*" and location to stdout + * Similar to `H5_FAILED(); AT();` from h5test.h + * + * *FAILED* at somefile.c:12 in function_name()... + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#define JSFAILED_AT() \ + { \ + HDprintf("*FAILED* at %s:%d in %s()...\n", __FILE__, __LINE__, FUNC); \ + } + +/*---------------------------------------------------------------------------- + * + * Macro: FAIL_IF() + * + * Purpose: + * + * Make tests more accessible and less cluttered than + * `if (thing == otherthing()) TEST_ERROR` + * paradigm. + * + * The following lines are roughly equivalent: + * + * `if (myfunc() < 0) TEST_ERROR;` (as seen elsewhere in HDF tests) + * `FAIL_IF(myfunc() < 0)` + * + * Prints a generic "FAILED AT" line to stdout and jumps to `error`, + * similar to `TEST_ERROR` in h5test.h + * + * Programmer: Jacob Smith + * 2017-10-23 + * + *---------------------------------------------------------------------------- + */ +#define FAIL_IF(condition) \ + if (condition) { \ + JSFAILED_AT() \ + goto error; \ + } + +/*---------------------------------------------------------------------------- + * + * Macro: FAIL_UNLESS() + * + * Purpose: + * + * TEST_ERROR wrapper to reduce cognitive overhead from "negative tests", + * e.g., "a != b". + * + * Opposite of FAIL_IF; fails if the given condition is _not_ true. + * + * `FAIL_IF( 5 != my_op() )` + * is equivalent to + * `FAIL_UNLESS( 5 == my_op() )` + * However, `JSVERIFY(5, my_op(), "bad return")` may be even clearer. + * (see JSVERIFY) + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#if 0 /* UNUSED */ +#define FAIL_UNLESS(condition) \ + if (!(condition)) { \ + JSFAILED_AT() \ + goto error; \ + } +#endif + +/*---------------------------------------------------------------------------- + * + * Macro: JSERR_LONG() + * + * Purpose: + * + * Print an failure message for long-int arguments. + * ERROR-AT printed first. + * If `reason` is given, it is printed on own line and newlined after + * else, prints "expected/actual" aligned on own lines. + * + * *FAILED* at myfile.c:488 in somefunc()... + * forest must be made of trees. + * + * or + * + * *FAILED* at myfile.c:488 in somefunc()... + * ! Expected 425 + * ! Actual 3 + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#define JSERR_LONG(expected, actual, reason) \ + { \ + JSFAILED_AT() \ + if (reason != NULL) { \ + HDprintf("%s\n", (reason)); \ + } \ + else { \ + HDprintf(" ! Expected %ld\n ! Actual %ld\n", (long)(expected), (long)(actual)); \ + } \ + } + +/*---------------------------------------------------------------------------- + * + * Macro: JSERR_STR() + * + * Purpose: + * + * Print an failure message for string arguments. + * ERROR-AT printed first. + * If `reason` is given, it is printed on own line and newlined after + * else, prints "expected/actual" aligned on own lines. + * + * *FAILED* at myfile.c:421 in myfunc()... + * Blue and Red strings don't match! + * + * or + * + * *FAILED* at myfile.c:421 in myfunc()... + * !!! Expected: + * this is my expected + * string + * !!! Actual: + * not what I expected at all + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#define JSERR_STR(expected, actual, reason) \ + { \ + JSFAILED_AT() \ + if ((reason) != NULL) { \ + HDprintf("%s\n", (reason)); \ + } \ + else { \ + HDprintf("!!! Expected:\n%s\n!!!Actual:\n%s\n", (expected), (actual)); \ + } \ + } + +#ifdef JSVERIFY_EXP_ACT + +/*---------------------------------------------------------------------------- + * + * Macro: JSVERIFY() + * + * Purpose: + * + * Verify that two long integers are equal. + * If unequal, print failure message + * (with `reason`, if not NULL; expected/actual if NULL) + * and jump to `error` at end of function + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#define JSVERIFY(expected, actual, reason) \ + if ((long)(actual) != (long)(expected)) { \ + JSERR_LONG((expected), (actual), (reason)) \ + goto error; \ + } /* JSVERIFY */ + +/*---------------------------------------------------------------------------- + * + * Macro: JSVERIFY_NOT() + * + * Purpose: + * + * Verify that two long integers are _not_ equal. + * If equal, print failure message + * (with `reason`, if not NULL; expected/actual if NULL) + * and jump to `error` at end of function + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#define JSVERIFY_NOT(expected, actual, reason) \ + if ((long)(actual) == (long)(expected)) { \ + JSERR_LONG((expected), (actual), (reason)) \ + goto error; \ + } /* JSVERIFY_NOT */ + +/*---------------------------------------------------------------------------- + * + * Macro: JSVERIFY_STR() + * + * Purpose: + * + * Verify that two strings are equal. + * If unequal, print failure message + * (with `reason`, if not NULL; expected/actual if NULL) + * and jump to `error` at end of function + * + * Programmer: Jacob Smith + * 2017-10-24 + * + *---------------------------------------------------------------------------- + */ +#define JSVERIFY_STR(expected, actual, reason) \ + if (strcmp((actual), (expected)) != 0) { \ + JSERR_STR((expected), (actual), (reason)); \ + goto error; \ + } /* JSVERIFY_STR */ + +#else +/* JSVERIFY_EXP_ACT not defined + * + * Repeats macros above, but with actual/expected parameters reversed. + */ + +/*---------------------------------------------------------------------------- + * Macro: JSVERIFY() + * See: JSVERIFY documentation above. + * Programmer: Jacob Smith + * 2017-10-14 + *---------------------------------------------------------------------------- + */ +#define JSVERIFY(actual, expected, reason) \ + if ((long)(actual) != (long)(expected)) { \ + JSERR_LONG((expected), (actual), (reason)); \ + goto error; \ + } /* JSVERIFY */ + +/*---------------------------------------------------------------------------- + * Macro: JSVERIFY_NOT() + * See: JSVERIFY_NOT documentation above. + * Programmer: Jacob Smith + * 2017-10-14 + *---------------------------------------------------------------------------- + */ +#define JSVERIFY_NOT(actual, expected, reason) \ + if ((long)(actual) == (long)(expected)) { \ + JSERR_LONG((expected), (actual), (reason)) \ + goto error; \ + } /* JSVERIFY_NOT */ + +/*---------------------------------------------------------------------------- + * Macro: JSVERIFY_STR() + * See: JSVERIFY_STR documentation above. + * Programmer: Jacob Smith + * 2017-10-14 + *---------------------------------------------------------------------------- + */ +#define JSVERIFY_STR(actual, expected, reason) \ + if (strcmp((actual), (expected)) != 0) { \ + JSERR_STR((expected), (actual), (reason)); \ + goto error; \ + } /* JSVERIFY_STR */ + +#endif /* ifdef/else JSVERIFY_EXP_ACT */ + +/******************************** + * OTHER MACROS AND DEFINITIONS * + ********************************/ + +#define MAXADDR (((haddr_t)1 << (8 * sizeof(HDoff_t) - 1)) - 1) + +#define S3_TEST_PROFILE_NAME "ros3_vfd_test" + +#define S3_TEST_MAX_URL_SIZE 256 + +#define S3_TEST_RESOURCE_TEXT_RESTRICTED "t8.shakespeare.txt" +#define S3_TEST_RESOURCE_TEXT_PUBLIC "Poe_Raven.txt" +#define S3_TEST_RESOURCE_H5_PUBLIC "GMODO-SVM01.h5" +#define S3_TEST_RESOURCE_MISSING "missing.csv" + +static char url_text_restricted[S3_TEST_MAX_URL_SIZE] = ""; +static char url_text_public[S3_TEST_MAX_URL_SIZE] = ""; +static char url_h5_public[S3_TEST_MAX_URL_SIZE] = ""; +static char url_missing[S3_TEST_MAX_URL_SIZE] = ""; +static char s3_test_bucket_url[S3_TEST_MAX_URL_SIZE] = ""; +static hbool_t s3_test_bucket_defined = FALSE; + +/* Global variables for aws test profile. + * An attempt is made to read ~/.aws/credentials and ~/.aws/config upon test + * startup -- if unable to open either file or cannot load region, id, and key, + * tests connecting with S3 will not be run + */ +static int s3_test_credentials_loaded = 0; +static char s3_test_aws_region[16]; +static char s3_test_aws_access_key_id[64]; +static char s3_test_aws_secret_access_key[128]; + +H5FD_ros3_fapl_t restricted_access_fa = {H5FD_CURR_ROS3_FAPL_T_VERSION, /* fapl version */ + TRUE, /* authenticate */ + "", /* aws region */ + "", /* access key id */ + ""}; /* secret access key */ + +H5FD_ros3_fapl_t anonymous_fa = {H5FD_CURR_ROS3_FAPL_T_VERSION, FALSE, "", "", ""}; + +/*--------------------------------------------------------------------------- + * + * Function: test_fapl_config_validation() + * + * Purpose: + * + * Test data consistency of fapl configuration. + * Tests `H5FD_ros3_validate_config` indirectly through `H5Pset_fapl_ros3`. + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 2017-10-23 + * + *--------------------------------------------------------------------------- + */ +static int +test_fapl_config_validation(void) +{ + + /********************* + * test-local macros * + *********************/ + + /************************* + * test-local structures * + *************************/ + + struct testcase { + const char * msg; + herr_t expected; + H5FD_ros3_fapl_t config; + }; + + /************************ + * test-local variables * + ************************/ + + hid_t fapl_id = -1; /* file access property list ID */ + H5FD_ros3_fapl_t config; + H5FD_ros3_fapl_t fa_fetch; + herr_t success = SUCCEED; + unsigned int i = 0; + unsigned int ncases = 8; /* should equal number of cases */ + struct testcase *case_ptr = NULL; /* dumb work-around for possible */ + /* dynamic cases creation because */ + /* of compiler warnings Wlarger-than */ + struct testcase cases_arr[] = { + { + "non-authenticating config allows empties.\n", + SUCCEED, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, /* version */ + FALSE, /* authenticate */ + "", /* aws_region */ + "", /* secret_id */ + "", /* secret_key */ + }, + }, + { + "authenticating config asks for populated strings.\n", + FAIL, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, + TRUE, + "", + "", + "", + }, + }, + { + "populated strings; key is the empty string?\n", + SUCCEED, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, + TRUE, + "region", + "me", + "", + }, + }, + { + "id cannot be empty.\n", + FAIL, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, + TRUE, + "", + "me", + "", + }, + }, + { + "region cannot be empty.\n", + FAIL, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, + TRUE, + "where", + "", + "", + }, + }, + { + "all strings populated.\n", + SUCCEED, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, + TRUE, + "where", + "who", + "thisIsA GREAT seeeecrit", + }, + }, + { + "incorrect version should fail\n", + FAIL, + { + 12345, + FALSE, + "", + "", + "", + }, + }, + { + "non-authenticating config cares not for (de)population" + "of strings.\n", + SUCCEED, + { + H5FD_CURR_ROS3_FAPL_T_VERSION, + FALSE, + "someregion", + "someid", + "somekey", + }, + }, + }; + + TESTING("ROS3 fapl configuration validation"); + + /********* + * TESTS * + *********/ + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + for (i = 0; i < ncases; i++) { + + /*--------------- + * per-test setup + *--------------- + */ + case_ptr = &cases_arr[i]; + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_id < 0) /* sanity-check */ + + /*----------------------------------- + * Actually test. + * Mute stack trace in failure cases. + *----------------------------------- + */ + H5E_BEGIN_TRY + { + /* `H5FD_ros3_validate_config(...)` is static/private + * to src/ros3.c and cannot (and should not?) be tested directly? + * Instead, validate config through public api. + */ + success = H5Pset_fapl_ros3(fapl_id, &case_ptr->config); + } + H5E_END_TRY; + + JSVERIFY(case_ptr->expected, success, case_ptr->msg) + + /* Make sure we can get back what we put in. + * Only valid if the fapl configuration does not result in error. + */ + if (success == SUCCEED) { + config = case_ptr->config; + JSVERIFY(SUCCEED, H5Pget_fapl_ros3(fapl_id, &fa_fetch), "unable to get fapl") + + JSVERIFY(H5FD_CURR_ROS3_FAPL_T_VERSION, fa_fetch.version, "invalid version number") + JSVERIFY(config.version, fa_fetch.version, "version number mismatch") + JSVERIFY(config.authenticate, fa_fetch.authenticate, "authentication flag mismatch") + JSVERIFY_STR(config.aws_region, fa_fetch.aws_region, NULL) + JSVERIFY_STR(config.secret_id, fa_fetch.secret_id, NULL) + JSVERIFY_STR(config.secret_key, fa_fetch.secret_key, NULL) + } + + /*----------------------------- + * per-test sanitation/teardown + *----------------------------- + */ + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + } /* for each test case */ + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (fapl_id < 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + return 1; +} /* test_fapl_config_validation */ + +/*------------------------------------------------------------------------- + * + * Function: test_ros3_fapl() + * + * Purpose: Tests the file handle interface for the ROS3 driver + * + * As the ROS3 driver is 1) read only, 2) requires access + * to an S3 server, this test is quite + * different from the other tests. + * + * For now, test only fapl & flags. Extend as the + * work on the VFD continues. + * + * Return: Success: 0 + * Failure: 1 + * + * Programmer: John Mainzer + * 7/12/17 + * + *------------------------------------------------------------------------- + */ +static int +test_ros3_fapl(void) +{ + /************************ + * test-local variables * + ************************/ + + hid_t fapl_id = -1; /* file access property list ID */ + hid_t driver_id = -1; /* ID for this VFD */ + unsigned long driver_flags = 0; /* VFD feature flags */ + H5FD_ros3_fapl_t ros3_fa_0 = { + H5FD_CURR_ROS3_FAPL_T_VERSION, /* version */ + FALSE, /* authenticate */ + "", /* aws_region */ + "", /* secret_id */ + "plugh", /* secret_key */ + }; + + TESTING("ROS3 fapl "); + + /* Set property list and file name for ROS3 driver. + */ + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_id < 0) + + FAIL_IF(FAIL == H5Pset_fapl_ros3(fapl_id, &ros3_fa_0)) + + driver_id = H5Pget_driver(fapl_id); + FAIL_IF(driver_id < 0) + + /**************** + * Check that the VFD feature flags are correct + * SPEC MAY CHANGE + ******************/ + + FAIL_IF(H5FDdriver_query(driver_id, &driver_flags) < 0) + + JSVERIFY_NOT(0, (driver_flags & H5FD_FEAT_DATA_SIEVE), + "bit(s) in `driver_flags` must align with " + "H5FD_FEAT_DATA_SIEVE") + + JSVERIFY(H5FD_FEAT_DATA_SIEVE, driver_flags, "H5FD_FEAT_DATA_SIEVE should be the only supported flag") + + PASSED(); + return 0; + +error: + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + + return 1; + +} /* test_ros3_fapl() */ + +/*--------------------------------------------------------------------------- + * + * Function: test_vfd_open() + * + * Purpose: + * + * Demonstrate/specify VFD-level "Open" failure cases + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 1027-11-03 + * + *--------------------------------------------------------------------------- + */ +static int +test_vfd_open(void) +{ + + /********************* + * test-local macros * + *********************/ + +#define FAPL_H5P_DEFAULT -2 +#define FAPL_FILE_ACCESS -3 +#define FAPL_ROS3_ANON -4 + + /************************* + * test-local structures * + *************************/ + + struct test_condition { + const char *message; + const char *url; + unsigned flags; + int which_fapl; + haddr_t maxaddr; + }; + + /************************ + * test-local variables * + ************************/ + + struct test_condition tests[] = { + { + "default property list (H5P_DEFAULT) is invalid", + url_text_public, + H5F_ACC_RDONLY, + FAPL_H5P_DEFAULT, + MAXADDR, + }, + { + "generic file access property list is invalid", + url_text_public, + H5F_ACC_RDONLY, + FAPL_FILE_ACCESS, + MAXADDR, + }, + { + "filename cannot be null", + NULL, + H5F_ACC_RDONLY, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "filename cannot be empty", + "", + H5F_ACC_RDONLY, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "filename must exist", + url_missing, + H5F_ACC_RDONLY, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "read-write flag not supported", + url_text_public, + H5F_ACC_RDWR, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "truncate flag not supported", + url_text_public, + H5F_ACC_TRUNC, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "create flag not supported", + url_text_public, + H5F_ACC_CREAT, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "EXCL flag not supported", + url_text_public, + H5F_ACC_EXCL, + FAPL_ROS3_ANON, + MAXADDR, + }, + { + "maxaddr cannot be 0 (caught in `H5FD_open()`)", + url_text_public, + H5F_ACC_RDONLY, + FAPL_ROS3_ANON, + 0, + }, + }; + H5FD_t * fd = NULL; + hbool_t curl_ready = FALSE; + hid_t fapl_id = -1; + hid_t fapl_file_access = -1; + unsigned i = 0; + unsigned tests_count = 10; + + TESTING("ROS3 VFD-level open"); + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + FAIL_IF(CURLE_OK != curl_global_init(CURL_GLOBAL_DEFAULT)) + curl_ready = TRUE; + + fapl_file_access = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_file_access < 0) + + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_id < 0) + FAIL_IF(FAIL == H5Pset_fapl_ros3(fapl_id, &anonymous_fa)) + + /********* + * TESTS * + *********/ + + /* all the test cases that will _not_ open + */ + for (i = 0; i < tests_count; i++) { + struct test_condition T = tests[i]; + hid_t _fapl_id = H5P_DEFAULT; + + fd = NULL; + + if (T.which_fapl == FAPL_FILE_ACCESS) + _fapl_id = fapl_file_access; + else if (T.which_fapl == FAPL_ROS3_ANON) + _fapl_id = fapl_id; + + H5E_BEGIN_TRY { fd = H5FDopen(T.url, T.flags, _fapl_id, T.maxaddr); } + H5E_END_TRY; + if (NULL != fd) + JSVERIFY(1, 0, T.message); /* wrapper to print message and fail */ + } + + FAIL_IF(NULL != fd) + + /* finally, show that a file can be opened + */ + fd = H5FDopen(url_text_public, H5F_ACC_RDONLY, fapl_id, MAXADDR); + FAIL_IF(NULL == fd) + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5FDclose(fd)) + fd = NULL; + + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + FAIL_IF(FAIL == H5Pclose(fapl_file_access)) + fapl_file_access = -1; + + curl_global_cleanup(); + curl_ready = FALSE; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (fd) { + (void)H5FDclose(fd); + } + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + if (fapl_file_access >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_file_access); } + H5E_END_TRY; + } + if (curl_ready == TRUE) { + curl_global_cleanup(); + } + + return 1; + +#undef FAPL_FILE_ACCESS +#undef FAPL_H5P_DEFAULT +#undef FAPL_ROS3_ANON + +} /* test_vfd_open */ + +/*--------------------------------------------------------------------------- + * + * Function: test_eof_eoa() + * + * Purpose: + * + * Demonstrate behavior of get_eof, get_eoa, and set_eoa. + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 2017-11-08 + * + *--------------------------------------------------------------------------- + */ +static int +test_eof_eoa(void) +{ + + /********************* + * test-local macros * + *********************/ + + /************************* + * test-local structures * + *************************/ + + /************************ + * test-local variables * + ************************/ + + H5FD_t *fd_shakespeare = NULL; + hbool_t curl_ready = FALSE; + hid_t fapl_id = -1; + + TESTING("ROS3 eof/eoa gets and sets"); + + if (s3_test_credentials_loaded == 0) { + SKIPPED(); + puts(" s3 credentials are not loaded"); + fflush(stdout); + return 0; + } + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + /********* + * SETUP * + *********/ + + FAIL_IF(CURLE_OK != curl_global_init(CURL_GLOBAL_DEFAULT)) + curl_ready = TRUE; + + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(0 > fapl_id) + FAIL_IF(FAIL == H5Pset_fapl_ros3(fapl_id, &restricted_access_fa)) + + fd_shakespeare = H5FDopen(url_text_restricted, H5F_ACC_RDONLY, fapl_id, HADDR_UNDEF); + FAIL_IF(NULL == fd_shakespeare) + + /********* + * TESTS * + *********/ + + /* verify as found + */ + JSVERIFY(5458199, H5FDget_eof(fd_shakespeare), NULL) + JSVERIFY(0, H5FDget_eoa(fd_shakespeare, H5FD_MEM_DEFAULT), "EoA should be unset by H5FDopen") + + /* set EoA below EoF + */ + JSVERIFY(SUCCEED, H5FDset_eoa(fd_shakespeare, H5FD_MEM_DEFAULT, 44442202), "unable to set EoA (lower)") + JSVERIFY(5458199, H5FDget_eof(fd_shakespeare), "EoF changed") + JSVERIFY(44442202, H5FDget_eoa(fd_shakespeare, H5FD_MEM_DEFAULT), "EoA unchanged") + + /* set EoA above EoF + */ + JSVERIFY(SUCCEED, H5FDset_eoa(fd_shakespeare, H5FD_MEM_DEFAULT, 6789012), "unable to set EoA (higher)") + JSVERIFY(5458199, H5FDget_eof(fd_shakespeare), "EoF changed") + JSVERIFY(6789012, H5FDget_eoa(fd_shakespeare, H5FD_MEM_DEFAULT), "EoA unchanged") + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5FDclose(fd_shakespeare)) + + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + curl_global_cleanup(); + curl_ready = FALSE; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (fd_shakespeare) + (void)H5FDclose(fd_shakespeare); + if (TRUE == curl_ready) + curl_global_cleanup(); + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + + return 1; + +} /* test_eof_eoa */ + +/*----------------------------------------------------------------------------- + * + * Function: test_H5FDread_without_eoa_set_fails() + * + * Purpose: + * + * Demonstrate a not-obvious constraint by the library, preventing + * file read before EoA is set + * + * Programmer: Jacob Smith + * 2018-01-26 + * + *----------------------------------------------------------------------------- + */ +static int +test_H5FDread_without_eoa_set_fails(void) +{ + char buffer[256]; + unsigned int i = 0; + H5FD_t * file_shakespeare = NULL; + hid_t fapl_id = -1; + + TESTING("ROS3 VFD read-eoa temporal coupling library limitation "); + + if (s3_test_credentials_loaded == 0) { + SKIPPED(); + puts(" s3 credentials are not loaded"); + fflush(stdout); + return 0; + } + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + /********* + * SETUP * + *********/ + + /* create ROS3 fapl + */ + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_id < 0) + FAIL_IF(FAIL == H5Pset_fapl_ros3(fapl_id, &restricted_access_fa)) + + file_shakespeare = H5FDopen(url_text_restricted, H5F_ACC_RDONLY, fapl_id, MAXADDR); + FAIL_IF(NULL == file_shakespeare) + + JSVERIFY(0, H5FDget_eoa(file_shakespeare, H5FD_MEM_DEFAULT), "EoA should remain unset by H5FDopen") + + for (i = 0; i < 256; i++) + buffer[i] = 0; /* zero buffer contents */ + + /******** + * TEST * + ********/ + + H5E_BEGIN_TRY{/* mute stack trace on expected failure */ + JSVERIFY(FAIL, H5FDread(file_shakespeare, H5FD_MEM_DRAW, H5P_DEFAULT, 1200699, 102, buffer), + "cannot read before eoa is set")} H5E_END_TRY; + JSVERIFY_STR("", buffer, "buffer should remain untouched") + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5FDclose(file_shakespeare)) + file_shakespeare = NULL; + + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (file_shakespeare) { + (void)H5FDclose(file_shakespeare); + } + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + + return 1; + +} /* test_H5FDread_without_eoa_set_fails */ + +/*--------------------------------------------------------------------------- + * + * Function: test_read() + * + * Purpose: + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 2017-11-06 + * + *--------------------------------------------------------------------------- + */ +static int +test_read(void) +{ + + /********************* + * test-local macros * + *********************/ + + /************************* + * test-local structures * + *************************/ + struct testcase { + const char *message; /* purpose of test case */ + haddr_t eoa_set; /* set file EOA to this prior to read */ + size_t addr; /* offset of read in file */ + size_t len; /* length of read in file */ + herr_t success; /* expected return value of read function */ + const char *expected; /* expected contents of buffer; failure ignores */ + }; + + /************************ + * test-local variables * + ************************/ + struct testcase cases[] = { + { + "successful range-get", + 6464, + 5691, + 32, /* fancy quotes are three bytes each(?) */ + SUCCEED, + "Quoth the Raven “Nevermore.”", + }, + { + "read past EOA fails (EOA < EOF < addr)", + 3000, + 4000, + 100, + FAIL, + NULL, + }, + { + "read overlapping EOA fails (EOA < addr < EOF < (addr+len))", + 3000, + 8000, + 100, + FAIL, + NULL, + }, + { + "read past EOA/EOF fails ((EOA==EOF) < addr)", + 6464, + 7000, + 100, + FAIL, + NULL, + }, + { + "read overlapping EOA/EOF fails (addr < (EOA==EOF) < (addr+len))", + 6464, + 6400, + 100, + FAIL, + NULL, + }, + { + "read between EOF and EOA fails (EOF < addr < (addr+len) < EOA)", + 8000, + 7000, + 100, + FAIL, + NULL, + }, + }; + unsigned testcase_count = 6; + unsigned test_i = 0; + struct testcase test; + herr_t open_return = FAIL; + char buffer[S3_TEST_MAX_URL_SIZE]; + unsigned int i = 0; + H5FD_t * file_raven = NULL; + hid_t fapl_id = -1; + + TESTING("ROS3 VFD read/range-gets"); + + if (s3_test_credentials_loaded == 0) { + SKIPPED(); + puts(" s3 credentials are not loaded"); + fflush(stdout); + return 0; + } + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + /********* + * SETUP * + *********/ + + /* create ROS3 fapl + */ + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_id < 0) + FAIL_IF(FAIL == H5Pset_fapl_ros3(fapl_id, &restricted_access_fa)) + + /* open file + */ + file_raven = + H5FDopen( /* will open with "authenticating" fapl */ + url_text_public, /* TODO: check return state: anon access of restricted says OK? (not NULL) + */ + H5F_ACC_RDONLY, fapl_id, HADDR_UNDEF); /* Demonstrate success with "automatic" value */ + FAIL_IF(NULL == file_raven) + + JSVERIFY(6464, H5FDget_eof(file_raven), NULL) + + /********* + * TESTS * + *********/ + + for (test_i = 0; test_i < testcase_count; test_i++) { + + /* -------------- * + * per-test setup * + * -------------- */ + + test = cases[test_i]; + open_return = FAIL; + + FAIL_IF(S3_TEST_MAX_URL_SIZE < test.len) /* buffer too small! */ + + FAIL_IF(FAIL == H5FD_set_eoa(file_raven, H5FD_MEM_DEFAULT, test.eoa_set)) + + for (i = 0; i < S3_TEST_MAX_URL_SIZE; i++) /* zero buffer contents */ + buffer[i] = 0; + + /* ------------ * + * conduct test * + * ------------ */ + + H5E_BEGIN_TRY + { + open_return = H5FDread(file_raven, H5FD_MEM_DRAW, H5P_DEFAULT, test.addr, test.len, buffer); + } + H5E_END_TRY; + + JSVERIFY(test.success, open_return, test.message) + if (open_return == SUCCEED) + JSVERIFY_STR(test.expected, buffer, NULL) + + } /* for each testcase */ + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5FDclose(file_raven)) + file_raven = NULL; + + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (file_raven) + (void)H5FDclose(file_raven); + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + + return 1; + +} /* test_read */ + +/*--------------------------------------------------------------------------- + * + * Function: test_noops_and_autofails() + * + * Purpose: + * + * Demonstrate the unavailable and do-nothing routines unique to + * Read-Only VFD. + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 2017-11-06 + * + *--------------------------------------------------------------------------- + */ +static int +test_noops_and_autofails(void) +{ + /********************* + * test-local macros * + *********************/ + + /************************* + * test-local structures * + *************************/ + + /************************ + * test-local variables * + ************************/ + + hbool_t curl_ready = FALSE; + hid_t fapl_id = -1; + H5FD_t * file = NULL; + const char data[36] = "The Force shall be with you, always"; + + TESTING("ROS3 VFD always-fail and no-op routines"); + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + /********* + * SETUP * + *********/ + + FAIL_IF(CURLE_OK != curl_global_init(CURL_GLOBAL_DEFAULT)) + curl_ready = TRUE; + + /* create ROS3 fapl + */ + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(fapl_id < 0) + JSVERIFY(SUCCEED, H5Pset_fapl_ros3(fapl_id, &anonymous_fa), NULL) + + /* open file + */ + file = H5FDopen(url_text_public, H5F_ACC_RDONLY, fapl_id, HADDR_UNDEF); + FAIL_IF(NULL == file) + + /********* + * TESTS * + *********/ + + /* auto-fail calls to write and truncate + */ + H5E_BEGIN_TRY{JSVERIFY(FAIL, H5FDwrite(file, H5FD_MEM_DRAW, H5P_DEFAULT, 1000, 35, data), + "write must fail")} H5E_END_TRY; + + H5E_BEGIN_TRY{JSVERIFY(FAIL, H5FDtruncate(file, H5P_DEFAULT, FALSE), "truncate must fail")} H5E_END_TRY; + + H5E_BEGIN_TRY{ + JSVERIFY(FAIL, H5FDtruncate(file, H5P_DEFAULT, TRUE), "truncate must fail (closing)")} H5E_END_TRY; + + /* no-op calls to `lock()` and `unlock()` + */ + /* + JSVERIFY(SUCCEED, H5FDlock(file, TRUE), "lock always succeeds; has no effect") + JSVERIFY(SUCCEED, H5FDlock(file, FALSE), NULL) + JSVERIFY(SUCCEED, H5FDunlock(file), NULL) + */ + /* Lock/unlock with null file or similar error crashes tests. + * HDassert in calling heirarchy, `H5FD[un]lock()` and `H5FD_[un]lock()` + */ + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5FDclose(file)) + file = NULL; + + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + curl_global_cleanup(); + curl_ready = FALSE; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + if (file) { + (void)H5FDclose(file); + } + if (curl_ready == TRUE) { + curl_global_cleanup(); + } + + return 1; + +} /* test_noops_and_autofails*/ + +/*--------------------------------------------------------------------------- + * + * Function: test_cmp() + * + * Purpose: + * + * Verify "file comparison" behavior. + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 2017-11-06 + * + *--------------------------------------------------------------------------- + */ +static int +test_cmp(void) +{ + + /********************* + * test-local macros * + *********************/ + + /************************* + * test-local structures * + *************************/ + + /************************ + * test-local variables * + ************************/ + + H5FD_t *fd_raven = NULL; + H5FD_t *fd_shakes = NULL; + H5FD_t *fd_raven_2 = NULL; + hbool_t curl_ready = FALSE; + hid_t fapl_id = -1; + + TESTING("ROS3 cmp (comparison)"); + + if (s3_test_credentials_loaded == 0) { + SKIPPED(); + puts(" s3 credentials are not loaded"); + fflush(stdout); + return 0; + } + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + /********* + * SETUP * + *********/ + + FAIL_IF(CURLE_OK != curl_global_init(CURL_GLOBAL_DEFAULT)) + curl_ready = TRUE; + + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(0 > fapl_id) + JSVERIFY(SUCCEED, H5Pset_fapl_ros3(fapl_id, &restricted_access_fa), NULL) + + fd_raven = H5FDopen(url_text_public, H5F_ACC_RDONLY, fapl_id, HADDR_UNDEF); + FAIL_IF(NULL == fd_raven) + + fd_shakes = H5FDopen(url_text_restricted, H5F_ACC_RDONLY, fapl_id, HADDR_UNDEF); + FAIL_IF(NULL == fd_shakes) + + fd_raven_2 = H5FDopen(url_text_public, H5F_ACC_RDONLY, fapl_id, HADDR_UNDEF); + FAIL_IF(NULL == fd_raven_2) + + /********* + * TESTS * + *********/ + + JSVERIFY(0, H5FDcmp(fd_raven, fd_raven_2), NULL) + JSVERIFY(-1, H5FDcmp(fd_raven, fd_shakes), NULL) + JSVERIFY(-1, H5FDcmp(fd_shakes, fd_raven_2), NULL) + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5FDclose(fd_raven)) + fd_raven = NULL; + FAIL_IF(FAIL == H5FDclose(fd_shakes)) + fd_shakes = NULL; + FAIL_IF(FAIL == H5FDclose(fd_raven_2)) + fd_raven_2 = NULL; + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + curl_global_cleanup(); + curl_ready = FALSE; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + + if (fd_raven != NULL) + (void)H5FDclose(fd_raven); + if (fd_raven_2 != NULL) + (void)H5FDclose(fd_raven_2); + if (fd_shakes != NULL) + (void)H5FDclose(fd_shakes); + if (TRUE == curl_ready) + curl_global_cleanup(); + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + + return 1; + +} /* test_cmp */ + +/*--------------------------------------------------------------------------- + * + * Function: test_H5F_integration() + * + * Purpose: + * + * Demonstrate S3 file-open through H5F API. + * + * Return: + * + * PASSED : 0 + * FAILED : 1 + * + * Programmer: Jacob Smith + * 2017-11-07 + * + *--------------------------------------------------------------------------- + */ +static int +test_H5F_integration(void) +{ + /********************* + * test-local macros * + *********************/ + + /************************* + * test-local structures * + *************************/ + + /************************ + * test-local variables * + ************************/ + + hid_t file = -1; + hid_t fapl_id = -1; + + TESTING("S3 file access through HD5F library (H5F API)"); + + if (s3_test_credentials_loaded == 0) { + SKIPPED(); + puts(" s3 credentials are not loaded"); + fflush(stdout); + return 0; + } + + if (FALSE == s3_test_bucket_defined) { + SKIPPED(); + puts(" environment variable HDF5_ROS3_TEST_BUCKET_URL not defined"); + fflush(stdout); + return 0; + } + + /********* + * SETUP * + *********/ + + fapl_id = H5Pcreate(H5P_FILE_ACCESS); + FAIL_IF(0 > fapl_id) + FAIL_IF(FAIL == H5Pset_fapl_ros3(fapl_id, &restricted_access_fa)) + + /********* + * TESTS * + *********/ + + /* Read-Write Open access is not allowed with this file driver. + */ + H5E_BEGIN_TRY{FAIL_IF(0 <= H5Fopen(url_h5_public, H5F_ACC_RDWR, fapl_id))} H5E_END_TRY; + + /* H5Fcreate() is not allowed with this file driver. + */ + H5E_BEGIN_TRY{FAIL_IF(0 <= H5Fcreate(url_missing, H5F_ACC_RDONLY, H5P_DEFAULT, fapl_id))} H5E_END_TRY; + + /* Successful open. + */ + file = H5Fopen(url_h5_public, H5F_ACC_RDONLY, fapl_id); + FAIL_IF(file < 0) + + /************ + * TEARDOWN * + ************/ + + FAIL_IF(FAIL == H5Fclose(file)) + file = -1; + + FAIL_IF(FAIL == H5Pclose(fapl_id)) + fapl_id = -1; + + PASSED(); + return 0; + +error: + /*********** + * CLEANUP * + ***********/ + HDprintf("\nerror!"); + fflush(stdout); + + if (fapl_id >= 0) { + H5E_BEGIN_TRY { (void)H5Pclose(fapl_id); } + H5E_END_TRY; + } + if (file > 0) + (void)H5Fclose(file); + + return 1; + +} /* test_H5F_integration */ + +#endif /* H5_HAVE_ROS3_VFD */ + +/*------------------------------------------------------------------------- + * + * Function: main + * + * Purpose: Tests the basic features of Virtual File Drivers + * + * Return: Success: 0 + * Failure: 1 + * + * Programmer: Jacob Smith + * 2017-10-23 + * + *------------------------------------------------------------------------- + */ +int +main(void) +{ +#ifdef H5_HAVE_ROS3_VFD + int nerrors = 0; + const char *bucket_url_env = NULL; + +#endif /* H5_HAVE_ROS3_VFD */ + + HDprintf("Testing ros3 VFD functionality.\n"); + +#ifdef H5_HAVE_ROS3_VFD + + /************************ + * initialize test urls * + ************************/ + + bucket_url_env = HDgetenv("HDF5_ROS3_TEST_BUCKET_URL"); + if (bucket_url_env == NULL || bucket_url_env[0] == '\0') { + HDprintf("WARNING: S3 bucket url is not defined in enviornment " + "variable 'HDF5_ROS3_TEST_BUCKET_URL'!\n"); + } + else { + HDstrncpy(s3_test_bucket_url, bucket_url_env, S3_TEST_MAX_URL_SIZE); + s3_test_bucket_defined = TRUE; + } + + if (S3_TEST_MAX_URL_SIZE < HDsnprintf(url_text_restricted, (size_t)S3_TEST_MAX_URL_SIZE, "%s/%s", + (const char *)s3_test_bucket_url, + (const char *)S3_TEST_RESOURCE_TEXT_RESTRICTED)) { + HDprintf("* ros3 setup failed (text_restricted) ! *\n"); + return 1; + } + if (S3_TEST_MAX_URL_SIZE < HDsnprintf(url_text_public, (size_t)S3_TEST_MAX_URL_SIZE, "%s/%s", + (const char *)s3_test_bucket_url, + (const char *)S3_TEST_RESOURCE_TEXT_PUBLIC)) { + HDprintf("* ros3 setup failed (text_public) ! *\n"); + return 1; + } + if (S3_TEST_MAX_URL_SIZE < HDsnprintf(url_h5_public, (size_t)S3_TEST_MAX_URL_SIZE, "%s/%s", + (const char *)s3_test_bucket_url, + (const char *)S3_TEST_RESOURCE_H5_PUBLIC)) { + HDprintf("* ros3 setup failed (h5_public) ! *\n"); + return 1; + } + if (S3_TEST_MAX_URL_SIZE < HDsnprintf(url_missing, S3_TEST_MAX_URL_SIZE, "%s/%s", + (const char *)s3_test_bucket_url, + (const char *)S3_TEST_RESOURCE_MISSING)) { + HDprintf("* ros3 setup failed (missing) ! *\n"); + return 1; + } + + /************************************** + * load credentials and prepare fapls * + **************************************/ + + /* "clear" profile data strings */ + s3_test_aws_access_key_id[0] = '\0'; + s3_test_aws_secret_access_key[0] = '\0'; + s3_test_aws_region[0] = '\0'; + + /* attempt to load test credentials + * if unable, certain tests will be skipped + */ + if (SUCCEED == H5FD_s3comms_load_aws_profile(S3_TEST_PROFILE_NAME, s3_test_aws_access_key_id, + s3_test_aws_secret_access_key, s3_test_aws_region)) { + s3_test_credentials_loaded = 1; + HDstrncpy(restricted_access_fa.aws_region, (const char *)s3_test_aws_region, + H5FD_ROS3_MAX_REGION_LEN); + HDstrncpy(restricted_access_fa.secret_id, (const char *)s3_test_aws_access_key_id, + H5FD_ROS3_MAX_SECRET_ID_LEN); + HDstrncpy(restricted_access_fa.secret_key, (const char *)s3_test_aws_secret_access_key, + H5FD_ROS3_MAX_SECRET_KEY_LEN); + } + + /****************** + * commence tests * + ******************/ + + h5_reset(); + + nerrors += test_fapl_config_validation(); + nerrors += test_ros3_fapl(); + nerrors += test_vfd_open(); + nerrors += test_eof_eoa(); + nerrors += test_H5FDread_without_eoa_set_fails(); + nerrors += test_read(); + nerrors += test_noops_and_autofails(); + nerrors += test_cmp(); + nerrors += test_H5F_integration(); + + if (nerrors > 0) { + HDprintf("***** %d ros3 TEST%s FAILED! *****\n", nerrors, nerrors > 1 ? "S" : ""); + nerrors = 1; + } + else { + HDprintf("All ros3 tests passed.\n"); + } + return nerrors; /* 0 if no errors, 1 if any errors */ + +#else + + HDprintf("SKIPPED - read-only S3 VFD not built\n"); + return EXIT_SUCCESS; + +#endif /* H5_HAVE_ROS3_VFD */ + +} /* main() */ |