summaryrefslogtreecommitdiffstats
path: root/src/H5FDros3.c
Commit message (Collapse)AuthorAgeFilesLines
* Increased H5FD_ROS3_MAX_SECRET_TOK_LEN to 4096 to accommodate long AWS ↵Michael Aivazis2024-03-071-24/+26
| | | | | secret tokens (#4064) ros3: increased H5FD_ROS3_MAX_SECRET_TOK_LEN to 4096; stratified the debugging statements so there is more control over the output
* Bump the ros3 VFD cache to 16 MiB (#3759)Dana Robinson2023-10-241-1/+1
|
* Add a simple cache to the ros3 VFD (#3753)Dana Robinson2023-10-231-22/+51
| | | | | | | | | Adds a small cache of the first N bytes of a file opened with the read-only S3 (ros3) VFD, where N is 4kiB or the size of the file, whichever is smaller. This avoids a lot of small I/O operations on file open. Addresses GitHub issue #3381
* Fix several spelling/grammar issues (#3621)jhendersonHDF2023-10-031-2/+2
|
* Strip HD prefix from string/char C API calls (#3540)Dana Robinson2023-09-151-14/+14
| | | | | | | | | | | | | | | | | | | | | * Strip HD prefix from string/char C API calls * HD(f)(put|get)(s|c) * HDstr* * HDv*printf * HD(s)(print|scan)f * HDperror But NOT: * HDstrcase* * HDvasprintf * HDstrtok_r * HDstrndup As those are not C99 and have portability work-around implementations. They will be handled later. * Fix th5_system.c screwup
* Convert hbool_t --> bool in src (#3496)Dana Robinson2023-09-051-6/+5
| | | | | | | | | | | * hbool_t --> bool in src * Does not remove TRUE/FALSE * Public header files are unchanged * Public API calls are unchanged * TRUE/FALSE --> true/false in src * Add deprecation notice for hbool_t
* Convert some H5MM calls to standard C equivalents (#2382)Dana Robinson2023-08-251-2/+2
| | | | | | | | | * H5MM_calloc and malloc are now mapped to stdlib C calls * H5MM_memcpy now maps directly to memcpy in release builds * H5MM_memcpy is still implemented as a separate function that checks for buffer overlap when H5MM_DEBUG is defined (default w/ debug builds) * Switches many library memcpy calls to use H5MM_memcpy * Fixes a possible zero allocation in H5Olayout.c
* Made HGOTO_ERROR a do-while loop (#3308)Sean McBride2023-08-021-41/+41
| | | | | * Made HGOTO_ERROR a do-while loop
* Another round of fixing -Wextra-semi-stmt warnings (#3264)Sean McBride2023-07-241-2/+2
| | | Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* ROS3: (fix) Replaced HDfprintf (#3266)Jan-Willem Blokland2023-07-241-3/+3
| | | | - Replaced the HDfprintf() functions by fprintf() to be consistent with other parts of the library.
* Fixed more warnings about extra semicolons (#3249)Sean McBride2023-07-181-20/+20
| | | | | | | * Require semi-colon after H5_CHECK_OVERFLOW calls Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* ROS3: (feature) Temporary security credentials (#3030)Jan-Willem Blokland2023-07-141-3/+280
| | | | | | | | | | - Implemented support for AWS temporary security credentials. For this kind of credentials also a session/security token should be included in the request by adding the x-amz-security-token header. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> Co-authored-by: Jordan Henderson <jhenderson@hdfgroup.org> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Remove programmer/date from comments (#3210)Dana Robinson2023-06-291-71/+0
| | | | * Removes Programmer: and Date: fields * Fixes a few Modifications: fields leftover from previous work
* Remove HD from C std lib file ops (#3206)Dana Robinson2023-06-291-1/+1
| | | | | | | | | * HDfclose * HDferror * HDfeof * HDfflush * HDfopen * HDfread * HDfwrite
* Rename HD(f)printf() to (f)printf() (#3194)Dana Robinson2023-06-281-51/+51
|
* Rename HDassert() to assert() (#3191)Dana Robinson2023-06-281-28/+28
| | | | | * Change HDassert to assert * Fix bin/make_err
* Many clang -Wextra-semi-stmt fixes (#2537)Sean McBride2023-06-161-2/+2
| | | | | * Adds semicolons to function-like macros * Adds a do..while(0) loop to some macros * Removes semicolons when inappropriate, especially H5E_TRY_BEGIN/END
* Addressed various Doxygen grammar issues. (#2524)Scot Breitenfeld2023-03-131-4/+4
| | | | | | | | | | * reviewed H5A * Addressed various grammar issues. * remove double words, spelling fixes * replace the use of rawdata with raw data
* Develop clang 13 format (#1933)Allen Byrne2022-07-261-15/+15
| | | | | * Update format source to clang 13 * More format changes
* OESS-168: Remove clang warnings. (#1376)H. Joe Lee2022-04-151-2/+2
|
* Removes the STATIC flavor of FUNC_ENTER macros (#1622)Dana Robinson2022-04-081-18/+18
| | | | | * Removes the STATIC flavor of FUNC_ENTER macros
* Merge initial version of selection I/O feature into develop (#1367)Neil Fortner2022-03-261-0/+5
|
* OESS-168: Remove clang warnings. (#1304)H. Joe Lee2022-01-071-1/+5
|
* Fixed Spelling Errors (#1166)Scot Breitenfeld2021-12-071-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * fixed missed closing of a dataset * fixed missed closing of a dataset * fixed typo in error return * Committing clang-format changes * minor edits * code format * Committing clang-format changes * code format * minor edit * switched from using MPI_count, to actual bytes written for H5FD_mpio_debug rw debugging * Committing clang-format changes * changed size_i in printf to reflect the I/O. * Committing clang-format changes * Fixed seg fault with xlf on BE with -qintsize=8 * fixed error function string * spelling corrections via codespell, added new spell check github actions * Committing clang-format changes * misc * misc * misc * misc * misc * misc * misc * misc * misc * misc * misc * misc * misc * misc * Committing clang-format changes * misc * misc * misc * misc * misc * misc * Committing clang-format changes * misc * work around for https://github.com/codespell-project/codespell/issues/2137 * misc * added missing file * misc * misc. * misc * switch to using Codespell with GitHub Actions * misc. * misc. * fixed more sp errors * Fix new typos found by codespell. * fixed proceed with precede * fixed variable in fortran test * fixed minnum * updated spelling list Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Simplify function enter macros for performance benefits (#1024)David Young2021-11-081-25/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Take a stab at using constructors to initialize instead of function-entry macros. This is a work in progress. It's good enough to run `many_dsets`. * Committing clang-format changes * Add the `many_dsets` benchmark and some scripts I used on jelly for setting up the build/test environment and for recording/flame-graphing profiles. * Committing clang-format changes * Change my Makefile and environment script to work both on jelly and on mayll (and probably on Summit). * Disable clang-format "fix." * Replace the `if (!H5_TERM_GLOBAL)` test in each FUNC_ENTER_ macro with `if (true)`. * Fix bad grammar in a comment. * Instead of labeling the H5*__init_package routines constructors, fold each into an initialization routine, H5*_init(), and call each of the H5*_init() routines. Call most of the H5*_init() routines from H5_init_library() in an explicit order that I found out earlier by instrumenting each __init_package routine and running the library tests. Roll H5FD*__init_package routines into H5FD*_init() routines. This change ends just-in-time initialization of package dependencies by package initializers. Don't track in per-package variables (H5_PKG_INIT_VAR) whether each package has been initialized. Instead, track in a single library variable whether the whole library is initialized or not. Drive the initialization of packages by H5_init_library() with a table of initializer routines. Also drive the termination of packages by H5_term_library() with a table. Perform initialization as needed from FUNC_ENTER_API_INIT(err). This basically restores the old behavior of that macro. Delete a bunch of #definitions in H5private.h that have fallen out of use with these changes. * Committing clang-format changes * Undo the bad auto-formatting that appears to have occurred in spite of my disabling it. Bracket some code in /* clang-format off */ /* clang-format on */ to prevent a recurrence. * Remove a diagnostic abort(). * Fix a logic error: print a comma between every package terminator run, and don't print an initial comma. * Complete the changes I started in H5_term_library() that undo the bad auto-formatting. Stop tracking whether package "tops" were initialized in per-package variables H5*_top_package_initialize_s. H5_term_library() takes care of that for them. Remove H5R_top_term_package() and H5R_term_package(), they don't do anything. * Committing clang-format changes * NFCI. Simplify macro text: replace `if (true) {` with `{`. * Fix formatting and suppress clang-format on a longer range. * Quiet some unused label, unused variable complaints that cropped up after I simplified the FUNC_ENTER_ macros for the sake of performance. * Committing clang-format changes * Delete some programs and scripts that don't belong in the pull request. * Use the right function-entry macro. * Use a sensible format and disable auto-formatting. * Stop calling do-nothing initializer H5FS_init(). Delete it. * Document what changes to make if the default VFD changes. * While I am here, change an `await_prior` flag on the terminator table to `true` to match the previous, non-table-driven code that was here. Found the oversight making the following changes: NFCI: insert an empty line and copy over slightly-edited comments from the previous version, where those comments still correctly explained how library termination operated. * NFCI: lower a staircase. * Replace every occurrence of FUNC_ENTER_NOAPI_INIT(...) with H5_PUSH_FUNC since that is all that that macro does any more. Quiet a bunch of new warnings by changing FUNC_ENTER_NOAPI(...) to FUNC_ENTER_NOAPI_NOERR and removing disused `done:` labels. * NFCI: add curly braces around a multiline statement. * Quiet a signed/unsigned comparison warning. * Add some documentation about library initialization and shutdown. * Make sure that the library is initialized, or else that initialization is already underway, before performing any VFD's initialization. * Committing clang-format changes * Committing clang-format changes * Reduce differences from `develop` branch. * Always initialize `tot_init`. * Committing clang-format changes * Fix typo: H5SL_init initializes skip lists, not VOL. * Remove H5_TERM_GLOBAL test in H5T_init. H5T_init was unusual in that it tested H5_TERM_GLOBAL and exited early if it was set. No other module initializers did that, and I cannot find any reason that should be necessary. Tests still pass when I remove it, so away it goes. * Use HD prefix. * Add function header comments. * Drop the intermediate variable, it's only used once. * Extract subroutine `H5FDperform_init(hid_t (*init)(void))` that initializes the library, if necessary, before calling its VFD-initializer argument. Use H5FDperform_init in the definition of the symbols H5FD_<vfd> (e.g., H5FD_SEC2), which may be evaluated before the library is initialized, like so: ``` ``` I implement H5FDperform_init in its own source file, H5FDperform.c, and exclude that file from trace processing because the `bin/trace` cannot deal with the function-pointer type. * Straggler from last: add new source file src/H5FDperform.c. * Committing clang-format changes * Add a missing file to the MANIFEST. * Switch to FUNC_ENTER_API_NOINIT in H5FDperform_init() and hbool_t in H5_term_library(). Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* VFD plugins (#602)jhendersonHDF2021-09-291-1/+2
| | | | | | | | | | | | | | | | | * Implement support for loading of Virtual File Drivers as plugins Fix plugin caching for VOL connector and VFD plugins Fix plugin iteration to skip paths that can't be opened * Enable dynamic loading of VFDs with HDF5_DRIVER environment variable * Temporarily disable error reporting during H5F_open double file open * Default to using HDstat in h5_get_file_size for unknown VFDs * Use macros for some environment variables that HDF5 interprets * Update "null" and "ctl testing" VFDs
* VFD ctl feature (#981)jhendersonHDF2021-09-231-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added "ctl" callback to the VFD interface, and the associated H5FDctl() and H5FD_ctl() calls. Modified the MPIO VFD accordingly -- specifically: Added ctl() call with op-code support to expose rank, size, and communicator. Modified H5FD_mpi_get_rank(), H5FD_mpi_get_size(), and H5FD_mpi_get_comm() to use the new ctl() callback. In passing removed the const qualifier from the file parameter of these functions, as the file parameter of the ctl callback is not const. Deleted the old H5FD__mpio_mpi_rank(), H5FD__mpio_mpi_size(), and H5FD__mpio_communicator() calls from the MPIO VFD. Deleted H5FD_class_mpi_t from H5FDprivate.h, and modified the MPIO VFD accordingly. Note that all VFDs now use H5FD_class_t, with no special class for VFDs that that support MPI. Some minor touch ups to the Neil's selection I/O mods in passing. Tested serial and parallel, debug and production on charis and jelly. * Reserve a range of VFD "ctl" opcodes for library and experimental usage * Add "ctl" callbacks to passthrough VFDs * Add RELEASE.txt entry for "ctl" callback * Use H5FDopen with H5F_ACC_RDWR flag instead of H5F_ACC_TRUNC in vfd test * Remove handling of passthrough "ctl" flag from multi VFD * Move logic for testing H5FD_CTL__TEST_OPCODE into a testing VFD Revise description of "ctl" callback in RELEASE.txt Remove unused H5FD_CTL__NUM_OPCODES definition Fix some warnings in multi VFD Co-authored-by: mainzer <mainzer#hdfgroup.org>
* Fix several warnings (#747)jhendersonHDF2021-06-161-0/+2
|
* Brings the native implementation of H5Fdelete() from Bitbucket (#524)Dana Robinson2021-04-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Committing clang-format changes * Brings the native VFD H5Fdelete() implementation from Bitbucket Only brings the 'del' callbacks, not the 'open/close' scheme. * Formatter changes * Committing clang-format changes * Fixes direct VFD callback name * Removes UNUSED macro from family API call * Adds barrier and rank 0 check to MPI-I/O VFD delete * Revert "Adds barrier and rank 0 check to MPI-I/O VFD delete" This reverts commit 909765f759d9d96e84f4b8b1cc14f7d2b3ac8143. * Revert "Revert "Adds barrier and rank 0 check to MPI-I/O VFD delete"" This reverts commit 9b04bef1157853fc79fcb8fcc3e8ba1371091702. * Adds a second barrier after the delete in MPI-I/O VFD * Only delete files in the core VFD when the backing store flag is set * Fixes string issues in multi VFD Also, h5test.c cleanup code now uses H5Fdelete(). * Formatted source * Rework fapl checks for MPI-I/O VFD delete callback Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Update license url (#332)Larry Knox2021-02-171-1/+1
| | | | | | * Modify temporary rpath for testing in java example scripts. * Update URL in source file Copyright headers for web copy of COPYING file - src and test directories.
* Removes lock/unlock callbacks from ros3 and hdfs VFDs (#258)Dana Robinson2021-01-141-2/+80
| | | | | | | | | | | | | | | | * Removes no-op callback stubs from read-only VFDs Also changes VFD registration to allow read-only VFDs with no write callback to be registered. * Adds a RELEASE.txt note for HDFFV-11205 For the read-only VFD registration change * Revert "Removes no-op callback stubs from read-only VFDs" This reverts commit a7a95497305d64d2de783fdb0e3186a532446a4a. * Removes lock callbacks from ros3 and hdfs VFDs
* Allow read-only VFD registration (#257)Dana Robinson2021-01-131-139/+4
| | | | | | | | | | * Removes no-op callback stubs from read-only VFDs Also changes VFD registration to allow read-only VFDs with no write callback to be registered. * Adds a RELEASE.txt note for HDFFV-11205 For the read-only VFD registration change
* Enhance API tracing (#120)Quincey Koziol2020-11-261-3/+3
| | | Enhance API tracing to handle more types, and to put tracing info in a string, allowing it to be used when reporting errors. Also refactor ref-counted strings (H5RS) module to add capabilities needed for the tracing. Refactored H5Gname.c routines to use new H5RS routines also. Added /*out*/ tags to API routines that are returning information to the application. Updated H5TRACE macros from running updated trace script over library code. Added tests for new H5RS routines.
* Basic alignment with async branch (#115)Quincey Koziol2020-11-231-8/+8
| | | | | | | * Basic alignment with async branch - trivial changes to reduce clutter in overall diff. * Update minor error code to reflect change within library * Update the error output to match library
* Replaces C std library calls with H5MM equivalentsDana Robinson2020-10-301-4/+4
| | | | Mostly H5MM_memcpy in newer VFDs
* Fixes minor issues in various virtual file driversDana Robinson2020-10-291-2/+1
| | | | | | | | | | | | - HDFS VFD files now ignored in the Autotools when the HDFS VFD is not being built. - All VFD init code uses idiomatic H5I_INVALID_HID instead of -1 or FAIL. - The HDFS VFD now includes H5FDdrvr_module.h in the right place. - The HDFS tests in test/hdfs.c no longer pass NULL to VERIFY macros, which raised warnings. - Minor tweaks to fix const, unused variables, etc. warnings in several VFDs.
* Clang-format of source filesAllen Byrne2020-09-301-274/+237
|
* Correct typos w/HGOTO_DONE & HGOTO_ERRORQuincey Koziol2020-08-101-20/+20
|
* Clean up private / package / static namespace issues (function naming, whichQuincey Koziol2020-08-061-443/+285
| | | | | | header file, FUNC_ENTER / LEAVE, etc). Removed remaining personal email addresses from library source code (still needs cleaned from other directories). Misc. warning, style, and whitespace cleanup.
* Fixed some minor cmp callback bugs and cleaned warnings.Dana Robinson2020-04-171-3/+5
|
* fix for statement declare of typeAllen Byrne2019-09-301-26/+27
|
* Untangle #ifdef mess in ROS3 code.Dana Robinson2019-08-201-104/+7
|
* Fix compiler warnings with diabled ROS3 and HDFS VFDs.Jacob Smith2019-08-141-155/+237
| | | | | | Fix misuse of [HD]strncpy and [HD]snprintf. Minor formatting changes. Fix test cleanup issue with java groups example.
* Add HD to string functions.Larry Knox2019-07-261-17/+17
| | | | | | Switched strtoul to strtoumax in H5FDs3comms.c. Removed unused functions and variables in s3 and hdfs TestH5Pfapl*.java. Update Copyright headers.
* Squashed commit of the following:Larry Knox2019-07-251-0/+1847
Merge changes from update_merged_S3_HDFS branch into develop. commit d5034315aea88629929ac0c9c59ebfafd5f21a31 Merge: 9c48823 d3fdcd8 Author: Larry Knox <lrknox@hdfgroup.org> Date: Thu Jul 25 08:24:53 2019 -0500 Merge branch 'develop' into update_merged_S3_HDFS
generated by cgit v0.12 at 2024-10-31 07:39:11 (GMT)