diff options
Diffstat (limited to 'release_docs/RELEASE.txt')
| -rwxr-xr-x[-rw-r--r--] | release_docs/RELEASE.txt | 1305 |
1 files changed, 914 insertions, 391 deletions
diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt index c1adf8d..0c3873b 100644..100755 --- a/release_docs/RELEASE.txt +++ b/release_docs/RELEASE.txt @@ -1,576 +1,1090 @@ -HDF5 version 1.10.1 released on 2017-04-27 +HDF5 version 1.10.2 released on 2018-03-29 ================================================================================ -INTRODUCTION - -This document describes the differences between HDF5-1.10.0-patch1 and -HDF5 1.10.1, and contains information on the platforms tested and known -problems in HDF5-1.10.1. For more details check the HISTORY*.txt files -in the HDF5 source. - -Links to HDF5 1.10.1 source code, documentation, and additional materials can -be found on The HDF5 web page at: - https://support.hdfgroup.org/HDF5/ +INTRODUCTION -The HDF5 1.10.1 release can be obtained from: +This document describes the differences between this release and the previous +HDF5 release. It contains information on the platforms tested and known +problems in this release. For more details check the HISTORY*.txt files in the +HDF5 source. - https://support.hdfgroup.org/HDF5/release/obtain5.html +Note that documentation in the links below will be updated at the time of each +final release. -User documentation for the snapshot can be accessed directly at this location: +Links to HDF5 documentation can be found on The HDF5 web page: - https://support.hdfgroup.org/HDF5/doc/ + https://portal.hdfgroup.org/display/HDF5/HDF5 -New features in the HDF5-1.10.x release series, including brief general -descriptions of some new and modified APIs, are described in the "New Features -in HDF5 Release 1.10" document: +The official HDF5 releases can be obtained from: - https://support.hdfgroup.org/HDF5/docNewFeatures/index.html + https://www.hdfgroup.org/downloads/hdf5/ -All new and modified APIs are listed in detail in the "HDF5 Software Changes -from Release to Release" document, in the section "Release 10.1 (current -release) versus Release 1.10.0 +Changes from Release to Release and New Features in the HDF5-1.10.x release series +can be found at: - https://support.hdfgroup.org/HDF5/doc/ADGuide/Changes.html + https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide If you have any questions or comments, please send them to the HDF Help Desk: - help@hdfgroup.org + help@hdfgroup.org CONTENTS -- Major New Features Introduced in HDF5 1.10.1 -- Other New Features and Enhancements -- Support for New Platforms, Languages, and Compilers -- Bug Fixes since HDF5-1.10.0-patch1 +- New Features +- Support for new platforms and languages +- Bug Fixes since HDF5-1.10.1 - Supported Platforms - Tested Configuration Features Summary - More Tested Platforms - Known Problems -Major New Features Introduced in HDF5 1.10.1 -============================================ +New Features +============ -For links to the RFCs and documentation in this section please view -https://support.hdfgroup.org/HDF5/docNewFeatures in a web browser. + Configuration and Build Systems: + -------------------------------- + - CMake builds + -------------- -________________________________________ -Metadata Cache Image -________________________________________ + - Changed minimum CMake required version to 3.10. - HDF5 metadata is typically small, and scattered throughout the HDF5 file. - This can affect performance, particularly on large HPC systems. The - Metadata Cache Image feature can improve performance by writing the - metadata cache in a single block on file close, and then populating the - cache with the contents of this block on file open, thus avoiding the many - small I/O operations that would otherwise be required on file open and - close. See the RFC for complete details regarding this feature. Also, - see the Fine Tuning the Metadata Cache documentation. + This change removed the need to support a copy of the FindMPI.cmake module, + which has been removed, along with its subfolder in the config/cmake_ext_mod + location. - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. + (ADB - 2018/03/09) -________________________________________ -Metadata Cache Evict on Close -________________________________________ + - Added pkg-config file generation - The HDF5 library's metadata cache is fairly conservative about holding on - to HDF5 object metadata (object headers, chunk index structures, etc.), - which can cause the cache size to grow, resulting in memory pressure on - an application or system. The "evict on close" property will cause all - metadata for an object to be evicted from the cache as long as metadata - is not referenced from any other open object. See the Fine Tuning the - Metadata Cache documentation for information on the APIs. + Added pkg-config file generation for the C, C++, HL, and HL C++ libraries. + In addition, builds on Linux will create h5cc, h5c++, h5hlcc, and h5hlc++ scripts in the bin + directory that use the pkg-config files. The scripts can be used to build HDF5 C and C++ + applications (i.e, similar to the compiler scripts produced by the Autotools builds). - At present, evict on close is disabled in parallel builds. + (ADB - 2018/03/08, HDFFV-4359) -________________________________________ -Paged Aggregation -________________________________________ + - Refactored use of CMAKE_BUILD_TYPE for new variable, which understands + the type of generator in use. - The current HDF5 file space allocation accumulates small pieces of metadata - and raw data in aggregator blocks which are not page aligned and vary - widely in sizes. The paged aggregation feature was implemented to provide - efficient paged access of these small pieces of metadata and raw data. - See the RFC for details. Also, see the File Space Management documentation. - -________________________________________ -Page Buffering -________________________________________ + Added new configuration macros to use new HDF_BUILD_TYPE variable. This + variable is set correctly for the type of generator being used for the build. - Small and random I/O accesses on parallel file systems result in poor - performance for applications. Page buffering in conjunction with paged - aggregation can improve performance by giving an application control of - minimizing HDF5 I/O requests to a specific granularity and alignment. - See the RFC for details. Also, see the Page Buffering documentation. + (ADB - 2018/01/08, HDFFV-10385, HDFFV-10296) - At present, page buffering is disabled in parallel builds. + - Autotools builds + ------------------ + - Removed version-specific gcc/gfortran flags for version 4.0 (inclusive) + and earlier. + The config/gnu-flags file, which is sourced as a part of the configure + process, adds version-specific flags for use when building HDF5. Most of + these flags control warnings and do not affect the final product. -Other New Features and Enhancements -=================================== + Flags for older versions of the compiler were consolidated into the + common flags section. Moving these flags simplifies maintenance of + the file. - Library - ------- - - Added a mechanism for disabling the SWMR file locking scheme. + The upshot of this is that building with ancient versions of gcc + (<= 4.0) will possibly no longer work without hand-hacking the file + to remove the flags not understood by that version of the compiler. + Nothing should change when building with gcc >= 4.1. - The file locking calls used in HDF5 1.10.0 (including patch1) - will fail when the underlying file system does not support file - locking or where locks have been disabled. To disable all file - locking operations, an environment variable named - HDF5_USE_FILE_LOCKING can be set to the five-character string - 'FALSE'. This does not fundamentally change HDF5 library - operation (aside from initial file open/create, SWMR is lock-free), - but users will have to be more careful about opening files - to avoid problematic access patterns (i.e.: multiple writers) - that the file locking was designed to prevent. + (DER - 2017/05/31, HDFFV-9937) - Additionally, the error message that is emitted when file lock - operations set errno to ENOSYS (typical when file locking has been - disabled) has been updated to describe the problem and potential - resolution better. + - -fno-omit-frame-pointer was added when building with debugging symbols + enabled. - (DER, 2016/10/26, HDFFV-9918) + Debugging symbols can be enabled independently of the overall build + mode in both the autotools and CMake. This allows (limited) debugging + of optimized code. Since many debuggers rely on the frame pointer, + we've disabled this optimization when debugging symbols are requested + (e.g.: via building with --enable-symbols). - - The return type of H5Pget_driver_info() has been changed from void * - to const void *. + (DER - 2017/05/31, HDFFV-10226) - The pointer returned by this function points to internal library - memory and should not be freed by the user. - (DER, 2016/11/04, HDFFV-10017) + Library: + -------- + - Added an enumerated value to H5F_libver_t for H5Pset_libver_bounds(). - - The direct I/O VFD has been removed from the list of VFDs that - support SWMR. + Currently, the library defines two values for H5F_libver_t and supports + only two pairs of (low, high) combinations as derived from these values. + Thus the bounds setting via H5Pset_libver_bounds() is rather restricted. - This configuration was never officially tested and several SWMR - tests fail when this VFD is set. + Added an enumerated value (H5F_LIBVER_V18) to H5F_libver_t and + H5Pset_libver_bounds() now supports five pairs of (low, high) combinations + as derived from these values. This addition provides the user more + flexibility in setting bounds for object creation. - (DER, 2016/11/03, HDFFV-10169) + (VC - 2018/03/14) - Configuration: - -------------- - - The minimum version of CMake required to build HDF5 is now 3.2.2. + - Added prefix option to VDS files. - (ADB, 2017/01/10) + Currently, VDS source files must be in the active directory to be + found by the virtual file. Adding the option of a prefix to be set + on the virtual file, using a data access property list (DAPL), + allows the source files to locate at an absolute or relative path + to the virtual file. + Private utility functions in H5D and H5L packages merged into single + function in H5F package. - - An --enable/disable-developer-warnings option has been added to - configure. + New public APIs: + herr_t H5Pset_virtual_prefix(hid_t dapl_id, const char* prefix); + ssize_t H5Pget_virtual_prefix(hid_t dapl_id, char* prefix /*out*/, size_t size); + The prefix can also be set with an environment variable, HDF5_VDS_PREFIX. - This disables warnings that do not indicate poor code quality such - as -Winline and gcc's -Wsuggest-attribute. Developer warnings are - disabled by default. + (ADB - 2017/12/12, HDFFV-9724, HDFFV-10361) - (DER, 2017/01/10) + - H5FDdriver_query() API call added to the C library. - - A bin/restore.sh script was added that reverts autogen.sh processing. + This new library call allows the user to query a virtual file driver + (VFD) for the feature flags it supports (listed in H5FDpublic.h). + This can be useful to determine if a VFD supports SWMR, for example. - (DER, 2016/11/08) + Note that some VFDs have feature flags that may only be present + after a file has been created or opened (e.g.: the core VFD will + have the H5FD_FEAT_POSIX_COMPAT_HANDLE flag set if the backing + store is switched on). Since the new API call queries a generic VFD + unassociated with a file, these flags will never be returned. - - CMake: Added NAMESPACE hdf5:: to package configuration files to allow - projects using installed HDF5 binaries built with CMake to link with - them without specifying the HDF5 library location via IMPORTED_LOCATION. + (DER - 2017/05/31, HDFFV-10215) - (ABD, 2016/10/17, HDFFV-10003) + - H5FD_FEAT_DEFAULT_VFD_COMPATIBLE VFD feature flag added to the C library. - - CMake: Changed the CTEST_BUILD_CONFIGURATION option to - CTEST_CONFIGURATION_TYPE as recommended by the CMake documentation. + This new feature flag indicates that the VFD is compatible with the + default VFD. VFDs that set this flag create single files that follow + the canonical HDF5 file format. - (ABD, 2016/10/17, HDFFV-9971) - + (DER - 2017/05/31, HDFFV-10214) - Fortran Library: - ---------------- + - The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h) + has been marked as deprectated. + + This ID type value is not used in the C library. i.e.: There are no + hid_t values that are of ID type H5I_REFERENCE. + + This enum value will be removed in a future major version of the library. + The code will remain unchanged in the HDF5 1.10.x releases and branches. + + (DER - 2017/04/05, HDFFV-10252) + + + Parallel Library: + ----------------- + - Enabled compression for parallel applications. + + With this release parallel applications can create and write compressed + datasets (or the datasets with the filters such as Fletcher32 applied). + + (EIP - 2018/03/29) + + - Addressed slow file close on some Lustre file systems. + + Slow file close has been reported on some Lustre file systems. + While the ultimate cause is not understood fully, the proximate + cause appears to be long delays in MPI_File_set_size() calls at + file close and flush. + + To minimize this problem pending a definitive diagnosis and fix, + PHDF5 has been modified to avoid MPI_File_set_size() calls when + possible. This is done by comparing the library's EOA (End of + Allocation) with the file systems EOF, and skipping the + MPI_File_set_size() call if the two match. + + (JRM - 2018/03/29) - - The HDF5 Fortran library can now be compiled with the NAG compiler. + - Optimized parallel open/location of the HDF5 super-block. - (MSB, 2017/2/10, HDFFV-9973) + Previous releases of PHDF5 required all parallel ranks to + search for the HDF5 superblock signature when opening the + file. As this is accomplished more or less as a synchronous + operation, a large number of processes can experience a + slowdown in the file open due to filesystem contention. + + As a first step in improving the startup/file-open performance, + we allow MPI rank 0 of the associated MPI communicator to locate + the base offset of the super-block and then broadcast that result + to the remaining ranks in the parallel group. Note that this + approach is utilized ONLY during file opens which employ the MPIO + file driver in HDF5 by previously having called H5Pset_fapl_mpio(). + + HDF5 parallel file operations which do not employ multiple ranks + e.g. specifiying MPI_COMM_SELF (whose MPI_Comm_size == 1) + as opposed to MPI_COMM_WORLD, will not be affected by this + optimization. Conversely, parallel file operations on subgroups + of MPI_COMM_WORLD are allowed to be run in parallel with each + subgroup operating as an independant collection of processes. + + (RAW - 2017/10/10, HDFFV-10294) + + - Added large (>2GB) MPI-IO transfers. + + Previous releases of PHDF5 would fail when attempting to + read or write greater than 2GB of data in a single IO operation. + This issue stems principally from an MPI API whose definitions + utilize 32 bit integers to describe the number of data elements + and datatype that MPI should use to effect a data transfer. + Historically, HDF5 has invoked MPI-IO with the number of + elements in a contiguous buffer represented as the length + of that buffer in bytes. + + Resolving the issue and thus enabling larger MPI-IO transfers + is accomplished first, by detecting when a user IO request would + exceed the 2GB limit as described above. Once a transfer request + is identified as requiring special handling, PHDF5 now creates a + derived datatype consisting of a vector of fixed sized blocks + which is in turn wrapped within a single MPI_Type_struct to + contain the vector and any remaining data. The newly created + datatype is then used in place of MPI_BYTE and can be used to + fulfill the original user request without encountering API + errors. + + (RAW - 2017/09/10, HDFFV-8839) C++ Library: ------------ - - The following C++ API wrappers have been added to the C++ Library: + + H5Lcreate_soft: + // Creates a soft link from link_name to target_name. + void link(const char *target_name, const char *link_name,...) + void link(const H5std_string& target_name,...) - // Sets/Gets the strategy and the threshold value that the library - // will employ in managing file space. - FileCreatPropList::setFileSpaceStrategy - H5Pset_file_space_strategy - FileCreatPropList::getFileSpaceStrategy - H5Pget_file_space_strategy + + H5Lcreate_hard: + // Creates a hard link from new_name to curr_name. + void link(const char *curr_name, const Group& new_loc,...) + void link(const H5std_string& curr_name, const Group& new_loc,...) - // Sets/Gets the file space page size for paged aggregation. - FileCreatPropList::setFileSpacePagesize - H5Pset_file_space_page_size - FileCreatPropList::getFileSpacePagesize - H5Pget_file_space_page_size + // Creates a hard link from new_name to curr_name in same location. + void link(const char *curr_name, const hid_t same_loc,...) + void link(const H5std_string& curr_name, const hid_t same_loc,...) - // Checks if the given ID is valid. - IdComponent::isValid - H5Iis_valid + Note: previous version of H5Location::link will be deprecated. - // Sets/Gets the number of soft or user-defined links that can be - // traversed before a failure occurs. - LinkAccPropList::setNumLinks - H5Pset_nlinks - LinkAccPropList::getNumLinks - H5Pget_nlinks + + H5Lcopy: + // Copy an object from a group of file to another. + void copyLink(const char *src_name, const Group& dst,...) + void copyLink(const H5std_string& src_name, const Group& dst,...) - // Returns a copy of the creation property list of a datatype. - DataType::getCreatePlist - H5Tget_create_plist + // Copy an object from a group of file to the same location. + void copyLink(const char *src_name, const char *dst_name,...) + void copyLink(const H5std_string& src_name,...) - // Opens/Closes an object within a group or a file, regardless of object - // type - Group::getObjId - H5Oopen - Group::closeObjId - H5Oclose + + H5Lmove: + // Rename an object in a group or file to a new location. + void moveLink(const char* src_name, const Group& dst,...) + void moveLink(const H5std_string& src_name, const Group& dst,...) - // Maps elements of a virtual dataset to elements of the source dataset. - DSetCreatPropList::setVirtual - H5Pset_virtual + // Rename an object in a group or file to the same location. + void moveLink(const char* src_name, const char* dst_name,...) + void moveLink(const H5std_string& src_name,...) - // Gets general information about this file. - H5File::getFileInfo - H5Fget_info2 + Note: previous version H5Location::move will be deprecated. - // Returns the number of members in a type. - IdComponent::getNumMembers - H5Inmembers + + H5Ldelete: + // Removes the specified link from this location. + void unlink(const char *link_name, + const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) + void unlink(const H5std_string& link_name, + const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) - // Determines if an element type exists. - IdComponent::typeExists - H5Itype_exists + Note: additional parameter is added to previous H5Location::unlink. - // Determines if an object exists. - H5Location::exists - H5Lexists. + + H5Tencode and H5Tdecode: + // Creates a binary object description of this datatype. + void DataType::encode() - C API H5Tencode() - // Returns the header version of an HDF5 object. - H5Object::objVersion - H5Oget_info for version + // Returns the decoded type from the binary object description. + DataType::decode() - C API H5Tdecode() + ArrayType::decode() - C API H5Tdecode() + CompType::decode() - C API H5Tdecode() + DataType::decode() - C API H5Tdecode() + EnumType::decode() - C API H5Tdecode() + FloatType::decode() - C API H5Tdecode() + IntType::decode() - C API H5Tdecode() + StrType::decode() - C API H5Tdecode() + VarLenType::decode() - C API H5Tdecode() - (BMR, 2017/03/20, HDFFV-10004, HDFFV-10139, HDFFV-10145) + + H5Lget_info: + // Returns the information of the named link. + H5L_info_t getLinkInfo(const H5std_string& link_name,...) - - New exception: ObjHeaderIException for H5O interface. + (BMR - 2018/03/11, HDFFV-10149) - (BMR, 2017/03/15, HDFFV-10145) + - Added class LinkCreatPropList for link create property list. - - New class LinkAccPropList for link access property list, to be used by - wrappers of H5Lexists. + (BMR - 2018/03/11, HDFFV-10149) - (BMR, 2017/01/04, HDFFV-10145) + - Added overloaded functions H5Location::createGroup to take a link + creation property list. + Group createGroup(const char* name, const LinkCreatPropList& lcpl) + Group createGroup(const H5std_string& name, const LinkCreatPropList& lcpl) - - New constructors to open datatypes in ArrayType, CompType, DataType, - EnumType, FloatType, IntType, StrType, and VarLenType. - - (BMR, 2016/12/26, HDFFV-10056) + (BMR - 2018/03/11, HDFFV-10149) - - New member functions: + - A document is added to the HDF5 C++ API Reference Manual to show the + mapping from a C API to C++ wrappers. It can be found from the main + page of the C++ API Reference Manual. - DSetCreatPropList::setNbit() to setup N-bit compression for a dataset. + (BMR - 2017/10/17, HDFFV-10151) - ArrayType::getArrayNDims() const - ArrayType::getArrayDims() const - both to replace the non-const versions. - (BMR, 2016/04/25, HDFFV-8623, HDFFV-9725) + Java Library: + ---------------- + - Wrapper added for enabling the error stack. + H5error_off would disable the error stack reporting. In order + to re-enable the reporting, the error stack info needs to be + saved so that H5error_on can revert state. - Tools: - ------ - - The following options have been added to h5clear: - -s: clear the status_flags field in the file's superblock - -m: Remove the metadata cache image from the file + (ADB - 2018/03/13, HDFFV-10412) - (QAK, 2017/03/22, PR#361) + - Wrappers were added for the following C APIs: + H5Pset_evict_on_close + H5Pget_evict_on_close + H5Pset_chunk_opts + H5Pget_chunk_opts + H5Pset_efile_prefix + H5Pget_efile_prefix + H5Pset_virtual_prefix + H5Pget_virtual_prefix + (ADB - 2017/12/20) - High-Level APIs: - --------------- - - Added New Fortran 2003 API for h5tbmake_table_f. + - The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h) + has been marked as deprectated. + + JNI code which refers to this value will be removed in a future + major version of the library. The code will remain unchanged in the + 1.10.x releases and branches. + + See the C library section, above, for further information. - (MSB, 2017/02/10, HDFFV-8486) + (HDFFV-10252, DER, 2017/04/05) + Tools: + ------ + - h5diff has a new option to display error stack. -Support for New Platforms, Languages, and Compilers -=================================================== + Updated h5diff with the --enable-error-stack argument, which + enables the display of the hdf5 error stack. This completes the + improvement to the main tools: h5copy, h5diff, h5dump, h5ls and + h5repack. - - Added NAG compiler + (ADB - 2017/08/30, HDFFV-9774) +Support for new platforms, languages and compilers. +======================================= + - None -Bug Fixes since HDF5-1.10.0-patch1 release +Bug Fixes since HDF5-1.10.1 release ================================== Library ------- - - Outdated data structure was used in H5D_CHUNK_DEBUG blocks, causing - compilation errors when H5D_CHUNK_DEBUG was defined. This is fixed. + - The data read after a direct chunk write to a chunked dataset with + one chunk was incorrect. + + The problem was due to the passing of a null dataset pointer to + the insert callback for the chunk index in the routine + H5D__chunk_direct_write() in H5Dchunk.c + The dataset was a single-chunked dataset which will use the + single chunk index when latest format was enabled on file creation. + The single chunk index was the only index that used this pointer + in the insert callback. - (BMR, 2017/04/04, HDFFV-8089) + Passed the dataset pointer to the insert callback for the chunk + index in H5D__chunk_direct_write(). - - SWMR implementation in the HDF5 1.10.0 and 1.10.0-patch1 releases has a - broken metadata flush dependency that manifested itself with the following - error at the end of the HDF5 error stack: + (VC - 2018/03/20, HDFFV-10425) - H5Dint.c line 846 in H5D__swmr_setup(): dataspace chunk index must be 0 - for SWMR access, chunkno = 1 - major: Dataset - minor: Bad value + - Added public routine H5DOread_chunk to the high-level C library. - It was also reported at https://github.com/areaDetector/ADCore/issues/203 + The patch for H5DOwrite_chunk() to write an entire chunk to the file + directly was contributed by GE Healthcare and integrated by The HDF Group + developers. + + (VC - 2017/05/19, HDFFV-9934) - The flush dependency is fixed in this release. + - Freeing of object header after failed checksum verification. - - Changed the plugins dlopen option from RTLD_NOW to RTLD_LAZY + It was discovered that the object header (in H5Ocache.c) was not released properly + when the checksum verification failed and a re-load of the object + header was needed. - (ABD, 2016/12/12, PR#201) + Freed the object header that failed the chksum verification only + after the new object header is reloaded, deserialized and set up. - - A number of issues were fixed when reading/writing from/to corrupted - files to ensure that the library fails gracefully in these cases: + (VC - 2018/03/14, HDFFV-10209) - * Writing to a corrupted file that has an object message which is - incorrectly marked as sharable on disk results in a buffer overflow / - invalid write instead of a clean error message. + - Updated H5Pset_evict_on_close in H5Pfapl.c - * Decoding data from a corrupted file with a dataset encoded with the - H5Z_NBIT decoding can result in a code execution vulnerability under - the context of the application using the HDF5 library. + Changed the minor error number from H5E_CANTSET to H5E_UNSUPPORTED for + parallel library. - * When decoding an array datatype from a corrupted file, the HDF5 library - fails to return an error in production if the number of dimensions - decoded is greater than the maximum rank. + (ADB - 2018/03/06, HDFFV-10414) - * When decoding an "old style" array datatype from a corrupted file, the - HDF5 library fails to return an error in production if the number of - dimensions decoded is greater than the maximum rank. + - Fixed the problems with the utility function that could not handle lowercase + Windows drive letters. - (NAF, 2016/10/06, HDFFV-9950, HDFFV-9951, HDFFV-9992, HDFFV-9993) + Added call to upper function for drive letter. - - Fixed an error that would occur when copying an object with an attribute - which is a compound datatype consisting of a variable length string. + (ADB - 2017/12/18, HDFFV-10307) - (VC, 2016/08/24, HDFFV-7991) + - Fixed H5Sencode() bug when the number of elements selected was > 2^32. - - H5DOappend will no longer fail if a dataset has no append callback - registered. + H5Sencode() incorrectly encodes dataspace selection with number of + elements exceeding 2^32. When decoding such selection via H5Sdecode(), + the number of elements in the decoded dataspace is not the same as + what is encoded. This problem exists for H5S_SEL_HYPER and + H5S_SEL_POINTS encoding. - (VC, 2016/08/14, HDFFV-9960) + The cause of the problem is due to the fact that the library uses 32 bits to + encode counts and block offsets for the selection. + The solution is to use the original 32 bit encodings if possible, + but use a different way to encode selection if more that 32 bits is needed. + See details in the RFC: H5Sencode/H5Sdecode Format Change i + https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5_Library/H5SencodeFormatChange. - - Fixed an issue where H5Pset_alignment could result in misaligned blocks - with some input combinations, causing an assertion failure in debug mode. + (VC - 2017/11/28, HDFFV-9947) - (NAF, 2016/08/11, HDFFV-9948) + - Fixed filter plugin handling in H5PL.c and H5Z.c to not require i availability of + dependent libraries (e.g., szip or zlib). - - Fixed a problem where a plugin compiled into a DLL in the default plugin - directory could not be found by the HDF5 library at runtime on Windows - when the HDF5_PLUGIN_PATH environment variable was not set. + It was discovered that the dynamic loading process used by + filter plugins had issues with library dependencies. - (ABD, 2016/08/01, HDFFV-9706) + CMake build process changed to use LINK INTERFACE keywords, which + allowed HDF5 C library to make dependent libraries private. The + filter plugin libraries no longer require dependent libraries + (such as szip or zlib) to be available. + + (ADB - 2017/11/16, HDFFV-10328) - - Fixed an error that would occur when calling H5Adelete on an attribute - which is attached to an externally linked object in the target file and - whose datatype is a committed datatype in the main file. + - Fixed rare object header corruption bug. - (VC, 2016/07/06, HDFFV-9940) + In certain cases, such as when converting large attributes to dense + storage, an error could occur which would either fail an assertion or + cause file corruption. Fixed and added test. - - (a) Throw an error instead of assertion when v1 btree level hits the 1 - byte limit. - (b) Modifications to better handle error recovery when conversion by - h5format_convert fails. + (NAF - 2017/11/14, HDFFV-10274) - (VC, 2016/05/29, HDFFV-9434) + - Updated H5Zfilter_avail in H5Z.c. - - Fixed a memory leak where an array used by the library to track SWMR - read retries was unfreed. + The public function checked for plugins, while the private + function did not. - The leaked memory was small (on the order of a few tens of ints) and - allocated per-file. The memory was allocated (and lost) only when a - file was opened for SWMR access. + Modified H5Zfilter_avail and private function, H5Z_filter_avail. + Moved check for plugin from public to private function. Updated + H5P__set_filter due to change in H5Z_filter_avail. Updated tests. - (DER, 2016/04/27, HDFFV-9786) + (ADB - 2017/10/10, HDFFV-10297, HDFFV-10319) - - Fixed a memory leak that could occur when opening a file for the first - time (including creating) and the call fails. + - h5dump produced SEGFAULT when dumping corrypted file. + + The behavior was due to the error in the internal function H5HL_offset_into(). - This occurred when the file-driver-specific info was not cleaned up. - The amount of memory leaked varied with the file driver, but would - normally be less than 1 kB. + (1) Fixed H5HL_offset_into() to return error when offset exceeds heap data + block size. + (2) Fixed other places in the library that call this routine to detect + error routine. - (DER, 2016/12/06, HDFFV-10168) + (VC - 2017/08/30, HDFFV-10216) - - Fixed a failure in collective metadata writes. + - Fixes for paged aggregation feature. - This failure only appeared when collective metadata writes - were enabled (via H5Pset_coll_metadata_write()). + Skip test in test/fheap.c when: + (1) multi/split drivers and + (2) persisting free-space or using paged aggregation strategy - (JRM, 2017/04/10, HDFFV-10055) + (VC, 2017/07/10) + Changes made based on RFC review comments: + (1) Added maximum value for file space page size + (2) Dropped check for page end metadata threshold + (3) Removed "can_shrink" and "shrink" callbacks for small section class - Parallel Library - ---------------- - - Fixed a bug that could occur when allocating a chunked dataset in parallel - with an alignment set and an alignment threshold greater than the chunk - size but less than or equal to the raw data aggregator size. + (VC - 2017/06/09) + + - Fixed for infinite loop in H5VM_power2up(). + + The function H5VM_power2up() returns the next power of 2 + for n. When n exceeds 2^63, it overflows and becomes 0 causing + the infinite looping. + + The fix ensures that the function checks for n >= 2^63 + and returns 0. + + (VC - 2017/07/10, HDFFV-10217) - (NAF, 2016/08/11, HDFFV-9969) + - Fixed for H5Ocopy doesn't work with open identifiers. + + Changes made so that raw data for dataset objects are copied from + cached info when possible instead of flushing objects to file and + read them back in again. + + (VC - 2017/07/05, HDFFV-7853) + + - An uninitialized struct could cause a memory access error when using + variable-length or reference types in a compressed, chunked dataset. + + A struct containing a callback function pointer and a pointer to some + associated data was used before initialization. This could cause a + memory access error and system crash. This could only occur under + unusual conditions when using variable-lenth and reference types in + a compressed, chunked dataset. + + On recent versions of Visual Studio, when built in debug mode, the + debug heap will complain and cause a crash if the code in question + is executed (this will cause the objcopy test to fail). + + (DER - 2017/11/21, HDFFV-10330) + + - Fixed collective metadata writes on file close. + + It was discovered that metadata was being written twice as part of + the parallel file close behavior, once independently and once + collectively. + + A fix for this error was included as part of the parallel compression + feature but remained undocumented here. + + (RAW - 2017/12/01, HDFFV-10272) + + - If an HDF5 file contains a filter pipeline message with a 'number of + filters' field that exceeds the maximum number of allowed filters, + the error handling code will attempt to dereference a NULL pointer. + + This issue was reported to The HDF Group as issue #CVE-2017-17505. + https://security-tracker.debian.org/tracker/CVE-2017-17505 + https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17505 + + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. + + This problem arose because the error handling code assumed that + the 'number of filters' field implied that a dynamic array of that + size had already been created and that the cleanup code should + iterate over that array and clean up each element's resources. If + an error occurred before the array has been allocated, this will + not be true. + + This has been changed so that the number of filters is set to + zero on errors. Additionally, the filter array traversal in the + error handling code now requires that the filter array not be NULL. + + (DER - 2018/02/06, HDFFV-10354) + + - If an HDF5 file contains a filter pipeline message which contains + a 'number of filters' field that exceeds the actual number of + filters in the message, the HDF5 C library will read off the end of + the read buffer. + + This issue was reported to The HDF Group as issue #CVE-2017-17506. + https://security-tracker.debian.org/tracker/CVE-2017-17506 + https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17506 + + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. + + The problem was fixed by passing the buffer size with the buffer + and ensuring that the pointer cannot be incremented off the end + of the buffer. A mismatch between the number of filters declared + and the actual number of filters will now invoke normal HDF5 + error handling. + + (DER - 2018/02/26, HDFFV-10355) + + - If an HDF5 file contains a malformed compound datatype with a + suitably large offset, the type conversion code can run off + the end of the type conversion buffer, causing a segmentation + fault. + + This issue was reported to The HDF Group as issue #CVE-2017-17507. + https://security-tracker.debian.org/tracker/CVE-2017-17506 + https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17506 + + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. + + THE HDF GROUP WILL NOT FIX THIS BUG AT THIS TIME + + Fixing this problem would involve updating the publicly visible + H5T_conv_t function pointer typedef and versioning the API calls + which use it. We normally only modify the public API during + major releases, so this bug will not be fixed at this time. + + (DER - 2018/02/26, HDFFV-10356) + + - If an HDF5 file contains a malformed compound type which contains + a member of size zero, a division by zero error will occur while + processing the type. + + This issue was reported to The HDF Group as issue #CVE-2017-17508. + https://security-tracker.debian.org/tracker/CVE-2017-17508 + https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17508 + + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. + + Checking for zero before dividing fixes the problem. Instead of the + division by zero, the normal HDF5 error handling is invoked. + + (DER - 2018/02/26, HDFFV-10357) + + - If an HDF5 file contains a malformed symbol table node that declares + it contains more symbols than it actually contains, the library + can run off the end of the metadata cache buffer while processing + the symbol table node. + + This issue was reported to The HDF Group as issue #CVE-2017-17509. + https://security-tracker.debian.org/tracker/CVE-2017-17509 + https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17509 + + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. + + Performing bounds checks on the buffer while processing fixes the + problem. Instead of the segmentation fault, the normal HDF5 error + handling is invoked. + + (DER - 2018/03/12, HDFFV-10358) + + - Fixed permissions passed to open(2) on file create. + + On Windows, the POSIX permissions passed to open(2) when creating files + were only incidentally correct. They are now set to the correct value of + (_S_IREAD | _S_IWRITE). + + On other platforms, the permissions were set to a mix of 666, 644, and + 000. They are now set uniformly to 666. + + (DER - 2017/04/28, HDFFV-9877) + + - The H5FD_FEAT_POSIX_COMPAT_HANDLE flag is no longer used to determine + if a virtual file driver (VFD) is compatible with SWMR. + + Use of this VFD feature flag was not in line with the documentation in + the public H5FDpublich.h file. In particular, it was being used as a + proxy for determining if SWMR I/O is allowed. This is unecessary as we + already have a feature flag for this (H5FD_SUPPORTS_SWMR_IO). + + (DER - 2017/05/31, HDFFV-10214) Configuration - ------------- - - Configuration will check for the strtoll and strtoull functions - before using alternatives + ------------- + - CMake changes - (ABD, 2017/03/17, PR#340) + - Updated CMake commands configuration. - - CMake uses a Windows pdb directory variable if available and - will generate both static and shared pdb files. + A number of improvements were made to the CMake commands. Most + changes simplify usage or eliminate unused constructs. Also, + some changes support better cross-platform support. - (ABD, 2017/02/06, HDFFV-9875) + (ADB - 2018/02/01, HDFFV-10398) - - CMake now builds shared versions of tools. + - Corrected usage of CMAKE_BUILD_TYPE variable. - (ABD, 2017/02/01, HDFFV-10123) + The use of the CMAKE_BUILD_TYPE is incorrect for multi-config + generators (Visual Studio and XCode) and is optional for single + config generators. Created a new macro to check + GLOBAL PROPERTY -> GENERATOR_IS_MULTI_CONFIG + Created two new HDF variable, HDF_BUILD_TYPE and HDF_CFG_BUILD_TYPE. + Defaults for these variables is "Release". - - Makefiles and test scripts have been updated to correctly remove files - created when running "make check" and to avoid removing any files under - source control. In-source builds followed by "make clean" and "make - distclean" should result in the original source files. + (ADB - 2018/01/10, HDFFV-10385) - (LRK, 2017/01/17, HDFFV-10099) + - Added replacement of fortran flags if using static CRT. - - The tools directory has been divided into two separate source and test - directories. This resolves a build dependency and, as a result, - 'make check' will no longer fail in the tools directory if 'make' was - not executed first. - - (ABD, 2016/10/27, HDFFV-9719) + Added TARGET_STATIC_CRT_FLAGS call to HDFUseFortran.cmake file in + config/cmake_ext_mod folder. - - CMake: Fixed a timeout error that would occasionally occur when running - the virtual file driver tests simultaneously due to test directory - and file name collisions. + (ADB - 2018/01/08, HDFFV-10334) - (ABD, 2016/09/19, HDFFV-9431) - - CMake: Fixed a command length overflow error by converting custom - commands inside CMakeTest.cmake files into regular dependencies and - targets. + - The hdf5 library used shared szip and zlib, which needlessly required + applications to link with the same szip and zlib libraries. - (ABD, 2016/07/12, HDFFV-9939) + Changed the target_link_libraries commands to use the static libs. + Removed improper link duplication of szip and zlib. + Adjusted the link dependencies and the link interface values of + the target_link_libraries commands. - - Fixed a problem preventing HDF5 to be built on 32-bit CYGWIN by - condensing cygwin configuration files into a single file and - removing outdated compiler settings. + (ADB - 2017/11/14, HDFFV-10329) - (ABD, 2016/07/12, HDFFV-9946) + - CMake MPI + CMake implementation for MPI was problematic and would create incorrect + MPI library references in the hdf5 libraries. - Fortran - -------- - - Changed H5S_ALL_F from INTEGER to INTEGER(HID_T) + Reworked the CMake MPI code to properly create CMake targets. Also merged + the latest CMake FindMPI.cmake changes to the local copy. This is necessary + until HDF changes the CMake minimum to 3.9 or greater. + + (ADB - 2017/11/02, HDFFV-10321) + + - Corrected FORTRAN_HAVE_C_LONG_DOUBLE processing in the autotools. - (MSB, 2016/10/14, HDFFV-9987) + A bug in the autotools Fortran processing code always set the + FORTRAN_HAVE_C_LONG_DOUBLE variable to be true regardless of + whether or not a C long double type was present. + This would cause compilation failures on platforms where a C + long double type was not available and the Fortran wrappers + were being built. + + (DER - 2017/07/05, HDFFV-10247) + + - The deprecated --enable-production and --enable-debug configure options + failed to emit errors when passed an empty string + (e.g.: --enable-debug=""). + + Due to the way we checked for these options being set, it was possible + to avoid the error message and continue configuration if an empty string + was passed to the option. + + Any use of --enable-production or --enable-debug will now halt the + configuration step and emit a helpful error message + (use --enable-build-mode=debug|production instead). + + (DER - 2017/07/05, HDFFV-10248) + + - CMake + + Too many commands for POST_BUILD step caused command line to be + too big on windows. + + Changed foreach of copy command to use a custom command with the + use of the HDFTEST_COPY_FILE macro. + + (ADB - 2017/07/12, HDFFV-10254) + + - CMake test execution environment + + The parallel HDF5 test: 't_pread' assumed the use of autotools + and the directory structure associated with that testing approach. + Modified the test code to check whether the 'h5jam' utility can be + found in the same directory as the test executable (which is + preferred directory structure utilized by cmake) and if found + will invoke the tool directly rather than utilizing a relative path. + + (RAW - 2017/11/03, HDFFV-10318) + + - Fortran compilation fails for xlf and CMake builds. + + Fixed CMake shared library build for H5match_types and modules + + (MSB - 2017/12/19, HDFFV-10363) + + - Shared libraries fail test on OSX with Fortran enabled with CMake. + + Fixed by removing the F77 use of EQUIVALENCE and COMMON, replaced + using MODULES. Updated CMake. + + (MSB - 2017/12/07, HDFFV-10223) + + - The bin/trace script now emits an error code on problems and autogen.sh + will fail if bin/trace fails. + + The bin/trace script adds tracing functionality to public HDF5 API calls. + It is only of interest to developers who modify the HDF5 source code. + Previously, bin/trace just wrote an error message to stdout when it + encountered problems, so autogen.sh processing did not halt and a broken + version of the library could be built. The script will now return an + error code when it encounters problems, and autogen.sh will fail. + + This only affects users who run autogen.sh to rebuild the Autotools files, + which is not necessary to build HDF5 from source in official releases of the + library. CMake users are unaffected as bin/trace is not run via CMake + at this time. + + (DER - 2017/04/25, HDFFV-10178) + + - FC_BASENAME was changed from gfortran40 to gfortran in a few places. + + In the autotools, FC_BASENAME was set to gfortran40 in a few locations + (config/gnu-fflags and config/freebsd). This was probably a historical + artifact and did not seem to affect many users. + + The value is now correctly set to gfortran. + + (DER - 2017/05/26, HDFFV-10249) + + - The ar flags were changed to -cr (was: -cru) + + The autotools set the flags for ar to -cru by default. The -u flag, + which allows selective replacement of only the members which have + changed, raises warnings on some platforms, so the flags are now set to + -cr via AR_FLAGS in configure.ac. This causes the static library to + always be completely recreated from the object files on each build. + + (DER - 2017/11/15, HDFFV-10428) + + + Fortran + -------- + - Fixed compilation errors when using Intel 18 Fortran compilers + (MSB - 2017/11/3, HDFFV-10322) Tools ----- - - h5diff now correctly ignores strpad in comparing strings. + - h5clear - (ABD, 2017/03/03, HDFFV-10128) + An enhancement to the tool in setting a file's stored EOA. - - h5repack now correctly parses the command line filter options. + It was discovered that a crashed file's stored EOA in the superblock + was smaller than the actual file's EOF. When the file was reopened + and closed, the library truncated the file to the stored EOA. - (ABD, 2017/01/24, HDFFV-10046) + Added an option to the tool in setting the file's stored EOA in the + superblock to the maximum of (EOA, EOF) + increment. + An option was also added to print the file's EOA and EOF. - - h5diff now correctly returns an error when it cannot read data due - to an unavailable filter plugin. + (VC - 2018/03/14, HDFFV-10360) - (ADB 2017/01/18, HDFFV-9994 ) + - h5repack - - Fixed an error in the compiler wrapper scripts (h5cc, h5fc, et al.) - in which they would erroneously drop the file argument specified via - the -o flag when the -o flag was specified before the -c flag on the - command line, resulting in a failure to compile. + h5repack changes the chunk parameters when a change of layout is not + specified and a filter is applied. - (LRK, 2016/11/04, HDFFV-9938, HDFFV-9530) + HDFFV-10297, HDFFV-10319 reworked code for h5repack and h5diff code + in the tools library. The check for an existing layout was incorrectly + placed into an if block and not executed. The check was moved into + the normal path of the function. - - h5repack User Defined (UD) filter parameters were not parsed correctly. + (ADB - 2018/02/21, HDFFV-10412) - The UD filter parameters were not being parsed correctly. Reworked coding - section to parse the correct values and verify number of parameters. + - h5dump - (ABD, 2016/10/19, HDFFV-9996, HDFFV-9974, HDFFV-9515, HDFFV-9039) + The tools library will hide the error stack during file open. - - h5repack allows the --enable-error-stack option on the command line. + While this is preferable almost always, there are reasons to enable + display of the error stack when a tool will not open a file. Adding an + optional argument to the --enable-error-stack will provide this use case. + As an optional argument it will not affect the operation of the + --enable-error-stack. h5dump is the only tool to implement this change. - (ADB, 2016/08/08, HDFFV-9775) + (ADB - 2018/02/15, HDFFV-10384) + - h5dump - C++ APIs - -------- - - The member function H5Location::getNumObjs() is moved to - class Group because the objects are in a group or a file only, - and H5Object::getNumAttrs to H5Location to get the number of - attributes at a given location. + h5dump would output an indented blank line in the filters section. + + h5dump overused the h5tools_simple_prefix function, which is a + function intended to account for the data index (x,y,z) option. + Removed the function call for header information. + + (ADB - 2018/01/25, HDFFV-10396) + + - h5repack + + h5repack incorrectly searched internal object table for name. + + h5repack would search the table of objects for a name, if the + name did not match it tried to determine if the name without a + leading slash would match. The logic was flawed! The table + stored names(paths) without a leading slash and did a strstr + of the table path to the name. + The assumption was that if there was a difference of one then + it was a match, however "pressure" would match "/pressure" as + well as "/pressure1", "/pressure2", etc. Changed logic to remove + any leading slash and then do a full compare of the name. + + (ADB - 2018/01/18, HDFFV-10393) + + - h5repack + + h5repack failed to handle command line parameters for customer filters. + + User defined filter parameter conversions would fail whenintegers were + represented on the command line with character string + larger then 9 characters. Increased local variable array for storing + the current command line parameter to prevent buffer overflows. + + (ADB - 2018/01/17, HDFFV-10392) + + - h5diff + + h5diff seg faulted if comparing VL strings against fixed strings. + + Reworked solution for HDFFV-8625 and HDFFV-8639. Implemented the check + for string objects of same type in the diff_can_type function by + adding an if(tclass1 == H5T_STRING) block. This "if block" moves the + same check that was added for attributes to this function, which is + used by all object types. This function handles complex type structures. + Also added a new test file in h5diffgenttest for testing this issue + and removed the temporary files used in the test scripts. - (BMR, 2017/03/17, PR#466) + (ADB - 2018/01/04, HDFFV-8745) - - Due to the change in the C API, the overloaded functions of - PropList::setProperty now need const for some arguments. They are - planned for deprecation and are replaced by new versions with proper - consts. + - h5repack - (BMR, 2017/03/17, PR#344) + h5repack failed to copy a dataset with existing filter. - - The high-level API Packet Table (PT) did not write data correctly when - the datatype is a compound type that has string type as one of the - members. This problem started in 1.8.15, after the fix of HDFFV-9042 - was applied, which caused the Packet Table to use native type to access - the data. It should be up to the application to specify whether the - buffer to be read into memory is in the machine's native architecture. - Thus, the PT is fixed to not use native type but to make a copy of the - user's provided datatype during creation or the packet table's datatype - during opening. If an application wishes to use native type to read the - data, then the application will request that. However, the Packet Table - doesn't provide a way to specify memory datatype in this release. This - feature will be available in future releases. + Reworked code for h5repack and h5diff code in the tools library. Added + improved error handling, cleanup of resources and checks of calls. + Modified H5Zfilter_avail and private function, H5Z_filter_avail. + Moved check for plugin from public to private function. Updated + H5P__set_filter due to change in H5Z_filter_avail. Updated tests. + Note, h5repack output display has changed to clarify the individual + steps of the repack process. The output indicates if an operation + applies to all objects. Lines with notation and no information + have been removed. - (BMR, 2016/10/27, HDFFV-9758) + (ADB - 2017/10/10, HDFFV-10297, HDFFV-10319) - - The obsolete macros H5_NO_NAMESPACE and H5_NO_STD have been removed from - the HDF5 C++ API library. + - h5repack - (BMR, 2016/10/23, HDFFV-9532) + h5repack always set the User Defined filter flag to H5Z_FLAG_MANDATORY. - - The problem where a user-defined function cannot access both, attribute - and dataset, using only one argument is now fixed. + Added another parameter to the 'UD=' option to set the flag by default + to '0' or H5Z_FLAG_MANDATORY, the other choice is '1' or H5Z_FLAG_OPTIONAL. - (BMR, 2016/10/11, HDFFV-9920) + (ADB - 2017/08/31, HDFFV-10269) - - In-memory array information, ArrayType::rank and - ArrayType::dimensions, were removed. This is an implementation - detail and should not affect applications. + - h5ls - (BMR, 2016/04/25, HDFFV-9725) + h5ls generated error on stack when it encountered a H5S_NULL + dataspace. + Adding checks for H5S_NULL before calling H5Sis_simple (located + in the h5tools_dump_mem function) fixed the issue. + + (ADB - 2017/08/17, HDFFV-10188) + + - h5repack + + Added tests to h5repack.sh.in to verify options added for paged + aggregation work as expected. + + (VC - 2017/08/03) + + - h5dump + + h5dump segfaulted on output of XML file. + + Function that escape'd strings used the full buffer length + instead of just the length of the replacement string in a + strncpy call. Using the correct length fixed the issue. + + (ADB - 2017/08/01, HDFFV-10256) + + - h5diff + + h5diff segfaulted on compare of a NULL variable length string. + + Improved h5diff compare of strings by adding a check for + NULL strings and setting the lengths to zero. + + (ADB - 2017/07/25, HDFFV-10246) + + - h5import + + h5import crashed trying to import data from a subset of a dataset. + + Improved h5import by adding the SUBSET keyword. h5import understands + to use the Count times the Block as the size of the dimensions. + Added INPUT_B_ORDER keyword to old-style configuration files. + The import from h5dump function expects the binary files to use native + types (FILE '-b' option) in the binary file. + + (ADB - 2017/06/15, HDFFV-10219) + + - h5repack + + h5repack did not maintain the creation order flag of the root + group. + + Improved h5repack by reading the creation order and applying the + flag to the new root group. Also added arguments to set the + order and index direction, which applies to the traversing of the + original file, on the command line. + + (ADB - 2017/05/26, HDFFV-8611) + + - h5diff + + h5diff failed to account for strpad type and null terminators + of char strings. Also, h5diff failed to account for string length + differences and would give a different result depending on file + order in the command line. + + Improved h5diff compare of strings and arrays by adding a check for + string lengths and if the strpad was null filled. + + (ADB - 2017/05/18, HDFFV-9055, HDFFV-10128) + + High-Level APIs: + ------ + - H5DOwrite_chunk() problems when overwriting an existing chunk with + no filters enabled. + + When overwriting chunks and no filters were being used, the library would + fail (when asserts are enabled, e.g. debug builds) or incorrectly + insert additional chunks instead of overwriting (when asserts are not + enabled, e.g. production builds). + + This has been fixed and a test was added to the hl/test_dset_opt test. + + (DER - 2017/05/11, HDFFV-10187) + + C++ APIs + -------- + - Removal of memory leaks. + + A private function was inadvertently called, causing memory leaks. This + is now fixed. + + (BMR - 2018/03/12 - User's reported in email) Testing ------- - - Fixed a problem that caused tests using SWMR to occasionally fail when - running "make check" using parallel make. + - Memory for three variables in testphdf5's coll_write_test was malloced + but not freed, leaking memory when running the test. + + The variables' memory is now freed. - (LRK, 2016/03/22, PR#338, PR#346, PR#358) + (LRK - 2018/03/12, HDFFV-10397) + + - Refactored the testpar/t_bigio.c test to include ALARM macros + + Changed the test to include the ALARM_ON and ALARM_OFF macros which + are intended to prevent nightly test hangs that have been observed + with this particular parallel test example. The code was also modified to + simplify status reporting (only from MPI rank 0) and additional + status checking added. + + (RAW - 2017/11/08, HDFFV-10301) Supported Platforms =================== - Linux 2.6.32-573.18.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 - (Red Hat 4.4.7-4) + Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) + #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) + (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) IBM XL C/C++ V13.1 IBM XL Fortran V15.1 - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) + Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 + (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) + Version 4.9.3, Version 5.2.0, Intel(R) C (icc), C++ (icpc), Fortran (icc) compilers: - Version 15.0.3.187 Build 20150407 + Version 17.0.0.098 Build 20160721 MPICH 3.1.4 compiled with GCC 4.9.3 SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc @@ -584,33 +1098,30 @@ Supported Platforms Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) Visual Studio 2013 w/ Intel Fortran 15 (cmake) Visual Studio 2015 w/ Intel Fortran 16 (cmake) + Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake) Visual Studio 2015 w/ MSMPI 8 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) + Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) gcc and gfortran compilers (GCC 5.4.0) (cmake and autotools) Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) + Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) gcc and gfortran compilers (GCC 5.4.0) (cmake and autotools) Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1 - 64-bit gfortran GNU Fortran (GCC) 4.8.2 - (swallow/kite) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (wren/quail) Intel icc/icpc/ifort version 15.0.3 - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 64-bit gfortran GNU Fortran (GCC) 4.9.2 (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3 from Xcode 7.3 + Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 16.0.2 + (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 + + Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42) + 64-bit gfortran GNU Fortran (GCC) 7.1.0 + (swallow/kite) Intel icc/icpc/ifort version 17.0.2 Tested Configuration Features Summary @@ -671,28 +1182,33 @@ Compiler versions for each platform are listed in the preceding More Tested Platforms ===================== - The following platforms are not supported but have been tested for this release. - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) + Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) #1 SMP x86_64 GNU/Linux compilers: (mayll/platypus) Version 4.4.7 20120313 - Version 4.8.4 + Version 4.9.3, 5.3.0, 6.2.0 PGI C, Fortran, C++ for 64-bit target on x86-64; - Version 16.10-0 + Version 17.10-0 Intel(R) C (icc), C++ (icpc), Fortran (icc) compilers: - Version 15.0.3.187 (Build 20150407) + Version 17.0.4.196 Build 20170411 MPICH 3.1.4 compiled with GCC 4.9.3 Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) + (jelly) with NAG Fortran Compiler Release 6.1(Tozai) + GCC Version 7.1.0 + OpenMPI 3.0.0-GCC-7.2.0-2.29 Intel(R) C (icc) and C++ (icpc) compilers - Version 15.0.3.187 (Build 20150407) + Version 17.0.0.098 Build 20160721 with NAG Fortran Compiler Release 6.1(Tozai) + Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 + #1 SMP x86_64 GNU/Linux + (moohan) + Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 (ostrich) and IBM XL Fortran for Linux, V15.1 @@ -703,16 +1219,16 @@ The following platforms are not supported but have been tested for this release. (cmake and autotools) Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 6.1.1 20160621 + gcc, g++ (GCC) 6.1.1 20160621 (Red Hat 6.1.1-3) - GNU Fortran (GCC) 6.1.1 20160621 + GNU Fortran (GCC) 6.1.1 20160621 (Red Hat 6.1.1-3) (cmake and autotools) Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux - gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2) + gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0 20160609 - GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) + GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0 20160609 (cmake and autotools) @@ -720,11 +1236,18 @@ The following platforms are not supported but have been tested for this release. Known Problems ============== - At present, metadata cache images may not be generated by parallel + At present, metadata cache images may not be generated by parallel applications. Parallel applications can read files with metadata cache images, but since this is a collective operation, a deadlock is possible if one or more processes do not participate. - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to + Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29: + testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) + t_shapesame (sscontig2) + t_pflush1/fails on exit + The first two tests fail attempting collective writes. + + Known problems in previous releases can be found in the HISTORY*.txt files + in the HDF5 source. Please report any new problems found to help@hdfgroup.org. + |