| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
(H5_DEBUG_BUILD)
tested on bb-8 with Serial and Parallel, debug and production builds.
|
|
|
|
|
|
|
|
|
|
| |
checks from metadata dxpls
- remove H5AC_ind_dxpl_id and use only H5AC_dxpl_id everywhere instead.
- remove flush_me_collectively flag from cache entries
- add a collective sanity check (MPI_Barrier) for every HDF5 API routine that could possibly touch the file. This is trigerred when the environment variable H5_COLL_API_SANITY_CHECK is set to a non 0 digit.
tested on BB-8 with serial and parallel.
|
|
|
|
|
|
|
| |
Tested on:
Ubuntu 15.10 (Linux 4.2.0 x86_64) gcc 5.2.1
serial only
(these changes have been in revise_chunks for a long time)
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor property list code to "deep copy" properties in the correct way,
retraining the rest of the library to copy & release things correctly. This
cleans up another batch of memory leaks, etc. within the library.
Tested on:
MacOSX/64 10.10.5 (amazon) w/serial & parallel
Linux/32 2.6.x (jam) w/serial & parallel
(h5committest forthcoming)
|
|
|
|
|
|
|
|
| |
Clean up H5FD interface, to align w/v3 metadata cache changes
Tested on:
MacOSX/64 10.10.3 (amazon) w/serial & parallel
Linux/32 2.6.* (jam) w/serial & parallel
|
| |
|
|
|
|
|
|
|
|
| |
and replace with driver feature flag(s).
Add an MPI atomicity feature flag and remove the macro from the library.
tested Parallel on Jam.
|
|
|
|
|
|
|
|
|
|
| |
Clean up EOF code within library and add 'mem_type' parameter to 'get_eof'
VFD callback, to avoid various ambiguous situations, particularly with the
multi VFD. (Supports changes for 'avoid_truncate' feature also)
Tested on:
MacOSX/64 10.10.1 (amazon) w/serial & parallel
h5committest forthcoming
|
|
|
|
|
|
|
|
|
|
|
| |
Make progress toward moving from DXPL IDs to property list structures
within the library. Also move the signature location code from the H5F
package to the H5FD package, where it's a better fit. Also, clean up some
more compiler warnings along the way.
Tested on:
Mac OSX/64 10.9.2 (amazon) w/C++, FORTRAN & parallel
(h5committest forthcoming)
|
|
|
|
|
|
|
|
|
| |
Begin process of migrating from using property list IDs internally to the
library to using the internal generic property list data structure.
Tested on:
Mac OSX/64 10.9.2 (amazon) w/C++, FORTRAN & parallel
(h5committest forthcoming)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove all traces of MPI-POSIX VFD and GPFS detection/code.
Remove remaining traces of stream VFD.
Remove testpar/t_posix_compliant test (it's not actually verifying anything).
Clean up H5D__mpio_opt_possible() further.
Moved environment variable that disables MPI collective operations into
MPI-IO VFD (instead of it being in src/H5S.c).
A few other small code cleanups.
Tested on:
Mac OSX/64 10.9.2 (amazon) w/parallel & serial
|
|
|
|
|
|
| |
https://jira.hdfgroup.uiuc.edu/browse/HDFFV-8203
tested: h5committest
|
|
|
|
|
|
|
|
| |
Clean up file driver property handling, along with some misc. warnings.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/debug, gcc 4.7.1, C++, FORTRAN & threadsafe
(h5committest upcoming)
|
|
|
|
|
|
|
|
| |
Merge "file image" changes from feature branch back to trunk.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug
(h5committest upcoming)
|
|
|
|
|
|
|
| |
opened with the MPI-IO VFD
Add test cases for these two routines
Jira issue HDFFV-7961
|
|
|
|
|
|
|
|
|
|
|
| |
Bring sec2 and log VFDs back into agreement. Correct several compiler
warnings. Also add some additional statistics to the log VFD and promote the
parameter for H5Pset_fapl_log from 'unsigned' to 'unsigned long long'.
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
FreeBSD/32 8.2 (loyalty) w/debug
FreeBSD/64 8.2 (freedom) w/debug
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix a few more problems when the user block is not aligned properly.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Switch from using a 'flags' parameter to the flush routine(s), to just
using a single 'closing' boolean parameter, since that's all we're doing with
the flags anyway and this makes things more obvious.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Final merge of changes from sblock_mdc branch back to trunk. The superblock is now managed by the metadata cache.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reference count
> 1.
Description: Added a new field 'app_count' to H5I_id_info_t struct, to track
the reference count on an id due to the application. the old 'count' field
tracks the total. Generally any id visible to the application gets placed
in app_count. Added app_ref boolean parameter to H5I_inc_ref, H5I_dec_ref,
H5I_register, H5I_clear_type, and a few other functions, to specify whether
the operation(s) being performed on the id(s) are due to the application
(TRUE) or not (FALSE). Test added for this case.
Tested: kagiso, smirom, linew (h5committest)
|
|
|
|
|
|
|
|
| |
Further cleanups & simplifications to prepare for next phase of work.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.5.1 (amazon)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add hash value for skip list string types, to reduce # of string
comparisons.
- Fixed bug with metadata/small data block aggregator adding size == 0
block into file free space list.
- Refactored metadata/small data block aggregator code into single set of
common routines.
- Changed block aggregator code to be smarter about releasing space in the
'other' block when the 'other' block has aggregated enough data.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
|
|
|
|
|
|
|
|
|
|
| |
Clean up some of the file space allocation formatting.
Beef up the attribute creation order tests to make certain they are
handling shared atributes correctly (which they apparently are :-)
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
|
|
|
|
|
|
|
|
|
| |
copyright notice.
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
|
|
|
|
|
|
|
| |
The problem
came from the EOA for the whole MULTI file. It's taken out because it's
meaningless for MULTI file. Instead, each individual file has its EOA.
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up various warnings reported by the Windows team.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Search free list when checking for, or actually performing, extension of
an object in the file.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Solaris 2.9 (shanti)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Allow global heap collections to grow in size (up to a 64K limit) if they
are able to. This allows them to grow to a more reasonable size than the 4K
minimum size.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/optimization
Description:
Query property list values once, at the beginning of the I/O routines,
instead of querying the property list values multiple (lots!) of times in
lower level routines.
Solution:
Create "property list caches" for internal library queries of the property
list values.
Platforms tested:
IBM p690 (copper) w/parallel & fphdf5
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix/optimization
Description:
Address slowdown in MPI-I/O file metadata operations that was introduced
mid-stream. We now _require_ a POSIX compliant parallel file system for the
MPI-I/O file driver (as well as for the MPI-POSIX file driver).
Also optimized file open operation when the file is being created by
reducing the number of collective & syncronizing calls.
Additionally, refactor the MPI routines into a common place, eliminating
duplicated code.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Warning cleanup
Description:
Make a parameter 'const' to clean up a warning during compiles.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Refactored handlier of VFL drivers in file access and data transfer property
lists in order to simplify and unify the code dealing with them.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add feature
Description:
Add H5Fget_freespace() routine, to check the amount of free space in a
file. This information is only valid until the file is closed currently,
however (until we start recording the free space information in the file
itself).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
The VFL driver ID in a file's access proprty list wasn't being reference
counted correctly, causing the VFL driver to get prematurely closed after
several calls to "H5Pget_access_plist->H5Pclose".
Solution:
Increment VFL driver ID reference count when copy of file's access property
list is made in H5Pget_access_plist()
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactoring code
Description:
Refactored "IS_H5FD_MPIO || IS_H5FD_MPIPOSIX || IS_H5FD_FPHDF5" combination
of macros in many places into single IS_H5FD_MPI macro, which has the same
definition, but should be easier to maintain.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Finish code cleanup
Description:
Wrap up the conversion of H5F_flush's multiple boolean flags into a single
bitfield of flags by pushing the flags down into the H5AC_flush and
H5F_istore_flush routines.
Also, changed the flags from H5_FLUSH_<foo> to H5F_FLUSH_<foo> to be more
consistent with rest of library.
And reverted the changes to H5FDflush and H5FD_flush routines.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Solaris 5.8 (sol)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactoring of Flush Logic
Description:
The Flushing logic passed in multiple flags to indicate what type of
flush it was: closing, invalidate, alloc only. This made extending
the function of the flush logic to handle other flags annoying.
Solution:
I changed it to be just one bitmasked flag to indicate what type of
flushing to do. I also added the CLEAR_ONLY flag, which will be used
in the FPHDF5 stuff.
Platforms tested:
h5committest doesn't work for me (my environment isn't setup on the
other machines I guess). I tested it manually:
Linux parallel & C++
Sol Fortran
Modi4 parallel & Fortran.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Big change to the H5FD.c module:
- Split apart H5FD_alloc and H5FD_free. H5FD_alloc was huge and
H5FD_free had a freeing of the freelist part which I needed to
call from the SAP.
- Added support for FPHDF5. If it's a client, then it sends the
allocation or free request to the SAP. The SAP will call the
same code, but it'll actually do the allocation/freeing in that
case.
Platforms tested:
Linux & Modi4
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
New API functions
Description:
Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
__DLL__ is a keyword in some platforms and __DLL__ is also defined as a macro for windows DLL applications.
That causes problems.
Description:
Solution:
Use H5_DLL*** to replace __DLL***__ at all header files.
Change the macro defination at H5api_adpt.h.
Platforms tested:
linux2.2.18smp, irix64, solaris 2.7 and windows 2000
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Add 'closing' parameter to H5FDflush and VFL "flush" functions, per
http://hdf.ncsa.uiuc.edu/RFC/VFLFlush/VFLFlush.html
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
| |
Back out changes.
Description:
Back out changes to VFL 'flush' API function, pending review.
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New Feature
Description:
The VFL flush function is called immediately before a file is closed.
This can cause duplicate syncronization actions to occur, if the VFL
close function also performs them.
Solution:
Added 'closing' parameter to VFL 'flush' operation. This allows the VFL
flush function to bypass operations that will be duplicated within the VFL
close function.
Additionally, use the 'closing' parameter to bypass calls to MPI_File_sync()
when set. Since MPI_File_close() also syncronizes the file, this avoids
the terrible performance hit taken when calling MPI_File_sync() as the file
is closing.
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The H5Gget_objinfo() function was not setting the 'fileno' field in the
H5G_stat_t struct passed in.
Solution:
Added a "file serial number" to each file currently open in the library
and put that in the 'fileno' field. If a file is opened twice (with
H5Fopen) and the VFL driver detects that it is the same file (i.e. the
two file structures have the same "shared file info" in the library's
memory structures), they will have the same serial number.
This serial number has two drawbacks:
- If a VFL driver doesn't/can't detect that two calls to H5Fopen with
the same file actually _are_ the same file, each will get a
different serial number
- If the same file is closed and re-opened, the serial number will be
different.
It is be possible to fix the second drawback for many VFL drivers, but it
would be a lot of effort and probably isn't worth it until we've got a
good reason to do it. Dunno if we'll ever be able to fix the first
drawback...
Platforms tested:
FreeBSD 4.5 (sleipnir)
VS: ----------------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup (sorta)
Description:
When the first versions of the HDF5 library were designed, I remembered
vividly the difficulties of porting code from a 32-bit platform to a 16-bit
platform and asked that people use intn & uintn instead of int & unsigned
int, respectively. However, in hindsight, this was overkill and
unnecessary since we weren't going to be porting the HDF5 library to
16-bit architectures.
Currently, the extra uintn & intn typedefs are causing problems for users
who'd like to include both the HDF5 and HDF4 header files in one source
module (like Kent's h4toh5 library).
Solution:
Changed the uintn & intn's to unsigned and int's respectively.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
| |
Bug Fix, Code Cleanup, Code Optimization, etc.
Description:
Fold in the hyperslab speedups, clean up compile warnings and change a
few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead.
Platforms tested:
FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Changed
#include <hdf_file.h>
construct to
#include "hdf_file.h"
so that the GNU compiler can more easily pick up the dependencies
which it places in the .depend and Dependencies files. Also
regenerated the Dependencies to go along with this.
Platforms tested:
Linux
|