| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add more comments
Description:
Add more comments to H5Dmpio.c, which describes a little bit more about collective IO management.
Solution:
Platforms tested:
Only test at heping since only comments were added.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
| |
Code update
Description:
Add basics of routine for reading information back out of a fractal heap.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Checkpoint code
Description:
Rework code to support increasing size of root indirect block, in order
to add [at least] a second row of direct blocks.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Solaris 9 (shanti) w/64-bit
Linux 2.4 (mir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Enhancing the optimiziation of collective IO per chunk
Description:
When the user does one of the following two things:
1. to do collective IO per chunk without using our optimization code
2. or the user passes the percent of number of process per chunk to be 0 when choosing
to do collective IO per chunk
It is not necessary that the library uses MPI-IO collective calls to do any optimization.
Solution:
Modify the code so that no MPI communication-involved analyses will be done for the above cases. Chunk addresses are obtained globally and IO modes are assigned to collective always.
Platforms tested:
h5committest
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Yesterday, H5E_NONE_MAJOR and H5E_NONE_MINOR were added to error
API by hand.
Solution: Today, correct the change by hand with the tool hdf5/bin/make_err.
The change by hand is only in src/H5err.txt.
Platforms tested: fuss - simple change.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Re-alphabetize the POSIX routines (again). Also, add proper prototype
to HDrand() definition.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
There was a duplicate definition for HDsrandom and HDsrand left over in H5private.h
Solution:
Removed the two duplicate definitions.
Platforms tested:
heping (minor change)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
minor change for collective code
Description:
Solution:
Platforms tested:
mir
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Optimization
Description:
Get rid of unnecessary function call for systems that don't have rand_r.
Solution:
If rand_r isn't present on a system, then macros can be used to simply call
the underlying random function instead of calling the HDrand/HDsrand functions
that keep track of the random seed within the library.
Platforms tested:
heping (minor change)
Misc. update:
|
|
|
|
|
|
|
|
|
| |
Description: Added H5E_NONE_MAJOR and H5E_NONE_MINOR as a major and minor
error message. Their messages are "no error", but because the old library
had them, they're added in for backward compatibility.
Platforms tested: fuss - simple change.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Move VAX floating-point definitions down by the other machine specific ones.
Re-work the version handling for VAX byte ordered floating-point numbers.
Version numbers ought to be strictly increasing, not a bit-field.
Platforms tested:
FreeBSD 4.11 (sleipnir)
|
|
|
|
|
|
|
| |
Description: Changed macro from VMS to H5_VMS.
Platforms tested: no test needed.
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Re-alphabetize POSIX macros.
Platforms tested:
None, just eyeballed, too minor.
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up formatting & comments, etc.
Platforms tested:
None, very minor
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code checkpoint
Description:
Checkpoint fractal heap changes - mostly internal changes to better
handle more direct blocks from root indirect block.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS X (amazon)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Solution: Support VAX float and double types with new byte order. There're some
details not yet included,
1. the alignment detection for VAX order in H5detect.c.
2. support for special values in library conversion functions.
3. the infinity for VAX in H5T_init_inf.
4. support for VAX types in printing hexadecimal format.
Platforms tested: h5committest and fuss.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New APIs to add for collective chunk IO
Description:
Three new APIs
H5Pset_dxpl_mpio_chunk_opt_ratio
H5Pset_dxpl_mpio_chunk_opt_num
H5Pset_dxpl_mpio_chunk_opt
for optional optimization choices from users.
Solution:
Haven't added tests yet, won't affect other parts of the library.
Will add tests after urgent investigations of memory leaking problems from NASA Aura team.
Platforms tested:
heping: both parallel and sequential
shanti
Misc. update:
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Correct function name in FUNC_ENTER macro.
Platforms tested:
None, just eyeballed, too minor to require more.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code checkpoint
Description:
The fractal heap code is reasonably stable and is passing all its
current tests, so checkpoint everything with CVS.
Also, add "statistics" module for v2 B-trees code, which is only a
stub right now.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4 (chicago)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Clean up a few code mis-matches that crept in during the last checkin.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5commitest
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Update "power of two" macro and add "general" log2() routine.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Add "variable-length" 32-bit encode/decode macros, matching the other
64-bit ones, for later use in the fractal heap code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor for full h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Update code
Description:
Add a new error value, for later use.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor for full committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
New features
Description:
Add "find node greater than or equal to key" and "remove first" operations
to skip lists.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix/update formatting, clean up typos, etc.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Made small changes to sec2 driver to reflect that VMS systems ino_t is 1dim array
of size 3; all changes are marked with H5_VMS variable.
Defined HDremove macro for VMS to remove multiple versions of the files.
Solution:
Platforms tested: heping, VMS server
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
Two functions (HDsrand and HDrand) are not declared. They caused
compiling errors on Windows. Declare these two functions:
H5_DLL int HDrand();
H5_DLL void HDsrand(unsigned int seed);
Solution:
Platforms tested:
Windows XP and heping
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
Fix typo. srand/srandom don't return anything, so don't attempt to return
their value.
Solution:
Platforms tested:
heping
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix/Feature
Description:
Modify the library to use rand_r, when available, instead of srand. If
rand_r is not available, it will try to use srandom/random, and then finally
fall back to srand/rand.
Solution:
A couple places in the library use the srand() function to seed the random
number generator. This can cause problems on certain platforms and could theoretically
cause problems for users who expect a certain sequence of random numbers following their
own call to srand().
Most platforms have an implementation of rand_r, which is identical to rand, except that
it allows for explicit storage of the seed value. The configure script will now check
for the existence of rand_r to facilitate its use in the library.
Two new functions are added, to replace the macros HDsrand/HDrand. These functions are HDrand and
HDsrand. HDrand will call rand_r, if available, or random(), if available, and fall back to rand() if neither
of those is available. HDsrand will store the seed value locally, if rand_r is available. Otherwise, it will call
srandom if available, and fall back to srand otherwise.
Platforms tested:
heping, mir. Really need to test on Red Storm, since that platform motivated this fix, but the machine
is currently not available.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Correct problem with new log2(n) routine on 64-bit machines.
Solution:
Put some casts in to make certain that the values are computed as 32-bit
integers.
Platforms tested:
Linux 2.4 (mir) - 64-bit
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Fix typo in H5HF_dblock_debug() prototype
Platforms tested:
FreeBSD 4.11 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature & code cleanup
Description:
Update fractal heap to be able to insert objects into a direct block
hanging off the header.
Extract "octal dump" code into separate routine so that both the local
heaps & fractal heaps can use it.
Other code cleanups & support to get this far.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix some warnings & formatting issues
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added HL examples
Description:
Examples existed for the HL lite, image, and table interfaces, but
were never included in the distro... until now!
Solution:
Added source files to CVS, updated Makefile and MANIFEST.
Platforms tested:
mir, heping, shanti
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Porting.
Description:
Red Storm's lustre used srand and rand to initialize their user mode
of lustre. HDF5 also uses srand and that messes up Lustre.
Solution:
Use -DRED_STORM to block out the HDF5 code that calls srand.
This is a temporary fix since Lustre has fixed its code from
calling srand and rand but the fix is not deployed at Red_storm
yet.
Platforms tested:
Tested at Red-Storm.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Incrementtal checkin
Description:
Revise & update v2 B-tree code to separate the internal package-specific
routines from the library-callable "private" routines.
Similar updates for the fractal heap code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Check in initial "fractal heap" code, for supporting the group redesign.
Also, remove some remnants of the segmented heap/block tracker/B+tree code
which I didn't find during the last pass.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Style fixes for consistency & other minor cleanups
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4
|
|
|
|
|
|
|
|
|
|
|
|
| |
Maintenance
Description:
Remove half-built (half-baked? :-) implementation of B+tree, block tracker
and segmented heaps. (In preparation for starting the new heap coding)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bug fix
Description:
ret_value is not set to SUCCEED in H5D_mpio_select_write
That causes the 64-bit intel compiler unhappy.
It will give a non-zero number and cause the testing fake failure.
Another one is picked up by cmpi again. For one optimization case,
another variable is not initialized properly and compiler set an unhappy number,
cause the test failed.
Solution:
Properly initialize those variables.
Platforms tested:
teragrid:parallel
mir:parallel
heping: parallel and sequential
tungsten:parallel
Misc. update:
h5committest doesn't finish due to no space left on device.
parallel tests still failed at tungsten with cmpi. It looks like it was a bug from cmpi.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bug fix for daily test
Description:
if (fm->select_chunk) is not NULL at sequential mode and not set by the library, it shouldn't be free.
Otherwise, it will cause failures in the daily test.
Solution:
Explicitly set (fm->select_chunk) to NULL for non-parallel case.
Platforms tested:
copper with parallel
heping with both sequential and parallel
shanti with sequential
Misc. update:
Hopefully only mir failed at daily test.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix in collective IO code
Description:
When select none occurs in one process, fm->select_chunk hasn't been allocated the memory(not allocated until create_file_hyper); however, this needs to be checked; it was caught with cmpi at NCSA linux cluster.
Solution:
Relocate the place to allocate the memory space for fm->select_chunk.
Platforms tested:
cmpi at tungsten
mpich at heping
kind of minor and let daily test catch other errors.
Misc. update:
This won't fix the failure at mir.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
code clean up
Description:
Clean up some warnings in collective chunk IO code
Solution:
Platforms tested:
heping
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bug fix
Description:
During the process of the combination of collective chunk IO code and the original IO code,
The return value of a function call hasn't been initialized. On 32-bit mode at modi4, that value isn't set to 0 by default. So a fake error value returns.
Solution:
Re-arrange the code so that the return value will be set properly.
Platforms tested:
parallel:
modi4 - 32bit
heping
seq:
heping
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Erase one printf line accidently inserted in the code.
Description:
Solution:
Platforms tested:
No need to test.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
Enhance the collective chunk IO support
Description:
Solution:
Platforms tested:
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Enhanced collective chunk IO supports
Description:
1. When using collective IO with chunking storage without any tuning, performance may become worse under some circumstances.
2. Current HDF5 handles raw-data IO per chunk. So for many small chunks, many small IOs will be passed into MPI-IO. That may cause bad performance.
3. For one IO per chunk case, sometimes performance with collective is worse than performance with independent. An obvious case is when only one process is doing IO and all other process are not doing IO, the collective IO will only add overheads for communication. We want to avoid this case. Some management inside our library needs to be done.
Solution:
- Added managements of collective IO supports for chunking storage
inside parallel HDF5
1) Implemented One IO with collective mode for all chunks in the
application by building one MPI derived datatype accross all
chunks.
2) Implemented the decision-making support to do collective IO inside
MPI-IO per chunk.
3) Added the decision-making support to do one IO accross all chunks
or to do multiple IOs with each IO per chunk.
4) Added the support to handle the case some processes won't do any IOs in
collectively.
5) Some MPI-IO package(mpich 1.2.6 or lower, e.g.) cannot handle
collective IO correctly for the case when some processes have no
contributions to IOs, a special macro is added to change
collective IO mode to independent IO mode inside HDF5 library.
Platforms tested:
Parallel:
IBM AIX 5.2(copper)
Linux (heping) mpich-1.2.6
SDSC Teragrid mpich-1.2.5
Linux(Tungsten) mpich-1.2.6
Altix(NCSA cobalt)
Seq:
Linux(heping)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix & new feature
Description:
Support variable-length datatypes in compact data storage and chunked
data storage, along with attributes.
Bug fix on the H5T_vlen_set_loc to allow for changing the file on a
variable-length datatype on disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Can't h5committest right now, due to missing cache files.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) Add parallel test for metadata cache
2) Split serial test for metadata cache into two parts
3) Fix bug in which cache was flushed needlessly when the
cache wasn't full.
4) Performance improvements
5) Update API for parallel cache coherency bug fix.
Description:
See above.
Solution:
See above.
Platforms tested:
h5committest
Misc. update:
|