| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix serial build
Description:
I accidentally put the "use_par_opt_io" variable in an #ifdef PARALLEL
section.
Solution:
Hoist it out of parallel section
Platforms tested:
Eyeballed it - very trivial
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Straighten out more goofiness in the MPI code dealing with collective I/O
transfers - mostly make certain that a view is set if-and-only-if collective
I/O is occurring on raw data (and vice versa for views and independent I/O)
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel & FPHDF5
too minor to repquire h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
More refactoring on setup of MPI-IO parameters: hoisted the code to change
the transfer mode from collective to independent up into the H5D_read and
H5D_write routines, instead of duplicating it inside each of the chunk/contig
read/write routines.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up setup code for collective I/O transfers to make the logic more
obvious.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
H5Dwrite was only checking the "top level" of the datatype to stop parallel
I/O on variable-length datatypes.
Solution:
Make checks "deeper".
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
More linting...
Platforms tested:
FreeBSD 4.8 (sleipnir)
too minor to need h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
I/O on chunked datasets with a scalar dataspace for the memory dataspace
was not working correctly.
Solution:
Translate the scalar dataspace into a n-dimensional (where n is the number
of dimensions of the file's dataspace) dataspace of dimensions 1x1x1...
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest (modulo modi4 which is not working correctly)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
I/O on chunked datasets with point selections was not working correctly.
Solution:
Re-wrote some parts of raw data I/O routines that build the selections for
each chunk to correctly handle point selections.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up more warnings from lint
Platforms tested:
FreeBSD 4.8 (sleipnir)
too minor for h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
H5T_BKG_TEMP was accidentally removed from library code, but is used by
application's datatype conversion routines.
Solution:
Revert removal of H5T_BKG_TEMP.
Platforms tested:
h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Refactoring code
Description:
Refactored "IS_H5FD_MPIO || IS_H5FD_MPIPOSIX || IS_H5FD_FPHDF5" combination
of macros in many places into single IS_H5FD_MPI macro, which has the same
definition, but should be easier to maintain.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Update code
Description:
Move "PABLO_MASK" definition above header file inclusion to avoid problems
with inline functions in header files.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
The dataset's modification time was getting set whenever raw data was
written with H5Dwrite. Unfortunately, this is a metadata change (which are
required to be performed collectively) and H5Dwrite may be called independently
from a parallel program, resulting in metadata cache corruption and/or program
hangs.
Solution:
Don't update the modification time when raw data it written. :-(
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committest
Misc. update:
Noted in release notes and also sent to Frank for updating the docs.
|
| |
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Don't attempt to perform collective I/O on chunked datasets.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committested
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up various warnings & comment out unused code.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
FreeBSD 4.8 (sleipnir) w/parallel
h5committested
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Performance improment
Description:
Speed up chunked dataset I/O. This breaks down into several areas:
- Compute chunk selections in the file by using hyperslab operations
instead of iterating over each element in the selection.
- If the file and memory selections are the same shape, use the file
chunk selections to compute the memory chunk selections.
This required several additional dataspace, dataspace selection and
hyperslab routines.
Platforms tested:
h5committestted (although Fortran tests failed for some reason)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Correct typo which didn't show up during my previous testing in production
mode.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix some unused parameter warnings.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
The chunk dataspace selection information for raw data I/O are being leaked.
Solution:
Free the chunk information during the cleanup code.
Platforms tested:
Solaris 2.6 (baldric) w/purify
h5committest not needed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Raw data I/O on chunked datasets would attempt to allocate data structures
proportional to the number of chunks in the dataset on disk, instead of just
the number of chunks that the I/O operation would interact with, causing
operations on datasets with large #'s of chunks to fail (or become very slow),
even though the actual I/O operation was very modest.
Solution:
This is the "scalability fix" for chunked datasets that I've mentioned
we need to do, althought it's not the complete fix for the issue. Read on
for the details...
Only create data structures for the chunks that the I/O operation will
actually act on, reducing the amount of information allocated in memory,
normally.
I say "normally", because this algorithm has the same problems as the
original algorithm (worse actually, since the data structure for each chunk
is larger now) if _all_ the chunks in a dataset with a lot of chunks are
actually involved in the I/O operation. If that is the case, this code
will fail in a similar way.
To truly fix the problem, we would need to only create data structures for
a particular number of chunks, perform the I/O on just those chunks, then
release the data structures for those chunks and create data structures for
the next set of chunks to access, etc. However, I think this case is pretty
rare right now and we should worry about it after the 1.6.0 release.
Platforms tested:
h5committested
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Small code cleanup
Description:
Remove redundant sanity check.
Added a note about a potential bug.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
|
| |
|
|
|
|
|
|
|
| |
Description: when a datatype element size is bigger than I/O datatype
conversion buffer(1MB), library reports failure.
Solution: adjust conversion buffer to be at least equal to one element size
Platforms tested: h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up warnings exposed by compiling on O2K. Also, revert some of Bill
and my changes to the H5S_mpi_opt_types_g, etc. and settle them back into their
original location.
Platforms tested:
h5committested.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
A variable wasn't declared in this file.
Solution:
Put the extern declaration of the variable in this file. It's defined
in the H5S.c module.
Platforms tested:
Modi4 (This affects only parallel platforms)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Clean up small memory leak.
Platforms tested:
h5committested.
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup.
Description:
Cleaned up an unusued variable and some commented out code.
Platforms tested:
FreeBSD 4.8 (sleipnir)
Triple check not needed.
|
|
|
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
|
