| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
| |
Make a tarball of 1.4.4-pre4 that contains CVS stuff for easier
commit later. Bump up version information by bin/h5vers -i.
Platforms tested:
No test since this is the same process the release script will do.
|
|
|
|
|
|
|
|
| |
Version bump
Description:
I'm making another prerelease available for the SAF team, with some
features I'd like them to test, so bump the prerelease number again.
|
|
|
|
|
|
|
|
|
| |
Update version info.
Description:
Bump the prelease number to reflect the fact that I gave out a prerelease
to the SAF developers yesterday and I don't want to confuse people when we
make another prelease tarball.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature.
Description:
There is some discussion among the SAF team as to whether it is better
to use MPI derived types for raw data transfers (thus needing a
MPI_File_set_view() call), or whether it is better to use a sequence of
low-level MPI types (i.e. MPI_BYTE) for the raw data transfer.
Solution:
Added an internal flag to determine whether derived types are preferred
(the default), or whether they should be avoided. An environment variable
("HDF5_MPI_PREFER_DERIVED_TYPES") can be set by users to control whether MPI
types should be used or not. Set the environment variable to "0" (i.e.:
'setenv HDF5_MPI_PREFER_DERIVED_TYPES 0') to avoid using MPI derived types.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
|
|
|
|
|
|
|
|
|
|
|
| |
Code improvement
Description:
Some small code cleanups and took out the code the was turning off the
metadata cache for parallel I/O (!)
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Removed more compiler warnings, etc.
Platforms tested:
Linux 2.2.x (eirene) w/parallel
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Remove "COALESCE_READS" ifdefs, which were brought in by the DPSS VFL
driver and are no longer used.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Turn on more warnings in the IRIX builds and clean them up.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel, both -n32 and the default (-64?) mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
The chunking code was using internal allocation routines to put blocks on
a free list for reuse, instead of using the system allocation routines (ie.
malloc, free, etc.). This causes problems when user filters attempt to
allocate/free chunks for their algorithm's use.
Solution:
Switched the chunking code back to using the system allocation routines,
we can address performance issues with them if it becomes a real problem.
Platforms tested:
Linux 2.2.x (eirene) && IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Avoid creating MPI types (and thus requiring a MPI_File_set_view() call)
when contiguous selections are used for dataset I/O. This should be a
performance improvement for those sorts of selections.
Platforms tested:
Linux 2.2.x (eirene) w/parallel && IRIX64 6.5 (modi4) w/parallel & FORTRAN
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Use dataset transfer property list to hold information about the MPI types
for the current transfer, instead of setting pseudo-global variables in
the file's struct.
Platforms tested:
Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel & FORTRAN
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Use dataset transfer property list to hold information about the MPI types
for the current transfer, instead of setting pseudo-global variables in
the file's struct.
Platforms tested:
Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel & FORTRAN
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Change MPI-I/O code to use the address of the dataset for the displacement,
instead of having a separate displacement value. Removed displacement
parameter from H5FD_mpio_setup parameters.
Platforms tested:
Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel.
|
|
|
|
|
|
|
|
|
| |
version information were not updated when 1.4.4-pre1 was created.
So, 1.4.4-pre1 actually contained 1.4.4-snap6 in it.
Updated the version information to 1.4.4-pre2.
Platforms tested:
eyeballed all changes. Pretty sure only text changes in some character
strings.
|
|
|
|
|
|
|
|
|
|
| |
Default change
Description:
Enable the use of MPI types for collective I/O by default.
Platforms tested:
Linux 2.2.x (eirene) w/parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Maintenance
Description:
PSC has new OSF cluster; uname -sr output is
OSF1 iam764 V5.1 1885 alpha
We have to include inttypes.h file, otherwise compilations fails
complaining about nondefined uint64_t type.
Solution:
Configure defined HAVE_INTTYPES_H macro; (fixed by Quincey) added
HAVE_INTTYPES_H and H5_HAVE_INTTYPES_H macros to define if
inttypes.h should be included.
Platforms tested:
OSF5.1 and Solaris 2.7
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up some compiler warnings before the code freeze...
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
I/O on "Regular" hyperslab selections could fail to transfer correctly
if the number of elements in the selection's row did now fit "evenly"
into the buffer being used for the transfer.
Solution:
Correct the calculation of the block & count offsets within the optimized
"regular" hyperslab routines.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Bug #774 fix
Description:
H5Tenum_valueof and H5Tenum_nameof functions did not fail
when non-existing name or non-existing value were supplied.
This happened because binary search algorithm did not check
if value or name found during the search were equal to the supplied
one.
Solution:
Added an appropriate check condition.
Platforms tested:
Solaris 2.7 and Linux 2.2.18
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
H5Dcreate and H5Tcommit allow "empty" compound and enumerated types (i.e.
ones with no members) to be stored in the file, but this causes an assertion
failure and is somewhat vapid.
Solution:
Check the datatype "makes sense" before using it for H5Dcreate and
H5Tcommit.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Copyright Fix
Description:
Switched the copyright statements from the old, bad version to the
new version we got from the lawyers. Note: not every file was
changed. There are some files which have copyrights not to NCSA (see
the Stream VFD and some of the GIF conversion modules didn't have
copyrights, so I didn't know if they were from others or from us). I
left those alone. If others think they should be changed, please
feel free to do so.
Platforms tested:
Linux
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix (#620)
Description:
Change slightly misleading error message when creating attribute with same
name as existing attribute to something a bit more clear.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix (#709)/Code improvement.
Description:
Allow chunks for chunked datasets to be cached when file is open for
read-only access.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix (bug #777)
Description:
Current code allows a compound datatype to be inserted into itself.
Solution:
Check if the ID for the member is the same as the ID for the compound
datatype and reject it if so.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Forgot to remove the code setting the small data block size from the metadata
block size setting routine.
Solution:
Took it out.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
| |
New Feature
Description:
Add H5P[set|get]_small_data_block_size API function, per our discussions at
last week's HDF5 meeting.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Allow H5FD_free to return successfully (but ignore) freeing objects with
size 0.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix for bug #789
Description:
Creating a 1-D dataset region reference caused the library to hang (go into
an infinite loop).
Solution:
Corrected algorithm for serializing hyperslab regions.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
Added extra check to locations where metadata and "small data" blocks were
being returned to the file's list of free space to avoid freeing 0-sized
blocks. This should fix the problems in last night's daily tests.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code Improvement
Description:
Split the metadata accumulator code into two parts: one for allowing writes
of the accumulator buffer during reads (when the buffer is dirty and needs
to be flushed to disk in order to hold the new metadata being read in) and
another for only allowing writes of the buffer during writes.
This allows the MPI-I/O VFL driver to use the metadata accumulator (but
only during writes) and benefit from the reduced number of metadata I/O
operations that it brings.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
H5F_OVERFLOW_HSIZET2OFFT macro is broken when SIZEOF_HSIZE_T is greater
than SIZEOF_OFF_T.
Solution:
Fix macro definition
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/bug fix
Description:
Re-add H5_SIZEOF_HSIZE_T macro definition, which seems to have gotten lost
sometime.
Platforms tested:
FreeBSD 4.5 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/bug fix
Description:
Unused space in the file was being "dropped on the floor" when the metadata
or "small data" block was moved to a new location in the file and there was
unused space in the old block.
Solution:
Put the space left in the allocation block into the free list of space for
the file. This allows it to be [potentially] reused and sometimes allows
the file to be smaller.
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code improvement.
Description:
Added boot block and driver info block checksumming feature to the shared
file information. This prevents these blocks from being written out
multiple times when they haven't changed.
This reduces the number of I/O operations which hit the disk for my test
program from 15 to 14 (i.e. from 393 to 14, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature.
Description:
Added a "small data" block allocation mechanism to the library, similar to
the mechanism used for allocating metadata currently.
See the RFC for more details:
http://hdf.ncsa.uiuc.edu/RFC/SmallData/SmallData.html
This reduces the number of I/O operations which hit the disk for my test
program from 19 to 15 (i.e. from 393 to 15, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code improvement
Description:
Now that the metadata accumulator is working correctly, the buffer size
tends to get rather large and stay there.
Solution:
Put a throttle on the accumulator buffer to reduce its size back down when
it isn't needed.
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Bug fix (#699), fix provided by a user, approved by Quincey
Description:
When a scalar dataspace was written to the file and then
subsequently queried with the H5Sget_simple_extent_type function,
type was reported H5S_SIMPLE instead H5S_SCALAR.
Solution:
Applied a fix
Platforms tested:
Solaris 2.7 and Linux 2.2.18
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code improvement
Description:
The metadata aggregation code in the library was not terribly smart about
extending contiguous regions of metadata in the file and would not extend
them as far as possible. This causes space in the file to be wasted, also.
Solution:
Be smarter about extending the space used in the file for metadata by
checking whether new metadata blocks allocated in the file are at the end
of the current metadata aggregation region and append them to the metadata
region if so. This has the nice side benefit of reducing the number of
bytes we waste in the file and reducing the size of the file by a small
amount in some cases.
This reduces the number of I/O operations which hit the disk for my test
program from 53 to 19 (i.e. from 393 to 19, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The "dirty" flag for symbol table entries and symbol table nodes was not
being cleared when they were flushed to the file, causing lots of extra
metadata I/O.
Solution:
Reset the symbol table entry & nodes' flags when thy are flushed to disk.
This reduces the number of I/O operations which hit the disk for my test
program from 83 to 53 (i.e. from 393 to 53, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/bug fix
Description:
The "metadata accumulator" cache in the library (which is designed to catch
small metadata writes/reads and bundle them together into larger I/O
buffers) was incorrectly detecting the important case of metadata pieces
being written sequentially to the file, adjoining but not overlapping.
Additionally, the metadata accumulator was not being used to cache data
read in from disk, only caching writes.
Solution:
Fix accumulator to correctly cache adjoining metadata writes and also to
cache metadata read from disk.
Between these two fixes, the number of I/O requests which resulted in actual
reads/writes to the filesystem dropped from 393 requests to 82. :-)
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code Bug Fix
Description:
Under certain [obscure] circumstances, an object header would get paged out
of the metadata cache, and when it was accessed again and brought back into
the cache, and immediately had additional metadata added to it (an
attribute, usually, or perhaps adding an object to a group), and needed to
be extended with a continuation message, but there was no room in any
existing object header chunks for the continuation message and an existing
object header message needed to be moved to the new object header chunk (I
told you it was obscure :-), the object header message moved to the new
chunk (not the new metadata being added) would get corrupted. *whew* :-)
Solution:
Actually copy the "raw" object header message information of the object
header message being moved to the new chunk, instead of relying on the
"native" object header message information being re-encoded when the object
header is flushed. This is because when an object header is paged out of
the metadata cache and subsequently brought back in, the "native"
information pointer in memory is reset to NULL and only the "raw"
information exists.
Platforms tested:
Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Guard against getting into metadata broadcast in write routine and improve
some error handling.
Platforms tested:
IRIX64 6.5 (modi4)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix/Code improvement.
Description:
Currently, the chunk data allocation routine invoked to allocate space for
the entire dataset is inefficient. It writes out each chunk in the dataset,
whether it is already allocated or not. Additionally, this happens not
only when it is created, but also anytime it is opened for writing, or the
dataset is extended. Worse, there's too much parallel I/O syncronization,
which slows things down even more.
Solution:
Only attempt to write out chunks that don't already exist. Additionally,
share the I/O writing between all the nodes, instead of writing everything
with process 0. Then, only block with MPI_Barrier if chunks were actually
created.
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
| |
Corrected typos in the FUNC_ENTER macros of H5FD_sec2_flush and
H5FD_log_flush.
Platforms tested:
modi4 (just tested it compiled fine.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/Performance enhancement
Description:
The code to extend the file size in H5FD_mpio_flush is getting run even when
the file size doesn't change. Also, it's sort of sidestepping MPI-I/O when
extending the file, instead of using MPI-I/O features to set the file's size
Solution:
Only extend the file's size when the allocated size has increased. Also
use MPI_File_set_size() to change the file's size. (Including barrier).
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
When H5Freopen is called, the file to reopen's 'intent' (read/write
permissions, etc) is not being copied to the new file's 'intent'
information.
Solution:
Copy it. :-)
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Calling MPI_Get_count needs to be done with the same MPI type as was used
for the transfer and we are always using MPI_BYTE, even when a different
MPI type was used for the transfer.
Solution:
Only query MPI_Get_count with MPI_BYTE when we really used MPI_BYTE for the
transfer. Wait for later to query MPI_Get_count with other MPI types.
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Performance enhancement
Description:
Doing an MPI_File_sync() just before a file is closed causing a large
performance loss.
Solution:
Add flag to MPI file driver to avoid performance the MPI_File_sync() when
the flag is set before a call to H5F_flush().
Platforms tested:
IRIX64 6.5 (modi4)
|