| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review and get code to conform to standard library coding style, also
add 'const' keyword to "set" routines.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) Removed references to H5AC2__CURR_JNL_CONFIG_VER from H5Fget_jnl_config()
and H5Pget_jnl_config(), and also references to
H5AC__CURR_CACHE_CONFIG_VERSION from H5Fget_mdc_config() and
H5Pget_mdc_config().
2) Removed H5Pset_journal() from H5C2journal.c, and modified test
code to use H5F/Pget/set_mdj_config() instead.
3) Implemented support for callbacks on metadata journaling status change
along with the associated registration / deregistration calls and
associated test code.
4) Fixed bug in journaling shutdown exposed by 3 above.
Tested and passed on Phoenix (serial), Linew (serial), and Kagiso (parallel).
However, while I was testing there were a couple of checkins, forcing
an update and second round of testing.
On the second round, tested and passed on Phoenix (serial) and
Kagiso (parallel), but failed on Linew (serial).
As best I can tell, this was caused by Mike M's checkin -- which
broke the smoke checks in cache2_journal on Linew but not Phoenix
or Kagiso. A typical delta in the architype files follows:
linew.hdfgroup.uiuc.edu% diff -ctw cache2_journal_sc00_000.jnl tmp/cache2_journal_sc00_000.jnl
*** cache2_journal_sc00_000.jnl Fri Aug 22 08:28:49 2008
--- tmp/cache2_journal_sc00_000.jnl Fri Aug 22 05:08:41 2008
***************
*** 1,5 ****
! 0 ver_num 1 target_file_name cache_journal_test.h5 creation_date Fri Aug 22 human_readable 1
! E eoa_value 0x0
C comment Begin transaction on transaction 1.0.
1 bgn_trans 1
2 trans_num 1 length 1 base_addr 0x401 body 01
--- 1,5 ----
! 0 ver_num 1 target_file_name cache_journal_test.h5 creation_date Wed Aug 20 human_readable 1
! E eoa_value 0x772a9c01
C comment Begin transaction on transaction 1.0.
1 bgn_trans 1
2 trans_num 1 length 1 base_addr 0x401 body 01
As you can see, it looks like garbage is getting into the first
eoa write on Linew.
I'm checking in anyway, as Quincey needs my changes, and I will not
have time to work on this for several days.
Mike: Let me know if you are tackling this one -- if not, I'll deal with it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
from the H5AC2_cache_config_t structure and the associated
H5P/H5Fget/set_mdc_config() API calls, and into the newly created
H5AC2_jnl_config_t structure and H5P/H5Fget/set_jnl_config() calls.
Updated test code accordingly.
Updated the trace file test code for journaling.
Also folded in a fix to an assertion bug in H5C2pkh.h
Tested serial on Phoenix and Linew, and parallel (with and without the
trace file enabled) on kagiso.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring changes from trunk from the time the branch was created (r14280)
up to the 1.8.0 release (r14525) back into the metadata journaling branch.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Switch v1 B-tree nodes from using previous cache to use the new journaling
cache code. This is a major switch for the cache callbacks.
Switched the metadata caching code from having a pointer to the file it's
in to receiving a pointer to the file, when needed. This was necessary in
order to avoid crashing when two files IDs were open on the same underlying
file and one of those files was closed with cache entries using the file
pointers. Also took out the check in the caching code for reading off the
end of the file, which didn't play nicely with the multi-file VFD.
Switching the cache from having a pointer internally to requiring one
externally meant tweaking almost all the test code. :-/
Added correct exit codes to cache2 tests that didn't have them already,
so the 'make check' will stop when they fail.
Use the path to the h5diff in this build in the tools/h5recover testing
script, since we can't guarantee a user has HDF5 already installed.
Assorted minor tweaks to get everything to compile more cleanly and pass
all the tests (on my Mac :-).
Tested on:
Mac OS X (10.5.2) w/C++
(more testing coming up shortly, I just didn't have my "rsync testbed" set
up for this branch when I started making changes to the code)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
into Makefile.am un
der /src. So when gmake check-install is run for HDF5 examples, it will fail. Th
e fix is to include a macro #ifdef H5_HAVE_WINDOW around #include H5FDwindows.h
so that H5FDwindows.h will only be included on windows, not on other platforms.
We've fixed with the same way for other source code that includes H5FDwindows.h.
This has been tested with h5committest. However, it takes a few hours to test all tests and except kagiso. Tests on linew and smirom cannot finish. It takes forever for the fheap test.
And for parallel test at kagiso, ph5diff test seems to be hanged. I think all of our changes will only affect windows. So in order not to block other developers to check in their code, we just go ahead to check our changes.
Kent Yang
|
|
|
|
|
|
|
| |
windows as the default vfd. Although sec2 is still default on Windows, it should now be possible to switch by simply changing H5pubconf.h .
Tested:
.NET 2003 on WinXP
|
|
|
|
|
|
|
|
|
|
|
| |
Hook "default VFD" configure macros into library.
Also, make "default VFD" default to sec2 instead of nothing.
Tested on:
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Mac OS X/32 10.4.9 (amazon)
|
|
|
|
|
|
|
|
|
|
|
|
| |
running on kagiso.
The static szip libraries are working as intended, but when shared szip
libraries are present and intended for use they cannot be opened. A check
has been implemented to test if shared szip libraries are functional on
current platform. If they are not, and the user is trying to use them,
szip is now disbaled and the user is informed.
This issue occurs on kagiso, and the resolution was tested on kagiso.
|
|
|
|
|
|
|
|
|
| |
copyright notice.
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
|
|
|
|
|
|
|
|
|
| |
haven't
been run in a while, so many of the source files were updated with
tracing macros, etc. No code changes by me.
Tested on kagiso and smirom.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor generic property list initialization code to put property list
specific routines in property list modules, instead of scattered to the four
winds. Also, introduce property list class initialization objects, to make
adding new property list classes in the library easier.
Fix daily test failure by using H5Pget_elink_prefix() API routine instead
of looking at the "raw" generic property list information.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Linux/64 2.4 (mir) w/build-all & 1.6 compat
|
|
|
|
|
|
|
| |
Re-run 'bin/reconfigure' script after recent checkins
Tested on:
none - shouldn't have any affect on compilation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add "use the latest version of the file format" flag to the file access
property list and internal file data structures.
Fix bug where metadata block size was retrieved instead of the small
data block size.
Categorize property list routine prototypes in the public header file.
Tested on:
Mac OS/PPC 10.4 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: See details from Bug #213. Family member file size wasn't saved
anywhere in file. When family file is opened, the first member size determine
the member size.
Solution: This is the fourth step of checkin. A test suit is added for h5repart,
including a program to generate the test files, a script file to run h5repart,
and a program to verify repartitioned files can be opened by the library.
There's a change from the first step of checkin. Family name template is no
longer saved in the superblock because different pathname can make the name
different.
In the third step of checkin, h5repart has been modified. If h5repart is used
to change the size of family member file, the new size(actual member size) is saved
in the superblock.
In the second step of checkin, multi driver is checked against the driver
name saved in superblock. Wrong driver will result in a failure with an error message indicating
multi driver should be used. This change includes split driver because it's a special case for multi
driver.
In the first step of checkin. Family member size and name template(unused at this stage) are saved
in file superblock. When file is reopened,the size passed in thrin superblock. A different size
will trigger a failure with an error message indicating the right size. Wrong driver to open family
file will cause a failure, too.
Platforms tested: h5committest and fuss.
Misc. update: MANIFEST
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up some compiler warnings.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/gcc 4.0
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add API calls allowing user control of the metadata cache.
Description:
Prior to this update, the metadata cache was not configurable
from outside the library.
Solution:
Add API calls allowing the user to configure the metadata cache
either at file open time, or for any open file. Also added calls
permitting the user to monitor cache size and hit rate. These
latter facilities are needed for "manual" cache size control
Platforms tested:
h5committested
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Refactored handlier of VFL drivers in file access and data transfer property
lists in order to simplify and unify the code dealing with them.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Version update
Description:
Removed 1.4 compatibility code in the library.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The H5HG module had a couple of lines like the following:
HDmemcpy(foo + 1, foo, size);
This would overwrite the first character of "foo" for the total of
"size" bytes.
The H5Pfapl.c is an update of the trace information.
Solution:
Use HDmemmove instead. (Suggested by George Lewandowski at Boeing.
Thanks! :-)
Platforms tested:
Modi4 (Fortran & Parallel)
Verbena (Fortran & C++)
Arabica (Fortran)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Limit the scope on more function prototypes/macros/typedefs.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/new feature.
Description:
Split FUNC_LEAVE into API and non-API specific versions. This allows a
solution to compiling this branch with C++, as well as reducing the size
of the binaries produced.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up a few more warnings and update dependencies.
Platforms tested:
Linux 2.2.18smp (eirene) serial & parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
New API functions
Description:
Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
|
|
|
|
|
|
|
|
|
| |
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
|
|
Code cleanup
Description:
Move get/set routines for each type of property list (file creation,
dataset creation, file access and dataset transfer) into their own source
code module.
Platforms tested:
FreeBSD 4.6 (sleipnir)
|