summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r5681] Purpose:Quincey Koziol2002-06-194-43/+96
| | | | | | | | | | | | | | | | | | | | New feature. Description: There is some discussion among the SAF team as to whether it is better to use MPI derived types for raw data transfers (thus needing a MPI_File_set_view() call), or whether it is better to use a sequence of low-level MPI types (i.e. MPI_BYTE) for the raw data transfer. Solution: Added an internal flag to determine whether derived types are preferred (the default), or whether they should be avoided. An environment variable ("HDF5_MPI_PREFER_DERIVED_TYPES") can be set by users to control whether MPI types should be used or not. Set the environment variable to "0" (i.e.: 'setenv HDF5_MPI_PREFER_DERIVED_TYPES 0') to avoid using MPI derived types. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5676] Purpose:Quincey Koziol2002-06-192-18/+4
| | | | | | | | | | | Code improvement Description: Some small code cleanups and took out the code the was turning off the metadata cache for parallel I/O (!) Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5675] Purpose:Quincey Koziol2002-06-197-29/+27
| | | | | | | | | | Code cleanup Description: Removed more compiler warnings, etc. Platforms tested: Linux 2.2.x (eirene) w/parallel
* [svn-r5671] Purpose:Quincey Koziol2002-06-193-26/+0
| | | | | | | | | | | Code cleanup Description: Remove "COALESCE_READS" ifdefs, which were brought in by the DPSS VFL driver and are no longer used. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5668] Purpose:Quincey Koziol2002-06-1911-52/+26
| | | | | | | | | | Code cleanup Description: Turn on more warnings in the IRIX builds and clean them up. Platforms tested: IRIX64 6.5 (modi4) w/parallel, both -n32 and the default (-64?) mode
* [svn-r5663] Purpose:Quincey Koziol2002-06-184-219/+26
| | | | | | | | | | | | | | | | | Bug fix. Description: The chunking code was using internal allocation routines to put blocks on a free list for reuse, instead of using the system allocation routines (ie. malloc, free, etc.). This causes problems when user filters attempt to allocate/free chunks for their algorithm's use. Solution: Switched the chunking code back to using the system allocation routines, we can address performance issues with them if it becomes a real problem. Platforms tested: Linux 2.2.x (eirene) && IRIX64 6.5 (modi4)
* [svn-r5661] Purpose:Quincey Koziol2002-06-185-96/+213
| | | | | | | | | | | | Code optimization Description: Avoid creating MPI types (and thus requiring a MPI_File_set_view() call) when contiguous selections are used for dataset I/O. This should be a performance improvement for those sorts of selections. Platforms tested: Linux 2.2.x (eirene) w/parallel && IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5653] Purpose:Quincey Koziol2002-06-173-30/+94
| | | | | | | | | | | | Code cleanup Description: Use dataset transfer property list to hold information about the MPI types for the current transfer, instead of setting pseudo-global variables in the file's struct. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5652] Purpose:Quincey Koziol2002-06-172-0/+10
| | | | | | | | | | | | Code cleanup Description: Use dataset transfer property list to hold information about the MPI types for the current transfer, instead of setting pseudo-global variables in the file's struct. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5651] Purpose:Quincey Koziol2002-06-173-29/+70
| | | | | | | | | | | | Code cleanup Description: Change MPI-I/O code to use the address of the dataset for the displacement, instead of having a separate displacement value. Removed displacement parameter from H5FD_mpio_setup parameters. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel.
* [svn-r5647] Description:Albert Cheng2002-06-171-2/+2
| | | | | | | | | version information were not updated when 1.4.4-pre1 was created. So, 1.4.4-pre1 actually contained 1.4.4-snap6 in it. Updated the version information to 1.4.4-pre2. Platforms tested: eyeballed all changes. Pretty sure only text changes in some character strings.
* [svn-r5635] Purpose:Quincey Koziol2002-06-141-2/+2
| | | | | | | | | | Default change Description: Enable the use of MPI types for collective I/O by default. Platforms tested: Linux 2.2.x (eirene) w/parallel
* [svn-r5631] Elena Pourmal2002-06-132-1/+6
| | | | | | | | | | | | | | | | Purpose: Maintenance Description: PSC has new OSF cluster; uname -sr output is OSF1 iam764 V5.1 1885 alpha We have to include inttypes.h file, otherwise compilations fails complaining about nondefined uint64_t type. Solution: Configure defined HAVE_INTTYPES_H macro; (fixed by Quincey) added HAVE_INTTYPES_H and H5_HAVE_INTTYPES_H macros to define if inttypes.h should be included. Platforms tested: OSF5.1 and Solaris 2.7
* [svn-r5604] Purpose:Quincey Koziol2002-06-126-20/+24
| | | | | | | | | | Code cleanup Description: Clean up some compiler warnings before the code freeze... Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5602] Purpose:Quincey Koziol2002-06-121-10/+46
| | | | | | | | | | | | | | | | Bug fix Description: I/O on "Regular" hyperslab selections could fail to transfer correctly if the number of elements in the selection's row did now fit "evenly" into the buffer being used for the transfer. Solution: Correct the calculation of the block & count offsets within the optimized "regular" hyperslab routines. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5593] Elena Pourmal2002-06-111-6/+17
| | | | | | | | | | | | | | | Purpose: Bug #774 fix Description: H5Tenum_valueof and H5Tenum_nameof functions did not fail when non-existing name or non-existing value were supplied. This happened because binary search algorithm did not check if value or name found during the search were equal to the supplied one. Solution: Added an appropriate check condition. Platforms tested: Solaris 2.7 and Linux 2.2.18
* [svn-r5585] Purpose:Quincey Koziol2002-06-113-1/+64
| | | | | | | | | | | | | | | | Bug Fix Description: H5Dcreate and H5Tcommit allow "empty" compound and enumerated types (i.e. ones with no members) to be stored in the file, but this causes an assertion failure and is somewhat vapid. Solution: Check the datatype "makes sense" before using it for H5Dcreate and H5Tcommit. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5579] Purpose:Bill Wendling2002-06-10127-677/+1748
| | | | | | | | | | | | | | Copyright Fix Description: Switched the copyright statements from the old, bad version to the new version we got from the lawyers. Note: not every file was changed. There are some files which have copyrights not to NCSA (see the Stream VFD and some of the GIF conversion modules didn't have copyrights, so I didn't know if they were from others or from us). I left those alone. If others think they should be changed, please feel free to do so. Platforms tested: Linux
* [svn-r5572] Purpose:Quincey Koziol2002-06-103-1/+3
| | | | | | | | | | | Bug fix (#620) Description: Change slightly misleading error message when creating attribute with same name as existing attribute to something a bit more clear. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5570] Purpose:Quincey Koziol2002-06-102-4/+4
| | | | | | | | | | | Bug Fix (#709)/Code improvement. Description: Allow chunks for chunked datasets to be cached when file is open for read-only access. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5569] Purpose:Quincey Koziol2002-06-101-0/+2
| | | | | | | | | | | | | | Bug fix (bug #777) Description: Current code allows a compound datatype to be inserted into itself. Solution: Check if the ID for the member is the same as the ID for the compound datatype and reject it if so. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5565] Purpose:Quincey Koziol2002-06-101-1/+1
| | | | | | | | | | | | | | Code cleanup Description: Forgot to remove the code setting the small data block size from the metadata block size setting routine. Solution: Took it out. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5564] Purpose:Quincey Koziol2002-06-102-2/+85
| | | | | | | | | | | New Feature Description: Add H5P[set|get]_small_data_block_size API function, per our discussions at last week's HDF5 meeting. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5561] Purpose:Quincey Koziol2002-06-102-5/+9
| | | | | | | | | | | Code cleanup Description: Allow H5FD_free to return successfully (but ignore) freeing objects with size 0. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5559] Purpose:Quincey Koziol2002-06-101-0/+2
| | | | | | | | | | | | | | Bug Fix for bug #789 Description: Creating a 1-D dataset region reference caused the library to hang (go into an infinite loop). Solution: Corrected algorithm for serializing hyperslab regions. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5552] Purpose:Quincey Koziol2002-06-072-4/+4
| | | | | | | | | | | | Bug fix. Description: Added extra check to locations where metadata and "small data" blocks were being returned to the file's list of free space to avoid freeing 0-sized blocks. This should fix the problems in last night's daily tests. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5549] Purpose:Quincey Koziol2002-06-073-40/+68
| | | | | | | | | | | | | | | | | Code Improvement Description: Split the metadata accumulator code into two parts: one for allowing writes of the accumulator buffer during reads (when the buffer is dirty and needs to be flushed to disk in order to hold the new metadata being read in) and another for only allowing writes of the buffer during writes. This allows the MPI-I/O VFL driver to use the metadata accumulator (but only during writes) and benefit from the reduced number of metadata I/O operations that it brings. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5548] Purpose:Quincey Koziol2002-06-061-1/+1
| | | | | | | | | | | | | | Bug Fix Description: H5F_OVERFLOW_HSIZET2OFFT macro is broken when SIZEOF_HSIZE_T is greater than SIZEOF_OFF_T. Solution: Fix macro definition Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5544] Purpose:Quincey Koziol2002-06-061-0/+3
| | | | | | | | | | | Code cleanup/bug fix Description: Re-add H5_SIZEOF_HSIZE_T macro definition, which seems to have gotten lost sometime. Platforms tested: FreeBSD 4.5 (sleipnir)
* [svn-r5541] Purpose:Quincey Koziol2002-06-062-16/+43
| | | | | | | | | | | | | | | | | Code cleanup/bug fix Description: Unused space in the file was being "dropped on the floor" when the metadata or "small data" block was moved to a new location in the file and there was unused space in the old block. Solution: Put the space left in the allocation block into the free list of space for the file. This allows it to be [potentially] reused and sometimes allows the file to be smaller. Platforms tested: Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5537] Purpose:Quincey Koziol2002-06-052-62/+106
| | | | | | | | | | | | | | | Code improvement. Description: Added boot block and driver info block checksumming feature to the shared file information. This prevents these blocks from being written out multiple times when they haven't changed. This reduces the number of I/O operations which hit the disk for my test program from 15 to 14 (i.e. from 393 to 14, overall). Platforms tested: Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5535] Purpose:Quincey Koziol2002-06-0514-54/+152
| | | | | | | | | | | | | | | | | New feature. Description: Added a "small data" block allocation mechanism to the library, similar to the mechanism used for allocating metadata currently. See the RFC for more details: http://hdf.ncsa.uiuc.edu/RFC/SmallData/SmallData.html This reduces the number of I/O operations which hit the disk for my test program from 19 to 15 (i.e. from 393 to 15, overall). Platforms tested: Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5528] Purpose:Quincey Koziol2002-06-041-4/+36
| | | | | | | | | | | | | | | Code improvement Description: Now that the metadata accumulator is working correctly, the buffer size tends to get rather large and stay there. Solution: Put a throttle on the accumulator buffer to reduce its size back down when it isn't needed. Platforms tested: Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5522] Elena Pourmal2002-06-041-6/+13
| | | | | | | | | | | | | Purpose: Bug fix (#699), fix provided by a user, approved by Quincey Description: When a scalar dataspace was written to the file and then subsequently queried with the H5Sget_simple_extent_type function, type was reported H5S_SIMPLE instead H5S_SCALAR. Solution: Applied a fix Platforms tested: Solaris 2.7 and Linux 2.2.18
* [svn-r5520] Purpose:Quincey Koziol2002-06-041-11/+36
| | | | | | | | | | | | | | | | | | | | | | | Code improvement Description: The metadata aggregation code in the library was not terribly smart about extending contiguous regions of metadata in the file and would not extend them as far as possible. This causes space in the file to be wasted, also. Solution: Be smarter about extending the space used in the file for metadata by checking whether new metadata blocks allocated in the file are at the end of the current metadata aggregation region and append them to the metadata region if so. This has the nice side benefit of reducing the number of bytes we waste in the file and reducing the size of the file by a small amount in some cases. This reduces the number of I/O operations which hit the disk for my test program from 53 to 19 (i.e. from 393 to 19, overall). Platforms tested: Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5518] Purpose:Quincey Koziol2002-06-041-1/+11
| | | | | | | | | | | | | | | | | | Bug Fix Description: The "dirty" flag for symbol table entries and symbol table nodes was not being cleared when they were flushed to the file, causing lots of extra metadata I/O. Solution: Reset the symbol table entry & nodes' flags when thy are flushed to disk. This reduces the number of I/O operations which hit the disk for my test program from 83 to 53 (i.e. from 393 to 53, overall). Platforms tested: Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5509] Purpose:Quincey Koziol2002-06-031-52/+93
| | | | | | | | | | | | | | | | | | | | | | | Code cleanup/bug fix Description: The "metadata accumulator" cache in the library (which is designed to catch small metadata writes/reads and bundle them together into larger I/O buffers) was incorrectly detecting the important case of metadata pieces being written sequentially to the file, adjoining but not overlapping. Additionally, the metadata accumulator was not being used to cache data read in from disk, only caching writes. Solution: Fix accumulator to correctly cache adjoining metadata writes and also to cache metadata read from disk. Between these two fixes, the number of I/O requests which resulted in actual reads/writes to the filesystem dropped from 393 requests to 82. :-) Platforms tested: Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5498] Purpose:Quincey Koziol2002-06-011-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Code Bug Fix Description: Under certain [obscure] circumstances, an object header would get paged out of the metadata cache, and when it was accessed again and brought back into the cache, and immediately had additional metadata added to it (an attribute, usually, or perhaps adding an object to a group), and needed to be extended with a continuation message, but there was no room in any existing object header chunks for the continuation message and an existing object header message needed to be moved to the new object header chunk (I told you it was obscure :-), the object header message moved to the new chunk (not the new metadata being added) would get corrupted. *whew* :-) Solution: Actually copy the "raw" object header message information of the object header message being moved to the new chunk, instead of relying on the "native" object header message information being re-encoded when the object header is flushed. This is because when an object header is paged out of the metadata cache and subsequently brought back in, the "native" information pointer in memory is reset to NULL and only the "raw" information exists. Platforms tested: Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
* [svn-r5464] Snapshot version 1.4 release 4 (snap5)HDF Admin2002-05-281-2/+2
|
* [svn-r5448] Purpose:Quincey Koziol2002-05-211-32/+38
| | | | | | | | | | | Code cleanup Description: Guard against getting into metadata broadcast in write routine and improve some error handling. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5438] Snapshot version 1.4 release 4 (snap4)HDF Admin2002-05-181-2/+2
|
* [svn-r5431] Purpose:Quincey Koziol2002-05-175-180/+260
| | | | | | | | | | | | | | | | | | | | | Bug fix/Code improvement. Description: Currently, the chunk data allocation routine invoked to allocate space for the entire dataset is inefficient. It writes out each chunk in the dataset, whether it is already allocated or not. Additionally, this happens not only when it is created, but also anytime it is opened for writing, or the dataset is extended. Worse, there's too much parallel I/O syncronization, which slows things down even more. Solution: Only attempt to write out chunks that don't already exist. Additionally, share the I/O writing between all the nodes, instead of writing everything with process 0. Then, only block with MPI_Barrier if chunks were actually created. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5421] Description:Albert Cheng2002-05-142-2/+2
| | | | | | | Corrected typos in the FUNC_ENTER macros of H5FD_sec2_flush and H5FD_log_flush. Platforms tested: modi4 (just tested it compiled fine.)
* [svn-r5420] Purpose:Quincey Koziol2002-05-141-19/+32
| | | | | | | | | | | | | | | | Code cleanup/Performance enhancement Description: The code to extend the file size in H5FD_mpio_flush is getting run even when the file size doesn't change. Also, it's sort of sidestepping MPI-I/O when extending the file, instead of using MPI-I/O features to set the file's size Solution: Only extend the file's size when the allocated size has increased. Also use MPI_File_set_size() to change the file's size. (Including barrier). Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5416] Purpose:Quincey Koziol2002-05-141-0/+8
| | | | | | | | | | | | | | | Bug Fix Description: When H5Freopen is called, the file to reopen's 'intent' (read/write permissions, etc) is not being copied to the new file's 'intent' information. Solution: Copy it. :-) Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5413] Purpose:Quincey Koziol2002-05-141-12/+49
| | | | | | | | | | | | | | | | Bug fix Description: Calling MPI_Get_count needs to be done with the same MPI type as was used for the transfer and we are always using MPI_BYTE, even when a different MPI type was used for the transfer. Solution: Only query MPI_Get_count with MPI_BYTE when we really used MPI_BYTE for the transfer. Wait for later to query MPI_Get_count with other MPI types. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5411] Purpose:Quincey Koziol2002-05-133-2/+49
| | | | | | | | | | | | | | | Performance enhancement Description: Doing an MPI_File_sync() just before a file is closed causing a large performance loss. Solution: Add flag to MPI file driver to avoid performance the MPI_File_sync() when the flag is set before a call to H5F_flush(). Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5391] Purpose:Quincey Koziol2002-05-106-99/+12
| | | | | | | | | | | | | | | | | | | | | | Code cleanup Description: The parallel I/O file driver is optimized to only write metadata with one process (and broadcast the results to the other processes). This is currently enabled by a separate call to H5FD_mpio_tas_allsame() before each metadata write to the file. This can easily lead to problems where the prelude function call is omitted before the actual write code or, in a threaded environment, lead to race condititions where the value set is reset before being used. Solution: Since we only want to write metadata from one process, key off of the 'type' parameter (which has information about whether the data being written it metadata or raw data) to H5FD_mpio_write() as the method for determining whether to only write from one process or not. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5387] Purpose:Quincey Koziol2002-05-101-4/+21
| | | | | | | | | | | | | | | New Feature Description: Currently, only process 0 writes metadata to disk, leading to a potential performance bottleneck as the other processors wait for it to catch up. Solution: Rotate the metadata responsibilities among all processes, speading out the work. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5382] Purpose:Quincey Koziol2002-05-091-32/+19
| | | | | | | | | | | Small code cleanup/improvement Description: Query the MPI rank only once and store it in the file structure for each process. Platforms tested: IRIX64 6.5 (modi4)
generated by cgit v0.12 at 2024-11-12 22:16:20 (GMT)