summaryrefslogtreecommitdiffstats
path: root/tools/h5repack/h5repack.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge/align with trunkAllen Byrne2016-09-281-275/+271
|
* [svn-r29309] HDFFV-9698, HDFFV-9241, HDFFV-9570: Merge fixes from trunk for ↵Allen Byrne2016-03-071-1/+1
| | | | subsetting index, default fill value display, and optional arg crash. Also some minor compiler warnings fixes.
* [svn-r24205] Merge HDFFV-8513/8522 from trunk (via cmake branch), h5repack ↵Allen Byrne2013-09-261-940/+870
| | | | | | | | UD plugins. Also warning session fixes. Tested: CMake local linux
* [svn-r24072] Merge trunk cmake changes to 1.8 branch. Allen Byrne2013-08-261-1/+1
| | | | | | | Also add default switch blocks to h5import. Merge h5dump any_path option from trunk. Tested: local linux
* [svn-r22814] Purpose:Jonathan Kim2012-09-261-0/+1
| | | | | | | | | | | | | | HDFFV-8012 - h5repack changes max dims and cause failure if only "-f none" is used without changing layout for chunked dataset when a chunk dim is bigger than a dataset dim Description: "h5repack -f <obj>:NONE <file.h5> out.h5" command failed if source file contains chunked dataset and a chunk dim is bigger than a dataset dim. Another issue is that the command changed max dims if chunk dim is smaller than the dataset dim. These issue occurred when dataset size is smaller than 64k (compact size limit) Fixed them. Merged from HDF5 trunk r22805 Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
* [svn-r22281] Purpose:Jonathan Kim2012-04-111-3/+6
| | | | | | | | | | | | | Fix for HDFFV-7993 - h5repack fails with error "chunk size must be <= maximum dimension size for fixed-sized dimensions" Description: Fixed a failure when change the chunk size of a specified chunked dataset with unlimited max dims. Also took care of converting to contiguous and compact from the dataset. Test cases were added and tagged with jira#. Merged from HDF5 trunk r22277. Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE cmake), Cmake (jam)
* [svn-r22216] Description:Quincey Koziol2012-03-311-3/+3
| | | | | | | | | | | | | Bring r22180, r22181 and r22214 from trunk to 1.8 branch: Add new "metadata block size" command line option ('-M <x>' or '--metadata_block_size=<x>') for h5repack. [With release managers blessing] Tested on: Mac OSX/64 10.7.3 (amazon) w/debug) (h5committest upcoming)
* [svn-r22013] HDFFV-7560:Allen Byrne2012-03-021-9/+9
| | | | | | | | | | Merge 1.8 and h5dump/tools and tests based on tools library from trunk. Reduced warnings. HDFFV-7949: Remove duplicated functions in h5ls Tested: local linux,h5committest
* [svn-r21870] Purpose:Jonathan Kim2012-01-101-5/+5
| | | | | | | | | | | | | | | Fix for HDFFV-7840 h5repack: memory leak over one of the h5diff test file Description: Turned out that there were two causes of memory leaks. 1. for handling variable length string in attribute. 2. for handling compound type with non-reference members. The first issue is fixed in copy_attr() which is updated to use h5tools_detect_vlen to take care of vlen string as well as vlen data. The second is fixed in copy_refs_attr() of compound handling code. Merged from HDF5 trunk r21869. Tested: jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
* [svn-r21400] Purpose:Jonathan Kim2011-09-201-25/+50
| | | | | | | | | | | | | HDFFV-5932 - h5repack breaks files with dimension scales Description: - Fixed h5repack to update values of references(object and region) of attributes in h5repack for 1) references, 2) ARRAY of references, 3) VLEN of references, and 4) COMPOUND of references. - Merged from HDF5 trunk 21393, 21382, 21386, 21389. (support Peter) Tested: jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE)
* [svn-r20062] Description:Quincey Koziol2011-02-081-45/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring r20061 from trunk to 1.8 branch: Bring changes from Coverity branch to trunk: r19930: Fix memory leaks involving VL attributes in h5repack and h5diff. The buffers in copy_attr and diff_attr were not checked for the presence of a vlen before being freed, and vlen storage was never reclaimed. Added checks and calls to H5D_vlen_reclaim(). r19933: Purpose: Fix memory leak in H5L_move_cb() Description: H5L_move_cb copied the source link using H5O_msg_copy() but freed it manually using H5MM_xfree(). Since H5O_link_copy allocates the link using H5FL_MALLOC, this causes the link to be allocated from the free list but is never put back on the free list when it is freed. This prevents the link free list from shutting down properly. Modified H5L_move_cb() and H5L_move_dest_cb() to free the link properly using H5O_msg_free(). r19973: Fix resource leaks by freeing string created by HD5f2string r19974: Issue #345: Inialize buf variable to null Tested on: Mac OS X/32 10.6.6 (amazon) w/debug & production (h5committested on Coverity branch)
* [svn-r19254] Description:Quincey Koziol2010-08-191-14/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | Bring r19252 & 19253 from trunk to 1.8 branch: Bring Coverity changes from branch to trunk: r19161: Fixed the part for matching the subset info with dataset r19189: BZ1646: h5dump does not check number of dimensions for subsetting parameters against the dataset Changed subset_t structure from holding hsize_t pointers to holding new subset_d pointers, which hold the original hsize_t pointer + len. this len is then checked against dataset ndims in the handle_dataset function of h5dump. Changed all references to use new data structure. Added tests for each subset parameter. r19190: Added new h5dump ddl files Tested on: Mac OS X/32 10.6.4 (amazon) w/debug & production (h5committested on Coverity branch)
* [svn-r18758] Corrected missed progname to function instanceAllen Byrne2010-05-111-1/+1
|
* [svn-r18757] Enable tools lib to be built as a dll on windows. Added two ↵Allen Byrne2010-05-111-17/+13
| | | | | | | | | | get/set functions for progname and d_status. Also add windows import/export declarations to functions. Updated windows - H5pubconf.h for packed bits function Tested: Windows, linux
* [svn-r18652] Description:Quincey Koziol2010-04-271-1/+1
| | | | | | | | | | | | Bring r18650 from trunk to 1.8 branch: Bring back various minor tweaks & cleanups from the revise_chunks branch. Tested on: FreeBSD/32 6.3 (duty) in debug mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode (h5committested on trunk)
* [svn-r18455] Purpose:Jonathan Kim2010-03-251-0/+331
| | | | | | | | | | | Fix for the bug1726 - NPOESS: h5repack loses attributes for datasets of type H5T_REFERENCE. Description: Merged from hdf5 trunk rXXX Tested: jam
* [svn-r18198] Description:Quincey Koziol2010-01-301-10/+10
| | | | | | | | | Trim trailing whitespace from source code files, with following command: find . \( -name "*.[ch]" -or -name "*.cpp" -or -name "*.f90" \) -print |xargs -n 1 sed -i "" 's/[[:blank:]]*$//' Tested on: None - eyeballed only
* [svn-r17982] Description:Quincey Koziol2009-12-101-13/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring r17980 from trunk to 1.8 branch: Bring Coverity changes into the trunk: (also other minor cleanups) r17955: Fix Coverity item 24. Add missing error condition to H5AC_ext_config_2_int_config. r17956: Fix Coverity item 24. Improve error checking in H5A_compact_build_table_cb. r17957: Fix Coverity item 150. Fix warning in H5A_compact_build_table_cb. r17958: Fix Coverity item 117. Fix error handling in H5B_shared_new. r17959: Fix Coverity item 209. Added an assertion for leaf->shared in H5B2_cache_leaf_dest. r17960: Fix Coverity item 208. Added an assertion for internal->shared in H5B2_cache_internal_dest. r17961: Fix Coverity item 89. Reworked the code to avoid array overrun in H5C__autoadjust__ageout__insert_new_marker. r17962: Fix for coverity Resource_leak 195,203,204,205. r17963: Fix Coverity item 44. Prevented potential NULL dereference in H5D_btree_debug. r17964: Fix Coverity issues #197, 198 & 199: memory not being released. (Also clean up other resource leaks in nearby and/or similar code). r17965: Fix Coverity issue #151: release resources on error r17966: Fix Coverity issue #187: Remove leftover code remnant from prior bugfix which was causing resource leak of open files. r17967: Fixed Coverity issues # 193 & 194. Removed unnecessary memory allocation and added comparison of length of path parameter to the size of the destination buffer in h5import.h/h5import.c. r17968: Fix Coverity item 144. Fixed memory leak on error in H5D_chunk_copy. r17969: Fix for coverity Resource_leak #196. r17970: Coverity 167-173: Initialized pointer of buffers. In error handling, closed types and free memory. Tested on: FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode (h5committested on trunk)
* [svn-r17704] Bug fix: (1192)Albert Cheng2009-10-211-1/+1
| | | | | | | Changed exit(1) to exit(EXIT_FAILURE) and exit(0) to exit(EXIT_SUCCESS) for better coding. Tested: jam.
* [svn-r16664] merge 16663 from trunkPedro Vicente Nunes2009-04-031-139/+148
| | | | | | | | note : lots of weird warnings of the type ../../../../hdf5/tools/h5repack/h5repacktst.c:3919: warning: will never be executed appear on the same code as 1.9 for the 1.8 branch tested: linux
* [svn-r15680] Pedro Vicente Nunes2008-09-221-4/+8
| | | | | Formatted code Tested: windows, linux
* [svn-r15663] fix commentsPedro Vicente Nunes2008-09-191-8/+15
| | | | tested: linux
* [svn-r15662] Cleaned compiler warnings Pedro Vicente Nunes2008-09-191-11/+11
| | | | | | | | Bug fixes: change the options block size variable from int to hsize_t, to match the H5Pset_userblock call, do not print print compression ratio when the requested filter is NONE Tested: windows, linux
* [svn-r15642] Pedro Vicente Nunes2008-09-171-3/+6
| | | | | | | | Add a default user block size of 1024 when none is given add a failure condition of writing user block Tested: windows, linux
* [svn-r15629] Description:Quincey Koziol2008-09-161-51/+51
| | | | | | | | | | | | | Remove trailing whitespace from C/C++ source files, with the following script: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Tested on: Mac OS X/32 10.5.5 (amazon) No need for h5committest, just whitespace changes...
* [svn-r15558] 1) There are 2 new command line parameters for h5repack. ↵Pedro Vicente Nunes2008-08-291-2/+13
| | | | | | | | | | | | | | Together they allow a call to H5Pset_alignment to be made -t T, --threshold=T Threshold value for H5Pset_alignment -a A, --alignment=A Alignment value for H5Pset_alignment 2) bug fix the printing of the dataset name was not done for references (verbose mode) tested: windows, linux
* [svn-r15533] #1184Pedro Vicente Nunes2008-08-261-3/+27
| | | | | | | | | | | Add a userblock to an HDF5 file during the repack. The user gives give a filename and userblock size as command line parameters to h5repack and the contents of that file are stored in the userblock for the HDF5 file created by h5repack. New flags to handle this -u and -b Tested : windows, linux
* [svn-r14368] clean code and reshufle functions between modules, remove ↵Pedro Vicente Nunes2008-01-031-119/+0
| | | | | | unused functions tested: windows, linux
* [svn-r14346] New feature: has_filters_obj function that checks if filters ↵Pedro Vicente Nunes2007-12-141-1/+1
| | | | | | | | | | | | | | | | | | | | match for each individual dataset Following the new feature of h5repack to allow multiple filters for all datasets and the new function has_filters that checks if the repacked file has all the filters requested, I added a new function has_filters_obj that does the same for each dataset. The previous function that checked this only ckecked if the user input filters were in the output dataset. This new function does this but checks if the filters are exactly the same. Currently the behavior of h5repack is to delete all filters that are present in the input file (dataset) and replace them with the requested ones, so they must match exactly. We might consider adding other logical operations, like keep the existing ones. Additionally , the function also checks if the filter parameters match. While doing this I noticed that for the shuffle filter , the values returned do not match and also the same for the N-bit and scale-offset The new function that checks for the filter values fails then, and so I commented the h5repack tests that do this for the N-bit and scale-offset filter (previously for the same bug on the shuffle filter I added special code on the compare filter function but this is temporary until I find the issue) tested: windows, linux, solaris
* [svn-r14345] substituted the symbol CD_VALUES , number of filter client data ↵Pedro Vicente Nunes2007-12-141-1/+1
| | | | | | | | values, for H5Z_COMMON_CD_VALUES that is defined in the library tested: windows, linux, solaris
* [svn-r14320] Added a function that matches the filters on the dataset's DCPL ↵Pedro Vicente Nunes2007-12-041-0/+1
| | | | | | | | and in the requested list Used to verify if the filters requested are present in the output dataset tested: windows, linux, solaris
* [svn-r14299] new feature: add support for h5repack to handle several global ↵Pedro Vicente Nunes2007-11-281-352/+529
| | | | | | | | filters usage is to repeat the -f option tested: windows, linux, solaris
* [svn-r13975] Add --latest, -compact=<size>, ... options to use the lastest ↵Peter Cao2007-07-131-1/+24
| | | | | | file format. Add test for the options to the daily test.
* [svn-r13810] Pedro Vicente Nunes2007-05-241-3/+23
| | | | | | | | Minor tunings to output verbose messages: 1)when there is not a filter request do not print a message saying the filter was not apllied when the dataset was too small 2) avoid printing the message that has a list of objects to modify when there is none Tested:linux
* [svn-r13589] Pedro Vicente Nunes2007-04-041-1/+1
| | | | | | h5repack code cleaning (required reconfigure) tested: linux (32, 64, parallel), solaris
* [svn-r13253] Updated all C and C++ style source code files with the THG ↵Albert Cheng2007-02-071-2/+3
| | | | | | | | | copyright notice. Tested platform: Kagiso only since it is only a comment block change. If it works in one machine, it should work in all, I hope. Still need to check the parallel build on copper.
* [svn-r12928] code revisionPedro Vicente Nunes2006-11-161-2/+3
|
* [svn-r12736] Description:Quincey Koziol2006-10-091-5/+3
| | | | | | | | | | | | | | | | | | | Add "use the latest format" support for dataspace object header encode/ decode routines and clean up format a bit for the latest format (new to 1.8.x releases) Remove storing 'perm' parameter for array datatypes in memory and the file, and add test to make certain that if any user applications are attempting to store them, we get some reports back. (Should be unlikely, since the RefMan says that the parameter is not implemented and is unsupported). Carry those changes into the tests, etc. Clean up a bunch more compiler warnings. Tested on: FreeBSD/32 4.11 (sleipnir) w/threadsafe Linux/32 2.4 (heping) w/FORTRAN & C++ Linux/64 2.4 (mir) w/enable-1.6-compat
* [svn-r12720] Pedro Vicente Nunes2006-10-051-8/+12
| | | | | | added output of error messages using the tools library function error_msg, that prints the program name and error on the message tested: kagiso
* [svn-r12440] Purpose:Quincey Koziol2006-06-271-1/+1
| | | | | | | | | | | Code cleanup Description: Trim trailing whitespace in Makefile.am and C/C++ source files to make diffing changes easier. Platforms tested: None necessary, whitespace only change
* [svn-r12159] Purpose:Pedro Vicente Nunes2006-03-271-3/+5
| | | | | | | | | | | | | | | | | bug fix Description: h5repack was not dealing with family files Solution: use the toolslib function h5tools_open to open the file instead of H5Fopen in h5repack Platforms tested: linux solaris AIX Misc. update:
* [svn-r11889] Purpose:Quincey Koziol2006-01-241-0/+1
| | | | | | | | | | Bug fix Description: Fix compile problem with C++ compiler after recent changes. Platforms tested: FreeBSD 4.11 (sleipnir) w/CC=g++
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-43/+43
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Trim trailing whitespace, which is making 'diff'ing the two branches difficult. Solution: Ran this script in each directory: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-2/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r9121] Purpose:Pedro Vicente Nunes2004-08-191-49/+0
| | | | | | | | | | | | | | | | | | | | new feature Description: added a check that the chunk size must be smaller than pixels per block in SZIP request prints a message and exits, if not met Solution: Platforms tested: linux aix solaris Misc. update:
* [svn-r9106] Purpose:Pedro Vicente Nunes2004-08-171-6/+16
| | | | | | | | | | | | | | | | | bug fix Description: the option CHUNK:NONE (remove chunking ) was not setting the layout to contiguous Solution: Platforms tested: linux solaris AIX Misc. update:
* [svn-r8904] Purpose:Pedro Vicente Nunes2004-07-201-3/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | h5diff and h5repack changes Description: h5diff introduced the following four modes of output: Normal mode: print the number of differences found and where they occured Report mode: print the above plus the differences Verbose mode: print the above plus a list of objects and warnings Quiet mode: do not print output (h5diff always returns an exit code of 1 when differences are found) h5repack added an extra parameter for SZIP filter (coding method) the new syntax is -f SZIP=<pixels per block,coding> (pixels per block is a even number in 2-32 and coding method is 'EC' or 'NN') Example of use: ./h5repack -i file1 -o file2 -f SZIP=8,NN -v updated usage messages, test scripts and files accordingly Solution: Platforms tested: linux AIX solaris Misc. update:
* [svn-r8882] Purpose:Pedro Vicente Nunes2004-07-151-15/+21
| | | | | | | | | | | | | | | | | | | | | | | | bug fix, new feature Description: fixed bug in the parse function: cases where we have an already inserted name but there is a new name also example: -f dset1:GZIP=1 -l dset1,dset2:CHUNK=20x20 dset1 is already inserted, but dset2 must also be (it was not) added a CHECK_SZIP symbol to enable/disable checking of library related szip parameters added the print of the filter name in verbose mode (confirms visually that the filter was applied ) Solution: Platforms tested: linux solaris AIX Misc. update:
* [svn-r8869] Purpose:Pedro Vicente Nunes2004-07-131-14/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | h5repack changes Description: there were some requests to change some minor h5repack features h5repack only made a warning about a non available filter in verbose mode ( -v ) without -v it kept silent, and users sometimes missed this warning the request was that it should print this warning always. so, the new format, is e.g ./h5repack -i test_szip.h5 -o out.h5 Warning: dataset </dset_szip> cannot be read, SZIP filter is not available due to this, and to avoid a lot of these messages in the shell test script, I modified the script h5repack.sh so that it detects the presence of all filters in the environment (previously it only detected SZIP) the test files were also divided in more files , to make the script code easier to follow Solution: Platforms tested: linux AIX (no szip) solaris (no szip, no gzip ) Misc. update:
* [svn-r8781] James Laird2004-07-011-2/+1
| | | | | | | | | | | | | | | | Purpose: HDF5 now supports SZIP with no encoder. Description: SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled. Solution: Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC. Platforms tested: Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++) Misc. update:
generated by cgit v0.12 at 2024-08-24 17:17:58 (GMT)