summaryrefslogtreecommitdiffstats
path: root/tools/h5repack/h5repack_copy.c
Commit message (Collapse)AuthorAgeFilesLines
* Split tools into src and test - remove foldersAllen Byrne2016-10-271-1581/+0
|
* [svn-r30113] Description:Quincey Koziol2016-06-281-1154/+1060
| | | | | | | | Clean up warnings (from 2774 -> 1560, with my standard debug build) Tested on: MacOSX/64 10.11.5 (amazon) w/serial, parallel & production (h5committest forthcoming)
* [svn-r29889] HDFFV-9775: Add enable-error-stack to tools lib and refactor ↵Allen Byrne2016-05-041-266/+264
| | | | h5repack error handling.
* [svn-r27133] - Add a new attribute function characterstic for format:Mohamad Chaarawi2015-06-011-1/+1
| | | | | | | | * H5_ATTR_FORMAT(X,Y,Z) __attribute__((format(X, Y, Z))) - Rename UNUSED attribute characterstic to H5_ATTR_UNUSED. - Rename NORETURN attribute characterstic to H5_ATTR_NORETURN tested with h5committest.
* [svn-r25531] remove use of C99 PRI that failed on windows in dtypes.cMohamad Chaarawi2014-08-111-1/+1
| | | | fix bugs where hid_t were assumed to be 32 bit ints or herr_t.
* [svn-r25496] Description:Quincey Koziol2014-07-301-1/+1
| | | | | | | | | Merge 64-bit ID changes from branch to trunk. (Plus a few minor cleanups that aren't on the branch) Tested on: Mac OSX/64 10.9.4 (amazon) w/C++ & FORTRAN (h5committested on branch already for a week)
* [svn-r24116] Correct indentation and create single exit point.Allen Byrne2013-09-091-1659/+1584
| | | | Tested : local linux
* [svn-r24104] HDFFV-8345: add User Defined filters to h5repack.Allen Byrne2013-09-061-3/+7
| | | | | | Reviewed in H5T-61 Tested: local linux - cmake and autotools
* [svn-r24050] HDFFV-8498: --minimum option was disabled by HDFFV-8214. ↵Allen Byrne2013-08-211-0/+10
| | | | | | Reverted code change and changed default to 0 from 1024. Changed limit test to use h5dump to compare repacked file instead of h5diff. Tested local linux with CMake
* [svn-r23407] Purpose: Jonathan Kim2013-03-201-10/+0
| | | | | | | | | | | | | HDFFV-8214 - h5repack failed converting small chunked dataset (size < 1K) to contiguous layout. Description: h5repack failed converting small chunked dataset (size < 1K) to contiguous layout. The first case was when chunk dim is bigger than the dataset dim (at leat one), h5repack failed with displaying error stacks. The other case is when chunk dim is smaller than the dataset dim, h5repack failed to change layout. Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), emu (solaris-BE),fred (mac64-LE), Windows (32-LE cmake), cmake (jam)
* [svn-r22805] Purpose:Jonathan Kim2012-09-241-0/+20
| | | | | | | | | | | | | HDFFV-8012 - h5repack changes max dims and cause failure if only "-f none" is used without changing layout for chunked dataset when a chunk dim is bigger than a dataset dim Description: "h5repack -f <obj>:NONE <file.h5> out.h5" command failed if source file contains chunked dataset and a chunk dim is bigger than a dataset dim. Another issue is that the command changed max dims if chunk dim is smaller than the dataset dim. These issue occurred when dataset size is smaller than 64k (compact size limit) Fixed them. Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
* [svn-r22646] Description:Quincey Koziol2012-08-081-1/+1
| | | | | | | | | Changes resulting from Klocwork static analysis tool, from Mark Miller @ LLNL (miller86@llnl.gov). Tested on: Mac OS X/64 10.7.4 (amazon) w/debug, C++ & FORTRAN, using gcc 4.7.x (too minor to require h5committest)
* [svn-r22277] Purpose:Jonathan Kim2012-04-111-5/+41
| | | | | | | | | | | | Fix for HDFFV-7993 - h5repack fails with error "chunk size must be <= maximum dimension size for fixed-sized dimensions" Description: Fixed a failure when change the chunk size of a specified chunked dataset with unlimited max dims. Also took care of converting to contiguous and compact from the dataset. Test cases were added and tagged with jira#. Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
* [svn-r22224] Description:Quincey Koziol2012-04-011-2/+2
| | | | | | | | | Clean up a few warnings and use constant values for free-space parameters for the metadata block size test. Tested on: Linux/64 2.6.18 (koala) w/production & szip (h5committested also)
* [svn-r22180] Description:Quincey Koziol2012-03-291-87/+65
| | | | | | | | | Add new "metadata block size" command line option ('-M <x>' or '--metadata_block_size=<x>') for h5repack. Tested on: Mac OSX/64 10.7.3 (amazon) w/debug) (h5committest upcoming)
* [svn-r22012] Purpose:Jonathan Kim2012-03-011-57/+269
| | | | | | | | | | | | | | | | Task for HDFFV-7862 - Select data by chunk direction to improve performance in h5repack Description: h5repack sometimes became very slow when handling big chunked datasets in certain cases. (when chunk boundary doesn't match with a hyperslab boundary.) The main issue was from figuring out a hypeslab without considering chunk boundary to read from and write to such datasets. The update was made to figure out a better hyperslab unit with considering chunk boundary to improve performance for such cases prior to the update. Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows
* [svn-r21974] Update HSIZE_T print statements to use correct macro.Allen Byrne2012-02-231-44/+44
| | | | | | Fix some more HD prefix issues in tools. Tested: local linux
* [svn-r21283] H5Eget_major() is depreciated. use the H5Eget_msg() instead.Peter Cao2011-08-221-6/+3
|
* [svn-r21265] h5repack: added macro to handle failure in H5Dread/write when ↵Peter Cao2011-08-191-11/+37
| | | | memory allocation failed inside the library
* [svn-r21105] Fixed two bugs:Peter Cao2011-07-151-0/+5
| | | | | | | - h5repack: h5repack failed to copy dataset if the layout is changed from chunked with unlimited dims to contiguous. (PC -- 2011/07/15) - h5diff: "--delta" option considers two NaN of the same type are different, which is wrong based on http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff. (PC -- 2011/07/15)
* [svn-r20103] Purpose:Jonathan Kim2011-02-141-20/+16
| | | | | | | | | | | | Change to use HDxxx macros. Description: Originally this started for fixing incorrect pointer usage. But that got fixed through coverity merge. So this is mainly changing to use HDxxx macros and clean up some related code. Tested: jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
* [svn-r20052] Description:Quincey Koziol2011-02-061-27/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clean up Coverity warnings, and fix some style issues: r19735: Fix for memory leak in test/mf found by valgrind. r19736: Fix memory leak in h5repack. The buffer in copy_objects, when copying the entire dataset at once, was not checked for the presence of a vlen, and vlen storage was never reclaimed. Added check and call to H5D_vlen_reclaim(). r19772: Change H5assert() to if (H5T_VLEN != src->shared->type || H5T_VLEN != dst->shared->type) HGOTO_ERROR(H5E_ARGS, H5E_BADTYPE, FAIL, "not a H5T_VLEN datatype") r19774: removed unused priv. r19775: removed unused variables r19778: Fix memory leak comparing for variable length data types. r19834: Fixed memory leaks found by valgrind. Memory errors remain for another day. Tested on: Mac OS X/32 10.6.6 (amazon) w/debug & production (h5committested on branch)
* [svn-r19818] Purpose:Jonathan Kim2010-11-181-3/+9
| | | | | | | Change to skip copying when a dataset is not allocated. Tested: jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE)
* [svn-r19392] Purpose:Jonathan Kim2010-09-151-3/+0
| | | | | | | | | | Additional fix for Bug 1821 - h5repack outputs compression information where it is not supposed to Description: Remove unnecessary lopping code, which only lowers performance. Tested: jam, amani, heiwa
* [svn-r19389] Purpose:Jonathan Kim2010-09-151-3/+9
| | | | | | | | | | Fix for Bug1896 h5repack - changing layout to COMPACT does not work Description: Make h5repack be able to convert a layout to COMPACT for small size dataset as default. Also add verifying layout changes in our test script. Tested: jam, amani, heiwa
* [svn-r19179] Description:Quincey Koziol2010-08-051-10/+6
| | | | | | | | | | | | | | | | | | | | | | Bring changes from Coverity branch back to trunk: r19079 & 19080: [BZ1942] h5dump -u to generate XML, it does not respect the -m option xml version of dump_data function didn't check for use of fp_format variable. Added new test expected file for committed bug 1942 r19103, 19104 & 19105: [BZ1821] h5repack -v did not display correct output for a selected compression. Needed new test for comparing output of -v option. Added new test file for solution to BZ1821 BZ1821 - Bring test changes from the shell script actually used. Tested on: Mac OS X/32 10.6.4 (amazon) debug & production (h5committested on branch)
* [svn-r18759] pull r18757 from 1.8 branchAllen Byrne2010-05-111-30/+29
| | | | | | | | | Enable tools lib to be built as a dll on windows. Added two get/set functions for progname and d_status. Also add windows import/export declarations to functions. Updated error_mesg() and warn_mesg() to remove progname argument and use get functions Tested: Windows, linux
* [svn-r18454] Purpose:Jonathan Kim2010-03-251-331/+0
| | | | | | | | | | | | Fix for the bug1726 - NPOESS: h5repack loses attributes for datasets of type H5T_REFERENCE. Description: include test cases. also test cases for attribute with object and region reference. Tested: jam, amani, linew
* [svn-r18197] Description:Quincey Koziol2010-01-301-117/+117
| | | | | | | | | Trim trailing whitespace from source code files with this command: find . \( -name "*.[ch]" -or -name "*.cpp" -or -name "*.f90" \) -print |xargs -n 1 sed -i "" 's/[[:blank:]]*$//' Tested on: None - eyeballed only
* [svn-r17582] Description:Quincey Koziol2009-10-021-1/+45
| | | | | | | | | | | | | | | | | | | | | Bring changes from file free space branch back to the trunk. *yay!* Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode Mac OS X/32 10.5.8 (amazon) in debug mode Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r17313] Description:Quincey Koziol2009-08-101-6/+2
| | | | | | | | | | | | | | | | | | | | | Bring back various minor code cleanups from the file free space branch Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode Mac OS X/32 10.5.8 (amazon) in debug mode Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r16800] Purpose: Fix bug 1516Neil Fortner2009-04-201-20/+217
| | | | | | | | | | | Description: h5repack previously would not take named datatypes into consideration when copying datasets and attributes. This would cause extra anonymous datatypes in the target file at best, and cause errors halfway through the repacking at worst. h5repack should now always handle named datatypes correctly. Named datatypes are also now converted to the native type when -n is given. Tested: jam, linew, smirom (h5committest)
* [svn-r16656] #1402 (B1) h5repack does not preserve creation order indexing. Pedro Vicente Nunes2009-04-021-678/+705
| | | | | | | | ISSUE : h5repack does not handle group creation order flags. ACTION: call H5P(g)(s)et_link_creation_order functions when handling groups, add new groups with these flags to the test generation program, and verify results in the test program. TEST: in the test program, function that compares property lists, added code to verify groups tested: windows, linux, solaris
* [svn-r16614] 3. #1501 (B1) tools bug if dataset is larger than ↵Pedro Vicente Nunes2009-03-251-3/+7
| | | | | | | | | | | | H5TOOLS_BUFSIZE limit. ISSUE : the tools use the following formula to read by hyperslabs: hyperslab_size[i] = MIN( dim_size[i], H5TOOLS_BUFSIZE / datum_size) where H5TOOLS_BUFSIZE is a constant defined of 1024K. This is OK as long as the datum_size does not exceed 1024K, otherwise we have a hyperslab size of 0 (since 1024K/(greater than 1024K) = 0). This affects h5dump. h5repack, h5diff SOLUTION: add a check for a 0 size and define as 1 if so. TEST FOR H5DUMP: Defined a case in the h5dump test generator program of such a type (an array type of doubles with a large array dimension, that was the case the user reported). Since the written file commited in svn would be around 1024K, opted for not writing the data (the part of the code where the hyperslab is defined is executed, since h5dump always reads the files). Defined a macro WRITE_ARRAY to enable such writing if needed. Added a run on the h5dump shell script. Added 2 new files to svn: tools/testfiles/tarray8.ddl, tools/testfiles/tarray8.h5. NOTE: while doing this I thought of adding this dataset case to an existing file, but that would add the large array output to those files (the ddls). The issue is that the file list is increasing. TEST FOR H5DIFF: for h5diff the check for reading by hyperslabs is H5TOOLS_MALLOCSIZE (128 * H5TOOLS_BUFSIZE) or 128 Mb. This makes it not possible to add such a file to svn, so used the same method as h5dump (only write the dataset if WRITE_ARRAY is defined). As opposed to h5dump, the hyperslab code is NOT executed when the dataset is empty (dataset is not read). Added the new dataset to existing files and shell run (tools/h5diff/testfiles/h5diff_dset1.h5 and tools/h5diff/testfiles/h5diff_dset2.h5 and output in tools/h5diff/testfiles/h5diff_80.txt). TEST FOR H5REPACK: similar issue as h5diff with the difference that the hyperslab code is run. Added a run to the shell script (with a filter, otherwise the code uses H5Ocopy). tested: linux (h5commitest failed , apparently it did not detect the code changes in /tools/lib that fix the bug: the error in an assertion in the hyperslab of 0. I am sure that making h5ccomitest --distclean will detect the new code , but don't want to wait more 3 hours :-) )
* [svn-r16112] #1375. Define a default chunk whose size is manageable. Defined ↵Pedro Vicente Nunes2008-11-211-7/+10
| | | | | | currently as the same size of hyperslab (that compares the dimension sizes against a predefined constant size and chooses the minimum between the two) Tested : linux, windows
* [svn-r15792] Bug fix. When detecting the user requested filters, the "all" ↵Pedro Vicente Nunes2008-10-061-37/+65
| | | | | | option was not taken in consideration when printing the compression ratio Tested: windows, linux
* [svn-r15652] Cleaned compiler warnings Pedro Vicente Nunes2008-09-181-14/+52
| | | | | | Bug fix: do not print print compression ratio when the requested filter is NONE Tested: windows, linux
* [svn-r15643] add a failure condition of writing user blockPedro Vicente Nunes2008-09-171-161/+178
| | | | Tested: windows, linux
* [svn-r15628] Description:Quincey Koziol2008-09-161-150/+150
| | | | | | | | | | | | | Remove trailing whitespace from C/C++ source files, with the following script: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Tested on: Mac OS X/32 10.5.5 (amazon) No need for h5committest, just whitespace changes...
* [svn-r15551] 1) There are 2 new command line parameters for h5repack. ↵Pedro Vicente Nunes2008-08-281-4/+44
| | | | | | | | | | | | Together they allow a call to H5Pset_alignment to be made -t T, --threshold=T Threshold value for H5Pset_alignment -a A, --alignment=A Alignment value for H5Pset_alignment 2) bug fix the printing of the dataset name was not done for references (verbose mode) tested: windows, linux
* [svn-r15547] fix typosPedro Vicente Nunes2008-08-271-1/+1
| | | | tested: linux
* [svn-r15527] turn off a debug print flagPedro Vicente Nunes2008-08-251-1/+1
| | | | tested: windows, linux
* [svn-r15526] userblock optionsPedro Vicente Nunes2008-08-251-4/+5
| | | | | | only write the user block of a pre-existing file with user block in case there is no input user block file tested: windows, linux
* [svn-r15521] #1184Pedro Vicente Nunes2008-08-221-5/+170
| | | | | | | | | | Add a userblock to an HDF5 file during the repack. The user gives give a filename and userblock size as command line parameters to h5repack and the contents of that file are stored in the userblock for the HDF5 file created by h5repack. New flags to handle this -u and -b Tested : windows, linux
* [svn-r15432] bug fixPedro Vicente Nunes2008-08-051-6/+8
| | | | | | the compression ratio was being printed for cases where a layout was requested and not a filter tested: windows, linux
* [svn-r14913] Introduced a compression ratio = uncompressed size / ↵Pedro Vicente Nunes2008-05-011-13/+15
| | | | | | | | compressed size in the printing of the compression with 3 digits of precision per hdf-forum NASA developers suggestion tested: windows, linux
* [svn-r14413] Description:Quincey Koziol2008-01-141-1/+1
| | | | | | | | | | | | | | | | | | | Change H5P[gs]et_format_bounds() => H5P[gs]et_libver_bounds() and also enumerated values H5F_FORMAT_{EARLIEST, LATEST} => H5F_LIBVER_{EARLIEST, LATEST} Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14362] Description:Quincey Koziol2008-01-031-1/+1
| | | | | | | | | | | | | | | | | | Switched from "H5P[gs]et_latest_format" to "H5P[gs]et_format_bounds". Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14361] add a warning message in case a dataset cannot be created with ↵Pedro Vicente Nunes2008-01-021-1/+7
| | | | | | modified DCPL tested: linux
* [svn-r14358] Description:Quincey Koziol2007-12-231-5/+5
| | | | | | | | | | | | | | | | | | | Tweak the constants for the shared message flags to be equal to the other flags useds for the underlying messages. Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
generated by cgit v0.12 at 2024-08-26 19:33:36 (GMT)