| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
Purpose:
minor code changes for SDS conversion with unlimited dimension case
Description:
Solution:
Platforms tested:
eirene
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
parameter file
Description:
handle parameters in various special cases.
Users donot even have to use this file if they don't have those special needs.
1) memory optimization (should always be set to 0 without handling huge SDS array conversion)
2) unlimited dimension with zero current size, users can use this file to define
their chunk size.
Solution:
Platforms tested:
RedHat Zoot 6.2
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug fix
a feature added
Description:
1.conversion of unlimited dimension data with the current dimensional size 0
2. Use a parameter file to control some special cases:
1) To subdivide the huge array into hyperslabs, a memory buffer size has i
to be set.
2) when current dimensional size is 0, a default chunk size can be set.
Solution:
bug fixed 1. Old approach: the current dimensional size is set to H5S_UNLIMITED, which is
a huge number. The default chunk size is set as a *FIXED* default value.
2. New approach: the current dimensional size is set to the current value 0.
Users can provide the chunk size through a parameter file.
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
|
| |
|
|
|
|
|
|
|
|
| |
Purpose:
bug fixed
Description:
Make HDF4 dataset converted from Vdata extensible.
Solution:
Platforms tested:
RedHat 6.2(Linux)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backward Compatibility Fix
Description:
One of H5P[gs]et_buffer's parameters changed between v1.4 and the
development branch.
Solution:
Added v1.4 compat stuff around H5P[gs]et_buffer implementation and testing
to allow v1.4.x users to continue to use their source code without
modification.
These changes are for everything except the FORTRAN wrappers - I spoke with
Elena and she will make the FORTRAN wrapper changes.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
A new feature
Description:
While testing h4toh5 utility with real NASA files, we find an example that the data array(one SDS) is so big that it exceeds the physical memory of some machine(>128 MB) and the conversion failed. Before the smart hyperslab operation is out, I am dividing the whole SDS into smaller hyperslabs with each hyperslab propotational to the original SDS array dimensions. For example, a three dimension array with 1000*1000*1000 elements, I can divide them into eight 500*500*500 pieces. I can read and write each piece and remember their starting and ending points. In this way, the memory allocation failure can be avoided; however, it may not be the efficient way.
I've tested this feature using SDS without chunking. It works fine. However, when testing SDS with chunking, it is extremely slow. This happens to be a bug in HDF5 library now. Quincey may fix this later and give me a more efficient way to handle the problem. Currently all my testing files are with UNLIMITED dimensions, so in HDF5 the chunking feature will be required.
SO by default, this feature will not be turned on.
Solution:
see the above
Platforms tested:
linux 2.2.18
|
| |
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix a bunch of warnings
Platforms tested:
Linux 2.2 (eirene)
|
| |
|
|
|
|
|
|
| |
Purpose:
Changed to the new generic property list for dataset creation property
list.
Platforms tested:
Arabica, modi4 and Hawkwind
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup for better compatibility with C++ compilers
Description:
C++ compilers are choking on our C code, for various reasons:
we used our UNUSED macro incorrectly when referring to pointer types
we used various C++ keywords as variables, etc.
we incremented enum's with the ++ operator.
Solution:
Changed variables, etc.to avoid C++ keywords (new, class, typename, typeid,
template)
Fixed usage of UNUSED macro from this:
char UNUSED *c
to this:
char * UNUSED c
Switched the enums from x++ to x=x+1
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
| |
|
|
|
|
|
| |
Purpose:
add another test case.
Description:
Solution:
|
| |
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug fix
Description:
change PIXEL_INTERLACE to INTERLACE_PIXEL and other interlace mode description
to fit for the image specification.
Solution:
Platforms tested:
eirene, sol2.7
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
1. fix a bug
2. turn off a feature
Description:
1. change the output of GRgetiminfo from NULL to &interlace_mode.
2. turn off the feature to change line-interleaved feature into
pixel-interleaved feature since inconsistent behaviour is found
in GR interface.
Solution:
see above
Platforms tested:
eirene, arabica
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
add a real raster-24 bit testing for interlace mode.
Description:
1. GR interfaces will never create an HDF4 file with interlace mode other than
pixel interleaved. DF24 interfaces can create HDF4 file with different interleaved.
There are inconsistent behaviors between GRreqimageil and GRreadimage, data read into the memory will not behave properly if a new interlace mode is asked.
2. Currently HDF5 image spec. supports pixel interleaved and plane interleaved.
We make a real image file to test whether the converter is doing the right thing.
Solution:
We use DF24 bit APIs to generate a real image file that can be tested by H5view.
Platforms tested:
linux and sol2.7
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup (sorta)
Description:
When the first versions of the HDF5 library were designed, I remembered
vividly the difficulties of porting code from a 32-bit platform to a 16-bit
platform and asked that people use intn & uintn instead of int & unsigned
int, respectively. However, in hindsight, this was overkill and
unnecessary since we weren't going to be porting the HDF5 library to
16-bit architectures.
Currently, the extra uintn & intn typedefs are causing problems for users
who'd like to include both the HDF5 and HDF4 header files in one source
module (like Kent's h4toh5 library).
Solution:
Changed the uintn & intn's to unsigned and int's respectively.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
| |
|
|
|
|
|
|
|
|
| |
Purpose:
check-in the second time to update the handling of data transfer in h4toh5.
This will make up for the cvs conflict checking a couple hours ago.
Description:
Solution:
Platforms tested:
eirene
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
1) fix the implementation of image according to image specfication
2) fix two bugs of SDS implemention. the first one is
to handle the unlimited SDS with the first dimensional size set to 0.
the second one is to change the way how HDF5 dataset is written.
Description:
1) mapping 24-bit image to 3D arrays instead of 2D compound datatype.
2) previously forgot considering unlimited SDS with the size set to 0.
3) H5P_set_buffer seems not working well for a extremely small size.
Solution:
1) see above.
2) add a special case to deal with this.
3) don't use H5Pset_buffer.
Platforms tested:
RedHat Zoot 6.2
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New Features!
Description:
Start migrating the internal use of property lists in the library from the
older implementation to the new generic property lists.
Currently, only the dataset transfer property lists are migrated to the
new architecture, all the rest of the property list types are still using
the older architecture.
Also, the backward compatibility features are not implemented yet, so
applications which use dataset transfer properties may need to make the
following changes:
H5Pcreate(H5P_DATASET_XFER) -> H5Pcreate_list(H5P_DATASET_XFER_NEW)
and
H5Pclose(<a dataset transfer property list>) -> H5Pclose_list(id)
This still may have some bugs in it, especially with Fortran, but I should
be wrapping up those later today.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Fixing a boo-boo
Description:
There was a problem with the generated Dependencies file. It listed
the H5pubconf.h header file as being in the $(top_srcdir) directory
when it's in the $(top_builddir) directory.
Solution:
Regenerated it.
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
| |
Purpose:
Update
Description:
Updated the Dependencies file.
Solution:
Reran "make Dependencies" in the tools/h4toh5 directory.
|
| |
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug in the comment
Description:
The structure of HDF4 file is not correct in the orginal comment
Solution:
Correct the wrong comment and add more explanation
Platforms tested:
eirene
|
| |
|
|
|
|
|
|
|
| |
Purpose:
New features for adding attribute options and modifying testing files
Description:
Solution:
Platforms tested:
eirene,arabica
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
new features
Description:
1. add an option to convert HDF4 file without HDF4 specified attributes such as
HDF4_OBJECT_TYPE, HDF4_REF_NUM etc.
it can be done by inputting "h4toh5 -na input.hdf"
The default converter will still keep HDF4 specfied attributes.
2. Add compression features (gzip) for image too. Now the compressed HDF4 image
can be supported by using HDF5 gzip. Not sure whether tools can read it. Need to be tested.
3. Change SPACEPAD to NULLTERM for HDF4 dimensional name list. We can use variable length HDF5 string to represent these names, however currently H5dump and H5view cannot support variable length HDF5 string. converter will wait for other tools' update.
Solution:
Platforms tested:
eirene(Red Hat 6.2) and arabica(solaris 2.7)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Add definations of two new functions
Description:
Solution:
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Add a constant(compression level for gzip)
Description:
For compression issue
Solution:
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
|
| |
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug fix
Description:
User can define "Real Vdata" as user-defined attribute. By using VSisattr, we can check this out. In order to keep this piece of information, We use "Vdata attribute" in the converted HDF5 file to distingush this kind of Vdata from independent Vdata.
Solution:
see above
Platforms tested:
eirene(Linux)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug fix
Description:
When Vsisattr is true, this Vdata still needs to be converted as an independent
real "Vdata", We will add object type of this vdata as "Vdata attribute".
Solution:
erease the evaluation of Vsisattr call.
Platforms tested:
Linux(eirene)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
bug fix Adding more features
Description:
Bugs: 1) hdf4 dimensional scale data can be none, but the dim name can still defined by users, so number of hdf4 dimensional names and number of object reference may be different
Previously, this problem is not considered.
2) SDcheckempty will return true when fill value is set to HDF4 SDS, and then fill value information is lost
3) check whether SDS have fill value set although SDcheckempty return true.
Use H5Psetfillvalue and H5Dcreate in HDF5 part, still needs to wait for the new development of HDF5 and also need to investigate whether this part of code has bugs.
New features: compressed SDS will get compressed with gzip when it is converted. That will save some space.
[describe the bug, or describe the new feature, etc]
Solution:
See above and design document
Platforms tested:
eirene(linux)
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug fix
Description:
when creating sds dimensional scale dataset, I accidently created two sds dimensional scale dataset with the same name, (say using name "dim1" for both rank 2 and rank 3).
hdf4 library doesn't give me complaints, hdp dumpsds doesn't generate complaints. h4toh5 converter since assumed that dimensional scale name has to be unique, simply skip if finding the same dimensional name, so accidently the result from hdp dumpsds are the same as the result from h5dump until I implement library API and find this bug.
Solution:
make unique sds dimensional scale name for this test file. Will also need to modify testfiles later.
suggestions: somebody check hdf4 library to disallow the same dimensional scale nameOused for the same sds object.
Platforms tested:
eirene, and this check in will not affect daily test.
|
| |
|
|
|
|
|
|
|
| |
Purpose:
avoid a windows bug for string handling
Description:
Solution:
Platforms tested:
windows 2000, linux
|
| |
|
|
|
|
|
|
|
| |
Purpose:
change macro RGB into HDF5_RGB since RGB is defined on windows platforms
Description:
Solution:
Platforms tested:
eirene
|
| |
|
|
|
|
|
|
|
|
|
| |
Purpose:
a bug fix
Description:
uninitialize the start and edge value for test_ras8 and test_ras24 functions
Solution:
initialize
Platforms tested:
eirene,arabica
|
|
|
Code Movement
Description:
Moved tools code into their own special subdirectories.
Platforms tested:
Linux, Kelgia
|
