path: root/release_docs/RELEASE.txt

HDF5 version 1.6.6-pre3 currently under development
================================================================================

INTRODUCTION
============
This document describes the differences between HDF5-1.6.5 and
HDF5-1.6.6  It contains information on the platforms tested and
known problems in HDF5-1.6.6. For more details, check the HISTORY.txt
file in the HDF5 source.

The HDF5 documentation can be found on the NCSA ftp server
(ftp.ncsa.uiuc.edu) in the directory:

     /HDF/HDF5/docs/

Documentation for the current release is also on the HDF web site:

     http://hdf.ncsa.uiuc.edu/HDF5/doc/

For more information look at the HDF5 home page at:

    http://hdf.ncsa.uiuc.edu/HDF5/

If you have any questions or comments, please send them to:

    help@hdfgroup.org


CONTENTS
========
- New Features
- Support for New Platforms, Languages and Compilers
- Bug Fixes since HDF5-1.6.5
- Documentation
- Platforms Tested
- Supported Configuration Features Summary
- Known Problems


New Features
============
    Configuration:
    -------------------------
        - HDF5 now uses autoconf 2.61 to generate configure. MAM - 2007/7/25.

    Source code distribution:
    -------------------------

    Library:
    --------
        - None

    Parallel Library:
    -----------------
        - None
      
    Tools:
    ------
       - h52gif and gif2h5: Both these tools were revised to include the High
         Level Image API support, and tests were added to /hl/tools/gif2h5.
          PVN - 2007/04/13
       - h5dump: added support for double long type H5T_NATIVE_LDOUBLE. PVN
          2007/03/13
       - h5dump: added support for binary output, see usage. PVN 2007/03/13
       - h5repack: added support for reading and writing  by hyperslabs for
         large files. PVN - 2007/03/01
       - h5repack: a new option allows the copy using the file type (default)
          instead of the previous conversion to native type. PVN - 2007/03/01
       - h5repack: output the percentage of compression used. PVN - 2007/03/01
       - h5diff: added support for comparing dataset regions. PVN - 2007/02/20
       - h5diff: added support for reading and comparing by hyperslabs for large files.
          PVN - 2007/02/20
       - h5diff: printing of dataset dimensions along with dataset name. PVN -
          2007/02/19

    F90 API:
    --------
       - None

    C++ API:
    --------
	- Added support for szip: DSetCreatPropList::setSzip - BMR 2007/01/24





Support for New Platforms, Languages and Compilers
==================================================
	- Added support for MAC Intel (Darwin 8.8.2)  (gcc 4.0.1 and g95 0.91)
				EIP - 2007/02/13
        - Added support for GNU C compiler version 4.2.1 for FreeBSD 6.2 
          (32 and 64-bit)


Bug Fixes since HDF5-1.6.5 Release
==================================

    Library
    -------
        - Fixed several bugs with writing fill values for datasets that have
            a variable-length datatype or component datatype.  QAK - 2007/06/19
        - STDIO driver didn't support files bigger than 2GB because the OFFSET
            parameter of fseek is of type LONG INT, not big enough for big files.
            Use fseeko instead for big files if it's available on the system.
            SLU - 2007/4/5
        - Relaxed restrictions on attribute operations to allow a file ID to
            be used as the "location ID".  If a file ID is used, the attribute
            operation will occur on the root group of the file.
            - QAK - 2007/02/09
        - Fixed a bug in H5Sselect_valid() that caused an incorrect value to
            be returned (0) instead of FAIL on error confitions such as the
            selection or extent not being defined. CMC - 2007/01/05
        - Fixed potential file corruption bug which could overwrite a portion
            of an object's header when an attribute was renamed.  If the new
            name for the attribute was longer than the old name, it's possible
            that the attribute could grow enough to overwrite another message
            in the object's header.  QAK - 2007/01/02
        - Fixed file corruption bug which could write an incorrect number of
            messages to an object's header under certain circumstances.
            Generally, the sequence of actions to generate this bug looks
            like this:
                - Create an object
                - Close the file
                - Re-open the file
                - Add 2 (or more) attributes to the object
                - Close the file
                - Re-open the file
                - Delete one of the attributes on the object
                - Add a smaller attribute to the object
                - Delete the smaller atttribute on the object
                - Add a larger attribute on the object

            After this, the number of header messages stored for the object
            will be off by one.  Other sequences of modifying attributes on an
            object could also trigger this bug.  If you are opening an
            object and the bottom few messages of the HDF5 error stack
            resembles this, the object has been affected by this bug:

              #007: ../../hdf5_v1.6/src/H5C.c line 3887 in H5C_load_entry(): unable to load entry
                major(08): Meta data cache layer
                minor(40): Unable to load metadata into cache
              #008: ../../hdf5_v1.6/src/H5Ocache.c line 332 in H5O_load(): corrupt object header - too few messages
                major(12): Object header layer
                minor(40): Unable to load metadata into cache

            Specifically, "corrupt object header" is the best string to search
            for in the HDF5 error stack output.

            If your files have been affected by this bug, or you are concerned
            that your files might have been, please contact the HDF Helpdesk
            at help@hdfgroup.org for a tool to detect and repair files
            affected by this problem.  QAK - 2006/6/16

        - Fixed various problems with retrieving names of objects, especially
            with mounted files.  QAK - 2005/12/25

    Configuration
    -------------
	- Changed to default to --disable-shared if parallel is enabled. - AKC
	  2007/5/12
	- Correct an coding error in configure when it tries to locate the
	  needed MPI and MPI-IO library for the fortran interface. - AKC
	  2007/5/9.
	- Changed default fortran compiler to g95 when gcc is used. - AKC
          2007/2/17.
	- Configure can now use any tr command.  No more need for
	  defining variable TR nor is it supported. -AKC 2006/05/20

    Performance
    -------------
       - None
   
    Tools
    -----
       - Fixed a bug on h5dump regarding the display of named datatypes
          attributes. PVN 2007/03/13
       - Fixed a bug on h5dump regarding the display of group comments. PVN
          2007/03/13
       - Fixed a bug on h5dump regarding the display of hardlinks pointing 
           to the root group. PVN 2007/03/13
       - Fixed a bug in the h5diff percentage option -p. PVN 2007/03/05
       - Fixed a bug in h5dump that caused array indices greater than 2 ^32-1
         not to be printed correctly - PVN 2007/2/19

    Documentation 
    ------------- 

    F90 API
    -------
       - None
                                
    C++ API
    -------
	- Changed
	        StrType::StrType(const size_t& size);
	  to
	        StrType::StrType(const int dummy, const size_t& size);
	  because the first one clashed with
	        StrType::StrType(const hid_t existing_id); - BMR 2007/05/03
	- Wrappers of H5Rcreate had incorrect prototypes.  Added these
	  overloaded functions:
	        IdComponent::reference(void* ref, const char* name, DataSpace&
	                dataspace, H5R_type_t ref_type = H5R_DATASET_REGION)
	        IdComponent::void reference(void* ref, const char* name)
	        IdComponent::reference(void* ref, const H5std_string& name)
	  and will remove these incorrect member functions:
	        DataSet::Reference
	        DataType::Reference
	        Group::Reference
	        H5File::Reference
	- Added wrappers for H5Rdereference as constructors
	        DataSet(IdComponent& obj, void* ref);
	        Group(IdComponent& obj, void* ref);
	        DataType(IdComponent& obj, void* ref);


   

Platforms Tested
================

    AIX 5.2 (32/64 bit)           xlc 6.0.0.8
    (datastar)                    xlC 6.0.0.9
                                  xlf 8.1.1.7
                                  mpcc_r 6.0.0.8
                                  mpxlf_r 8.1.1.7
    AIX 5.3 (32/64 bit)           xlc 7.0.0.0
    (copper)                      xlC 7.0.
                                  xlf 9.1.0.3
    FreeBSD 6.2 (32 and 64-bit)   gcc  and g++3.4.6
    (duty and liberty)            gcc and g++ 4.2.1

    HP-UX B.11.23                 HP aC++/ANSI C B3910B A.06.02
    (sirius)                      HP F90 v3.1 (32-bit only)
                                  HP aC++/ANSI C B3910B A.06.02
    
    IRIX64 6.5                    MIPSpro cc 7.4.4m
    (ucar mop1 64 & n32)          F90 MIPSpro 7.4.4m (64 only)
                                  C++ MIPSpro cc 7.4.4m

    Linux 2.4.21-47.ELsmp #1 SMP  gcc 3.2.3
    i686 i386 GNU/Linux
    (osage)

    Linux 2.6.9-42.0.10.ELsmp #1  gcc (GCC) 3.4.6
    SMP i686 i386                 G95 (GCC 4.0.3 (g95 0.91!) Nov 21 2006)
    (kagiso)                      PGI C, Fortran, C++ 6.2-5 32-bit 
                                  icc (ICC) 9.1 
                                  Intel(R) C++ Compiler for 32-bit applications, 
                                  Version 9.1
                                  Intel(R) Fortran Compiler for 32-bit applications, 
                                  Version 9.1 
                                  Absoft 32-bit Fortran 95 10.0.4
                                  MPICH mpich2-1.0.4p1 compiled with  
                                  gcc 3.4.6 and G95 (GCC 4.0.3 (g95 0.91!) 

    Linux 2.6.16.46-0.12-debug #1 
          SMP ia64 GNU/Linux      Intel(R) C++ Version 10.0.025
    (ucar hir1)                   Intel(R) Fortran Itanium(R) Version 10.0.025

    Linux 2.6.16.46-0.14-smp #1   Intel(R) C++ for Intel(R) EM64T Ver. 9.1.037
	 SMP x86_64 GNU/Linux     Intel(R) Fortran Intel(R) EM64T Ver. 9.1.031
    (smirom)			  PGI C, Fortran, C++ Version 6.2-5
				         for 64-bit target on x86-64
                                  tested for both 32 and 64-bit binaries
    Linux 2.6.5-7.283-rtgfx Altix 
          SMP ia64                Intel(R) C++ Version 9.0
    (cobalt)                      Intel(R) Fortran Itanium(R) Version 9.0
                                  SGI MPI

    OSF1 V5.1 (QSC)               Compaq C V6.5-011 
    (see known problems.)         HP Fortran V5.5A-3548
				  Compaq C++ V6.5-036  
				  MPIX200_64_r13.4

    SunOS 5.8 32 and 64-bit       Sun WorkShop 6 update 2 C 5.3
    (sol)                         Sun WorkShop 6 update 2 Fortran 95 6.2
                                  Sun WorkShop 6 update 2 C++ 5.3

    SunOS 5.10 32 and 64-bit      Sun WorkShop 6 update 2 C 5.8
    (linew)                       Sun WorkShop 6 update 2 Fortran 95 8.2
                                  Sun WorkShop 6 update 2 C++ 5.8
                                  Patch 121019-06 

    Xeon Linux 2.4.21-32.0.1.ELsmp-perfctr-lustre
    (tungsten)	                  Intel(R) C++ Version 9.0
                                  Intel(R) Fortran Compiler Version 9.0
				  PGI C Version 6.0
				  PGI C++ Version 6.0
				  PGI Fortran Version 6.0
                                  gcc (GCC) 3.2.2 (Red Hat Linux 3.2.2-5)

    Xeon Linux 2.6.9-42.0.10.EL_lustre-1.4.10.1smp
    (abe)                         Intel(R) C++ Version 10.0
                                  Intel(R) Fortran Compiler Version 10.0
                                  Open MPI 1.2.2
    IA-64 Linux 2.4.21.SuSE_292.til1  gcc (GCC) 3.2.2
    ia64  (NCSA tg-login)         Intel(R) C++ Version 8.0
                                  Intel(R) Fortran Compiler Version 8.0
				  mpich-gm-1.2.5..10-intel-r2
    Windows XP                    Visual Studio 6.0
                                  Visual Studio .NET (with Intel Fortran 9.0)
                                  Visual Studio 2005 (with Intel Fortran 9.0)
                                  cygwin (gcc 3.4.4)
    MAC OS X Intel                Darwin 8.10.1
                                  i686-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1
                                  g95 0.91


Supported Configuration Features Summary
========================================

Key:   y   = tested and supported
       n   = not supported or not tested in this release
       x   = not working in this release
       dna = does not apply
       ( ) = footnote appears below second table

Platform                             C        F90    F90      C++  zlib  SZIP
                                     parallel        parallel      


Platform                             Shared        static-  Thread-  STREAM- 
                                     libraries(4)  exec     safe     VFD    

Compiler versions for each platform are listed in the "Platforms Tested" 
table found elsewhere in this file (RELEASE.txt).  Unless otherwise noted, 
compilers used are the system compilers.

Footnotes:  


Known Problems
==============
* QSC (an HP alpha-based OSF1 cluster) does not create h5pfc correctly. It
  does not include the -lmpi properly.  This will be fixed in the next
  release.  AKC - 2007/08/07.

* On IRIX64 6.5(UCAR SGI mop1), when testing HDF5 tools with HDF5 shared
  library, especially for h5repacktst and h52gifgentst; The following (or
  similar error message may appear:
  "523648:lt-h5repacktst: rld: Fatal Error: Cannot Successfully map soname
  'libh5test.so.1' under any of the filenames .......(bunch of directories)
  "
  And the testing will fail. 
  We believe this is a libtool problem. One way to get rid of this is to
  add the paths of libh5test.so.1 and libh5.so.1 to the shared library path.

  For 32-bit:
  you may do this, under csh,
  setenv LD_LIBRARYN32_PATH .......(existing pathes):[full path of HDF5
  directory/test/.libs]:[full path of HDF5 directory/src/.libs]

  For 64-bit:
  setenv LD_LIBRARY64_PATH ......(existing pathes):[full path of HDF5
  directory/test/.libs]:[full path of HDF5 directory/src/.libs]

  NOTE: This problem ONLY affects the testing of HDF5 library when you build
  from source. It won't affect any applications who would like to link with HDF5
  shared library since the shared library path needs to be set anyway.
  KY - 2007/8/2

* QSC (an HP alpha-based OSF1 cluster) failed the testpar/testphdf5 sub-test
  "calloc".  All other tests passed.  This indicates dataset using chunked
  storage created by serial HDF5 may not work properly with parallel HDF5.
  The calloc test can be skipped by running "prun ... testphdf5 -x calloc".
  AKC - 2007/7/12.

* Newer versions of g95 are not compatible with v1.6.5.  Last known g95
  version that is still compatiable with v1.6.5 is May 17, 2006.
  AKC - 2006/11/29.

* Intel Compilers for Linux x86_86 platforms (EM64T-based, v8.1) has
  optimization error in the data types conversion code.  Before running
  configure, edit the file config/intel-flags by changing the setting of
  PROD_CFLAGS from -O3 to -O0, then run configure.  AKC - 2005/11/10.

* When testing parallel HDF5 with the C compiler version MIPSpro 7.4.3 at IRIX
  6.5, set enviroment variable MPI_TYPE_MAX to be a bigger number, for example 
  120000, in order to pass the complicated collective IO tests inside parallel 
  HDF5 library. This is not a problem inside parallel HDF5 library. You can 
  always set a bigger number in your system.
                                  KY - 2005/10/6

* A contiguous or chunked dataset created by a sequential version may
  not be modified with a parallel version of the library. 
  Use the H5Pset_alloc_time function with H5D_ALLOC_TIME_EARLY to set up the 
  dataset creation property list to avoid the problem.
                                    EIP - 2005/09/09

* The dataset created or rewritten with the v1.6.3 library or after can't 
  be read with the v1.6.2 library or before when Fletcher32 EDC(filter) is 
  enabled.  There was a bug in the calculating code of the Fletcher32 
  checksum in the library before v1.6.3.  The checksum value wasn't consistent 
  between big-endian and little-endian systems.  This bug was fixed in 
  Release 1.6.3.  However, after fixing the bug, the checksum value is no 
  longer the same as before on little-endian system.  The library release 
  after 1.6.4 can still read the dataset created or rewritten with the library
  of v1.6.2 or before.  SLU - 2005/7/8

* For version 6 (6.02 and 6.04) of the Portland Group compiler on AMD Opteron
  processor, there's a bug in the compiler for optimization(-O2).  The library
  failed in several tests but all related to multi driver.  The problem has 
  been reported to the vendor.  

* test/big fails sometimes with the message "Possible overlap with another
  region."  The test selects regions randomly, and this error occurs when
  two regions overlap each other; it is a bug in the test and not in
  HDF5.  Since the error is triggered by a random situation, it will
  usually disappear if the test is re-run.

* Newer SGI MIPSpro compilers (version 7.4.x) support C99 features but it
  has a "guard" statement in stdint.h that will #error and skip the rest
  of the header file if C99 option is not used explicitly.  Hardset
  $CC to c99 will resolve the problem. AKC - 2004/12/13

* On IBM AIX systems, parallel HDF5 mode will fail some tests with error
  messages like "INFO: 0031-XXX ...".  This is from the command poe.
  Set the environment variable MP_INFOLEVEL to 0 to minimize the messages
  and run the tests again.
  The tests may fail with messages like "The socket name is already
  in use".  HDF5 does not use sockets (except for stream-VFD).  This is
  due to problems of the poe command trying to set up the debug socket.
  Check if there are many old /tmp/s.pedb.* staying around.  These are
  sockets used by the poe command and left behind due to failed commands.
  Ask your system administrator to clean them out.  Lastly, request IBM
  to provide a means to run poe without the debug socket.

* The h5dump tests may fail to match the expected output on some platforms
  (e.g. parallel jobs, Windows) where the error messages directed to
  "stderr" do not appear in the "right order" with output from stdout.
  This is not an error.

* The stream-vfd test uses ip port 10007 for testing. If another
  application is already using that port address, the test will hang
  indefinitely and has to be terminated by the kill command. To try the
  test again, change the port address in test/stream_test.c to one not
  being used in the host.

* The --enable-static-exec configure flag fails to compile for Solaris
  platforms. This is due to the fact that not all of the system
  libraries on Solaris are available in a static format.

  The --enable-static-exec configure flag also fails to correctly compile
  on IBM SP2 platform for the serial mode. The parallel mode works fine
  with this option.

  The --enable-static-exec configure flag also fails to correctly compile
  on the HPUX 11.00.
  
  It is suggested that you don't use this option on these platforms
  during configuration.

* The Stream VFD is not supported under Windows.


* Use --disable-shared configure flag if building with Absoft Fortran
  compiler.

* Information about building with PGI and Intel compilers is available in
  INSTALL file sections 5.7 and 5.8.

* In LANL QSC, the new cc compiler has problems converting small values of
  long long (absolute values less than 1**-308) to double.  This triggers
  the test/dtypes to report failure in the
      Testing random sw long double -> double conversions
  If -ieee is used, the converted doubles spread over the range 0.0 to 1**-308.
  If -ieee is not used, the converted double values are mostly 0.0 but
  occasionally as 1**-308.  This has been reported to the system staff.
  All other tests have passed.

* On at least one system, SDSC DataStar, the scheduler (in this case
  LoadLeveler) sends job status updates to standard error when you run
  any executable that was compiled with the parallel compilers.

  This causes problems when running "make check" on parallel builds, as
  many of the tool tests function by saving the output from test runs,
  and comparing it to an exemplar.

  The best solution is to reconfigure the target system so it no longer
  inserts the extra text.  However, this may not be practical.

  In such cases, one solution is to "setenv HDF5_Make_Ignore yes" prior to
  the configure and build.  This will cause "make check" to continue after
  detecting errors in the tool tests.  However, in the case of SDSC DataStar,
  it also leaves you with some 150 "failed" tests to examine by hand.

  A second solution is to write a script to run serial tests and filter
  out the text added by the scheduler.  A sample script used on SDSC
  DataStar is given below, but you will probably have to customize it
  for your installation.

  Observe that the basic idea is to insert the script as the first item
  on the command line which executes the test.  The script then
  executes the test and filters out the offending text before passing
  it on.

        #!/bin/csh

        set STDOUT_FILE=~/bin/serial_filter.stdout
        set STDERR_FILE=~/bin/serial_filter.stderr

        rm -f $STDOUT_FILE $STDERR_FILE

        ($* > $STDOUT_FILE) >& $STDERR_FILE

        set RETURN_VALUE=$status

        cat $STDOUT_FILE

        tail +3 $STDERR_FILE

        exit $RETURN_VALUE

  You get the HDF make files and test scripts to execute your filter script
  by setting the environment variable "RUNSERIAL" to the full path of the
  script prior to running configure for parallel builds.  Remember to
  "unsetenv RUNSERIAL" before running configure for a serial build.

  Note that the RUNSERIAL environment variable exists so that we can
  prefix serial runs as necessary on the target system.  On DataStar,
  no prefix is necessary.  However, on an MPICH system, the prefix might
  have to be set to something like "/usr/local/mpi/bin/mpirun -np 1" to
  get the serial tests to run at all.

  In such cases, you will have to include the regular prefix in your
  filter script.