path: root/release_docs/INSTALL

Instructions for the Installation of HDF5 Software
==================================================

This file provides instructions for installing the HDF5 software.

For help with installing, questions can be posted to the HDF Forum or sent to the HDF Helpdesk:

   HDF Forum:     https://forum.hdfgroup.org/
   HDF Helpdesk:  https://portal.hdfgroup.org/display/support/The+HDF+Help+Desk

CONTENTS
--------
        1. Obtaining HDF5

        2. Quick installation
        2.1. Windows
        2.2. RedStorm (Cray XT3)

        3. HDF5 dependencies
        3.1. Zlib
        3.2  Szip (optional)
        3.3. MPI and MPI-IO

        4. Full installation instructions for source distributions
        4.1. Unpacking the distribution
        4.1.1. Non-compressed tar archive (*.tar)
        4.1.2. Compressed tar archive (*.tar.Z)
        4.1.3. Gzip'd tar archive (*.tar.gz)
        4.1.4. Bzip'd tar archive (*.tar.bz2)
        4.2. Source versus build directories
        4.3. Configuring
        4.3.1. Specifying the installation directories
        4.3.2. Using an alternate C compiler
        4.3.3. Additional compilation flags
        4.3.4. Compiling HDF5 wrapper libraries
        4.3.5. Specifying other programs
        4.3.6. Specifying other libraries and headers
        4.3.7. Static versus shared linking
        4.3.8. Optimization versus symbolic debugging
        4.3.9. Parallel versus serial library
        4.3.10. Threadsafe capability
        4.3.11. Backward compatibility
        4.4. Building
        4.5. Testing
        4.6. Installing HDF5

        5. Using the Library


*****************************************************************************

1. Obtaining HDF5
        The latest supported public release of HDF5 is available from
        https://www.hdfgroup.org/downloads/hdf5/.  For Unix and UNIX-like
        platforms, it is available in tar format compressed with gzip.
        For Microsoft Windows, it is in ZIP format.


2. Quick installation
        For those who don't like to read ;-) the following steps can be used
        to configure, build, test, and install the HDF5 library, header files,
        and support programs. For example, to install HDF5 version X.Y.Z at
        location /usr/local/hdf5, use the following steps.

            $ gunzip < hdf5-X.Y.Z.tar.gz | tar xf -
            $ cd hdf5-X.Y.Z
            $ ./configure --prefix=/usr/local/hdf5 <more configure_flags>
            $ make
            $ make check                # run test suite.
            $ make install
            $ make check-install        # verify installation.

        Some versions of the tar command support the -z option. In such cases,
        the first step above can be simplified to the following:

            $ tar zxf hdf5-X.Y.Z.tar.gz

        <configure_flags> above refers to the configure flags appropriate
        to your installation.  For example, to install HDF5 with the
        Fortran and C++ interfaces and with SZIP compression, the
        configure line might read as follows:

            $ ./configure --prefix=/usr/local/hdf5 --enable-fortran \
                          --enable-cxx --with-szlib=PATH_TO_SZIP

        In this case, PATH_TO_SZIP would be replaced with the path to the
        installed location of the SZIP library.

2.1. Windows
        Users of Microsoft Windows should see the INSTALL_Windows files for
        detailed instructions.

2.2. RedStorm (Cray XT3)
        Users of the Red Storm machine, after reading this file, should read
        the Red Storm section in the INSTALL_parallel file for specific
        instructions for the Red Storm machine.  The same instructions would
        probably work for other Cray XT3 systems, but they have not been
        verified.


3. HDF5 dependencies
3.1. Zlib
        The HDF5 library includes a predefined compression filter that
        uses the "deflate" method for chunked datasets. If zlib-1.1.2 or
        later is found, HDF5 will use it.  Otherwise, HDF5's predefined
        compression method will degenerate to a no-op; the compression
        filter will succeed but the data will not be compressed.

3.2. Szip (optional)
        The HDF5 library includes a predefined compression filter that
        uses the extended-Rice lossless compression algorithm for chunked
        datasets. For information on Szip compression, license terms,
        and obtaining the Szip source code, see:

           https://portal.hdfgroup.org/display/HDF5/Szip+Compression+in+HDF+Products

        Building instructions are available with the Szip source code.

        The HDF Group does not distribute separate Szip precompiled libraries,
        but the HDF5 pre-built binaries provided on The HDF Group download page
        include the Szip library with the encoder enabled. These can be found
        here:

           https://www.hdfgroup.org/downloads/hdf5/

        To configure the HDF5 library with the Szip compression filter, use
        the '--with-szlib=/PATH_TO_SZIP' flag. For more information, see
        section 4.3.7, "Specifying other libraries and headers."

        Please notice that if HDF5 configure cannot find a valid Szip library,
        configure will not fail; in this case, the compression filter will
        not be available to the applications.

        To check if Szip compression was successfully configured in,
        check the "I/O filters (external):" line in the configure output,
        summary section, printed to the standard output.

3.3. MPI and MPI-IO
        The parallel version of the library is built upon the foundation
        provided by MPI and MPI-IO. If these libraries are not available
        when HDF5 is configured, only a serial version of HDF5 can be built.


4. Full installation instructions for source distributions

4.1. Unpacking the distribution
        The HDF5 source code is distributed in a variety of formats which
        can be unpacked with the following commands, each of which creates an
        'hdf5-X.Y.Z' directory, where X.Y.Z is the HDF5 version numbers.

4.1.1. Non-compressed tar archive (*.tar)

            $ tar xf hdf5-X.Y.Z.tar

4.1.2. Compressed tar archive (*.tar.Z)

            $ uncompress -c < hdf5-X.Y.Z.tar.Z | tar xf -
            Or
            $ tar Zxf hdf5-X.Y.Z.tar.Z

4.1.3. Gzip'd tar archive (*.tar.gz)

            $ gunzip < hdf5-X.Y.Z.tar.gz | tar xf -
            Or
            $ tar zxf hdf5-X.Y.Z.tar.gz

4.1.4. Bzip'd tar archive (*.tar.bz2)

            $ bunzip2 < hdf5-X.Y.Z.tar.bz2 | tar xf -
            Or
            $ tar jxf hdf5-X.Y.Z.tar.bz2

4.2. Source versus build directories
        On most systems the build can occur in a directory other than the
        source directory, allowing multiple concurrent builds and/or
        read-only source code. In order to accomplish this, one should
        create a build directory, cd into that directory, and run the
        `configure' script found in the source directory (configure
        details are below). For example,
            $ mkdir built-fortran
            $ cd build-fortran
            $ ../hdf5-X.Y.Z/configure --enable-fortran ...


4.3. Configuring
        HDF5 uses the GNU autoconf system for configuration, which
        detects various features of the host system and creates the
        Makefiles. On most systems it should be sufficient to say:

            $ ./configure
            Or
            $ sh configure

        The configuration process can be controlled through environment
        variables, command-line switches, and host configuration files.
        For a complete list of switches type:

            $ ./configure --help

        The host configuration files are located in the `config'
        directory and are based on architecture name, vendor name, and/or
        operating system which are displayed near the beginning of the
        `configure' output. The host config file influences the behavior
        of configure by setting or augmenting shell variables.

4.3.1. Specifying the installation directories
        The default installation location is the HDF5 directory created in
        the build directory. Typing `make install' will install the HDF5
        library, header files, examples, and support programs in hdf5/lib,
        hdf5/include, hdf5/doc/hdf5/examples, and hdf5/bin. To use a path
        other than hdf5, specify the path with the `--prefix=PATH' switch:

            $ ./configure --prefix=/usr/local

        If shared libraries are being built (the default), the final
        home of the shared library must be specified with this switch
        before the library and executables are built.

        HDF5 can be installed into a different location than the prefix
        specified at configure time; see section 4.6, "Installing HDF5,"
        for more details.

4.3.2. Using an alternate C compiler
        By default, configure will look for the C compiler by trying
        `gcc' and `cc'. However, if the environment variable "CC" is set
        then its value is used as the C compiler.  For instance, one would
        use the following line to specify the native C compiler on a system
        that also has the GNU gcc compiler (users of csh and derivatives
        will need to prefix the commands below with `env'):

            $ CC=cc ./configure

        A parallel version of HDF5 can be built by specifying `mpicc'
        as the C compiler.  (The `--enable-parallel' flag documented
        below is optional in this case.)  Using the `mpicc' compiler
        will insure that the correct MPI and MPI-IO header files and
        libraries are used.

            $ CC=/usr/local/mpi/bin/mpicc ./configure


4.3.3. Additional compilation flags
        If additional flags must be passed to the compilation commands,
        specify those flags with the CFLAGS variable. For instance,
        to enable symbolic debugging of a production version of HDF5, one
        might say:

            $ CFLAGS=-g ./configure --enable-build-mode=production

4.3.4. Compiling HDF5 wrapper libraries
        One can optionally build the Fortran, C++, and Java interfaces to
        the HDF5 C library. By default, these options are disabled. To build
        them, specify '--enable-fortran', '--enable-cxx', or '--enable-java',
        respectively.

            $ ./configure --enable-fortran
            $ ./configure --enable-cxx
            $ ./configure --enable-java

        Configuration will halt if a working Fortran 90 or 95 compiler or
        C++ compiler is not found. Currently, the Fortran configure tests
        for these compilers in order: f90, pgf90, f95. To use an
        alternate compiler specify it with the FC variable:

            $ FC=/usr/local/bin/g95 ./configure --enable-fortran


4.3.5. Specifying other programs
        The build system has been tuned for use with GNU make but also
        works with other versions of make.  If the `make' command runs a
        non-GNU version but a GNU version is available under a different
        name (perhaps `gmake'), then HDF5 can be configured to use it by
        setting the MAKE variable. Note that whatever value is used for
        MAKE must also be used as the make command when building the
        library:

            $ MAKE=gmake ./configure
            $ gmake

        The `AR' and `RANLIB' variables can also be set to the names of
        the `ar' and `ranlib' (or `:') commands to override values
        detected by configure.

        The HDF5 library, include files, and utilities are installed
        during `make install' (described below) with a BSD-compatible
        install program detected automatically by configure. If none is
        found, the shell script bin/install-sh is used. Configure does not
        check that the install script actually works; if a bad install is
        detected on your system (e.g., on the ASCI blue machine as of
        March 2, 1999) you have two choices:

            1. Copy the bin/install-sh program to your $HOME/bin
               directory, name it `install', and make sure that $HOME/bin
               is searched before the system bin directories.

            2. Specify the full path name of the `install-sh' program
               as the value of the INSTALL environment variable. Note: do
               not use `cp' or some other program in place of install
               because the HDF5 makefiles also use the install program to
               change file ownership and/or access permissions.

4.3.6. Specifying other libraries and headers
        Configure searches the standard places (those places known by the
        systems compiler) for include files and header files. However,
        additional directories can be specified by using the CPPFLAGS
        and/or LDFLAGS variables:

            $ CPPFLAGS=-I/home/robb/include \
              LDFLAGS=-L/home/robb/lib \
              ./configure

        HDF5 uses the zlib library to support the HDF5 deflate
        data compression filter.  Configure searches the standard places
        (plus those specified above with the CPPFLAGS and LDFLAGS variables)
        for the zlib headers and library. The search can be disabled by
        specifying `--without-zlib' or alternate directories can be specified
        with `--with-zlib=INCDIR,LIBDIR' or through the CPPFLAGS and LDFLAGS
        variables:

            $ ./configure --with-zlib=/usr/unsup/include,/usr/unsup/lib

            $ CPPFLAGS=-I/usr/unsup/include \
              LDFLAGS=-L/usr/unsup/lib \
              ./configure

        HDF5 includes Szip as a predefined compression method (see 3.2).
        To enable Szip compression, the HDF5 library must be configured
        and built using the Szip library:

            $ ./configure --with-szlib=/Szip_Install_Directory

4.3.7. Static versus shared linking
        The build process will create static libraries on all systems and
        shared libraries on systems that support dynamic linking to a
        sufficient degree. Either form of the library may be suppressed by
        saying `--disable-static' or `--disable-shared'.

            $ ./configure --disable-shared

        Shared C++ and Fortran libraries will be built if shared libraries
        are enabled.

        To build only statically linked executables on platforms which
        support shared libraries, use the `--enable-static-exec' flag.

            $ ./configure --enable-static-exec

4.3.8. Optimization versus symbolic debugging
        The library can be compiled to provide symbolic debugging support
        so it can be debugged with gdb, dbx, ddd, etc., or it can be
        compiled with various optimizations. To compile for symbolic
        debugging (the default for snapshots), say
        `--enable-build-mode=production'; to compile with optimizations
        (the default for supported public releases),
        say `--enable-build-mode=production'. On some systems the
        library can also be compiled for profiling with gprof by saying
        `--enable-production=profile'.

            $ ./configure --enable-build-mode=debug       #symbolic debugging
            $ ./configure --enable-build-mode=production  #optimized code
            $ ./configure --enable-production=profile     #for use with gprof

        Regardless of whether support for symbolic debugging is enabled,
        the library can also perform runtime debugging of certain packages
        (such as type conversion execution times and extensive invariant
        condition checking). To enable this debugging, supply a
        comma-separated list of package names to the `--enable-internal-debug'
        switch.

        Debugging can be disabled by saying `--disable-internal-debug'.
        The default debugging level for snapshots is a subset of the
        available packages; the default for supported releases is no
        debugging (debugging can incur a significant runtime penalty).

            $ ./configure --enable-internal-debug=s,t  #debug only H5S and H5T
            $ ./configure --enable-internal-debug      #debug normal packages
            $ ./configure --enable-internal-debug=all  #debug all packages
            $ ./configure --disable-internal-debug     #no debugging

        HDF5 can also print a trace of all API function calls, their
        arguments, and the return values. To enable or disable the
        ability to trace the API say `--enable-trace' (the default for
        snapthots) or `--disable-trace' (the default for public releases).
        The tracing must also be enabled at runtime to see any output.

4.3.9. Parallel versus serial library
        The HDF5 library can be configured to use MPI and MPI-IO for
        parallelism on a distributed multi-processor system.  Read the
        file INSTALL_parallel for detailed information.

        The threadsafe, C++ and Java interfaces are not compatible
        with the parallel option.
        Unless --enable-unsupported has been specified on the configure line,
        the following options must be disabled:
            --enable-threadsafe, --enable-cxx, --enable-java


4.3.10. Threadsafe capability
        The HDF5 library can be configured to be thread-safe (on a very
        large scale) with the `--enable-threadsafe' flag to the configure
        script.  Some platforms may also require the '-with-pthread=INC,LIB'
        (or '--with-pthread=DIR') flag to the configure script.
        For further information, see:

           https://portal.hdfgroup.org/display/knowledge/Questions+about+thread-safety+and+concurrent+access

        The high-level, C++, Fortran and Java interfaces are not compatible
        with the thread-safety option because the lock is not hoisted
        into the higher-level API calls.
        Unless --enable-unsupported has been specified on the configure line,
        the following options must be disabled:
            --enable-hl, --enable-cxx, --enable-fortran, --enable-java


4.3.11. Backward compatibility
        The 1.10 version of the HDF5 library can be configured to operate
        identically to the v1.8 library with the
            --with-default-api-version=v18
        configure flag, or identically to the v1.6 library with the
            --with-default-api-version=v16
        configure flag. This allows existing code to be compiled with the
        v1.10 library without requiring immediate changes to the application
        source code.  For additional configuration options and other details,
        see "API Compatibility Macros":

            https://portal.hdfgroup.org/display/HDF5/API+Compatibility+Macros

4.4. Building
        The library, confidence tests, and programs can be built by
        specifying:

            $ make

        Note that if you have supplied some other make command via the MAKE
        variable during the configuration step, that same command must be
        used here.

        When using GNU make, you can add `-j -l6' to the make command to
        compile in parallel on SMP machines. Do not give a number after
        the `-j' since GNU make will turn it off for recursive invocations
        of make.

            $ make -j -l6

4.5. Testing
        HDF5 comes with various test suites, all of which can be run by
        specifying:

            $ make check

        To run only the tests for the library, change to the `test'
        directory before issuing the command. Similarly, tests for the
        parallel aspects of the library are in `testpar' and tests for
        the support programs are in `tools'.

        The `check' consists of two sub-tests, check-s and check-p, which
        are for serial and parallel tests, respectively.  Since serial tests
        and parallel tests must be run with single and multiple processes
        respectively, the two sub-tests work nicely for batch systems in
        which the number of processes is fixed per batch job.  One may submit
        one batch job, requesting 1 process, to run all the serial tests by
        "make check-s"; and submit another batch job, requesting multiple
        processes, to run all the parallel tests by "make check-p".

        Temporary files will be deleted by each test when it completes,
        but may continue to exist in an incomplete state if the test
        fails. To prevent deletion of the files, define the HDF5_NOCLEANUP
        environment variable.

        The HDF5 tests can take a long time to run on some systems.  To perform
        a faster (but less thorough) test, set the HDF5TestExpress environment
        variable to 2 or 3 (with 3 being the shortest run).  To perform a
        longer test, set HDF5TestExpress to 0.  1 is the default.

4.6. Installing HDF5
        The HDF5 library, include files, and support programs can be
        installed by specifying `make install'. The files are installed under the
        directory specified with `--prefix=DIR' (or if not specified, in 'hdf5'
        in the top directory of the HDF5 source code). They will be
        placed in directories named `lib', `include', and `bin'. The directories,
        if not existing, will be created automatically, provided the mkdir command
        supports the -p option.

        If `make install' fails because the install command at your site
        somehow fails, you may use the install-sh that comes with the
        source. You will need to run ./configure again.

            $ INSTALL="$PWD/bin/install-sh -c" ./configure ...
            $ make install

        If you want to install HDF5 in a location other than the location
        specified by the `--prefix=DIR' flag during configuration (or
        instead of the default location, `hdf5'), you can do that
        by running the deploy script:

            $ bin/deploy NEW_DIR

        This will install HDF5 in NEW_DIR.  Alternately, you can do this
        manually by issuing the command:

            $ make install prefix=NEW_DIR

        where NEW_DIR is the new directory where you wish to install HDF5.
        If you do not use the deploy script, you should run h5redeploy in
        NEW_DIR/bin directory.  This utility will fix the h5cc, h5fc and
        h5c++ scripts to reflect the new NEW_DIR location.

        The library can be used without installing it by pointing the
        compiler at the `src' and 'src/.libs' directory for include files and
        libraries. However, the minimum which must be installed to make
        the library publicly available is:

            The library:
                ./src/.libs/libhdf5.a

            The public header files:
                ./src/H5*public.h, ./src/H5public.h
                ./src/H5FD*.h except ./src/H5FDprivate.h,
                ./src/H5api_adpt.h

            The main header file:
                ./src/hdf5.h

            The configuration information:
                ./src/H5pubconf.h

            The support programs that are useful are:
            ./tools/h5ls/h5ls               (list file contents)
            ./tools/h5dump/h5dump           (dump file contents)
            ./tools/misc/h5repart           (repartition file families)
            ./tools/misc/h5debug            (low-level file debugging)
            ./tools/h5import/h5import       (imports data to HDF5 file)
            ./tools/h5diff/h5diff           (compares two HDF5 files)
            ./tools/gifconv/h52gif          (HDF5 to GIF converter)
            ./tools/gifconv/gif2h5          (GIF to HDF5 converter)


5. Using the Library

        For information on using HDF5 see the documentation, tutorials and examples
        found here:

           https://portal.hdfgroup.org/display/HDF5/HDF5

        A summary of the features included in the built HDF5 installation can be found
        in the libhdf5.settings file in the same directory as the static and/or
        shared HDF5 libraries.