From 5ffb19669bb798b5240705cb14bf82f6196ee43e Mon Sep 17 00:00:00 2001 From: lrknox Date: Sun, 3 Jun 2018 14:16:17 -0500 Subject: Update various INSTALL files for 1.8.21 release. --- release_docs/INSTALL | 194 +++++++++++++++----------------------- release_docs/INSTALL_Cygwin.txt | 13 +-- release_docs/INSTALL_parallel | 28 ++++-- release_docs/USING_HDF5_CMake.txt | 4 +- release_docs/USING_HDF5_VS.txt | 4 +- 5 files changed, 105 insertions(+), 138 deletions(-) diff --git a/release_docs/INSTALL b/release_docs/INSTALL index 812d7ec..124dba9 100644 --- a/release_docs/INSTALL +++ b/release_docs/INSTALL @@ -2,18 +2,19 @@ Instructions for the Installation of HDF5 Software ================================================== This file provides instructions for installing the HDF5 software. -If you have any problems with the installation, please see The HDF Group's -support page at the following location: - http://www.hdfgroup.org/services/support.html +For help with installing, questions can be posted to the HDF Forum or sent to the HDF Helpdesk: + + HDF Forum: https://forum.hdfgroup.org/ + HDF Helpdesk: https://portal.hdfgroup.org/display/support/The+HDF+Help+Desk CONTENTS -------- 1. Obtaining HDF5 2. Quick installation - 2.1. UNIX platforms - 2.2. Windows and Cygwin + 2.1. Windows and Cygwin + 2.2. RedStorm (Cray XT3) 3. HDF5 dependencies 3.1. Make @@ -46,23 +47,17 @@ CONTENTS 5. Using the Library - 6. Support - - A. Building and testing with other compilers - A.1. Building and testing with Intel compilers - A.2. Building and testing with PGI compilers ***************************************************************************** 1. Obtaining HDF5 The latest supported public release of HDF5 is available from - https://support.hdfgroup.org/HDF5/release/obtain5.html. For Unix and - UNIX-like platforms, it is available in tar format compressed with gzip. + https://www.hdfgroup.org/downloads/hdf5/. For Unix and UNIX-like + platforms, it is available in tar format compressed with gzip. For Microsoft Windows, it is in ZIP format. 2. Quick installation -2.1. UNIX platforms For those who don't like to read ;-) the following steps can be used to configure, build, test, and install the HDF5 Library, header files, and support programs. For example, to install HDF5 version X.Y.Z at @@ -98,10 +93,17 @@ CONTENTS In this case, PATH_TO_SZIP would be replaced with the path to the installed location of the SZIP library. -2.2. Windows and Cygwin +2.1. Windows and Cygwin Users of Microsoft Windows should see the INSTALL_Windows files for detailed instructions. INSTALL_Cygwin also exists for those platforms. +2.2. RedStorm (Cray XT3) + Users of the Red Storm machine, after reading this file, should read + the Red Storm section in the INSTALL_parallel file for specific + instructions for the Red Storm machine. The same instructions would + probably work for other Cray XT3 systems, but they have not been + verified. + 3. HDF5 dependencies 3.1. Make @@ -119,17 +121,19 @@ CONTENTS 3.3. Szip (optional) The HDF5 Library includes a predefined compression filter that uses the extended-Rice lossless compression algorithm for chunked - datasets. For more information about Szip compression and license - terms, see http://hdfgroup.org/doc_resource/SZIP/. + datasets. For information on Szip compression, license terms, + and obtaining the Szip source code, see: + + https://portal.hdfgroup.org/display/HDF5/Szip+Compression+in+HDF+Products - The Szip source code can be obtained from the HDF5 Download page - http://www.hdfgroup.org/HDF5/release/obtain5.html#extlibs. Building - instructions are available with the Szip source code. + Building instructions are available with the Szip source code. The HDF Group does not distribute separate Szip precompiled libraries, - but the HDF5 binaries available from - http://www.hdfgroup.org/HDF5/release/obtain5.html include - the Szip encoder enabled binary for the corresponding platform. + but the HDF5 pre-built binaries provided on The HDF Group download page + include the Szip library with the encoder enabled. These can be found + here: + + https://www.hdfgroup.org/downloads/hdf5/ To configure the HDF5 Library with the Szip compression filter, use the '--with-szlib=/PATH_TO_SZIP' flag. For more information, see @@ -165,16 +169,12 @@ CONTENTS $ gunzip < hdf5-X.Y.Z.tar.gz | tar xf - Or $ tar zxf hdf5-X.Y.Z.tar.gz - Or - $ tar xf hdf5-X.Y.Z.tar.gz 4.1.3. Bzip'd tar archive (*.tar.bz2) $ bunzip2 < hdf5-X.Y.Z.tar.bz2 | tar xf - Or $ tar jxf hdf5-X.Y.Z.tar.bz2 - Or - $ tar xf hdf5-X.Y.Z.tar.bz2 4.2. Source versus build directories @@ -237,11 +237,13 @@ CONTENTS $ CC=cc ./configure - A parallel version of HDF5 can be built by specifying `mpicc' as - the C compiler. Using the `mpicc' compiler will insure that the - correct MPI and MPI-IO header files and libraries are used. + A parallel version of HDF5 can be built by specifying `mpicc' + as the C compiler. (The `--enable-parallel' flag documented + below is optional in this case.) Using the `mpicc' compiler + will insure that the correct MPI and MPI-IO header files and + libraries are used. - $ CC=/usr/local/mpi/bin/mpicc ./configure --enable-parallel + $ CC=/usr/local/mpi/bin/mpicc ./configure 4.3.3. Configuring for 64 or 32 bit support Some machine architectures support 32-bit or 64-bit binaries. @@ -294,7 +296,7 @@ CONTENTS fort lf95 g95 ifc efc gfc. To use an alternate compiler specify it with the FC variable: - $ FC=/usr/local/bin/gfortran ./configure --enable-fortran --enable-fortran2003 + $ FC=/usr/local/bin/g95 ./configure --enable-fortran --enable-fortran2003 Note: The Fortran and C++ interfaces are not supported on all the platforms the main HDF5 Library supports. Also, the Fortran @@ -399,10 +401,7 @@ CONTENTS (such as type conversion execution times and extensive invariant condition checking). To enable this debugging, supply a comma-separated list of package names to to the `--enable-debug' - switch. See "Debugging HDF5 Applications" for a list of package - names: - - http://www.hdfgroup.org/HDF5/doc/H5.user/Debugging.html + switch. Debugging can be disabled by saying `--disable-debug'. The default debugging level for snapshots is a subset of the @@ -418,22 +417,21 @@ CONTENTS arguments, and the return values. To enable or disable the ability to trace the API say `--enable-trace' (the default for snapshots) or `--disable-trace' (the default for public releases). - The tracing must also be enabled at runtime to see any output - (see "Debugging HDF5 Applications," reference above). + The tracing must also be enabled at runtime to see any output. 4.3.10. Parallel versus serial library The HDF5 Library can be configured to use MPI and MPI-IO for parallelism on a distributed multi-processor system. Read the - file INSTALL_parallel for detailed explanations. + file INSTALL_parallel for detailed information. 4.3.11. Threadsafe capability The HDF5 Library can be configured to be thread-safe (on a very large scale) with the `--enable-threadsafe' flag to the configure script. Some platforms may also require the '-with-pthread=INC,LIB' (or '--with-pthread=DIR') flag to the configure script. - For further details, see "HDF5 Thread Safe Library": + For further information, see: - http://www.hdfgroup.org/HDF5/doc/TechNotes/ThreadSafeLibrary.html + https://portal.hdfgroup.org/display/knowledge/Questions+about+thread-safety+and+concurrent+access 4.3.12. Backward compatibility The 1.8 version of the HDF5 Library can be configured to operate @@ -441,14 +439,14 @@ CONTENTS --with-default-api-version=v16 configure flag. This allows existing code to be compiled with the v1.8 library without requiring immediate changes to the application - source code. For addtional configuration options and other details, - see "API Compatibility Macros in HDF5": + source code. For additional configuration options and other details, + see "API Compatibility Macros": - http://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html + https://portal.hdfgroup.org/display/HDF5/API+Compatibility+Macros 4.4. Building The library, confidence tests, and programs can be built by - saying just: + specifying: $ make @@ -465,7 +463,7 @@ CONTENTS 4.5. Testing HDF5 comes with various test suites, all of which can be run by - saying + specifying $ make check @@ -494,12 +492,13 @@ CONTENTS longer test, set HDF5TestExpress to 0. 1 is the default. 4.6. Installing HDF5 - The HDF5 Library, include files, and support programs can be - installed in a (semi-)public place by saying `make install'. The - files are installed under the directory specified with `--prefix=DIR' - (default is 'hdf5' in the build directory) in directories named `lib', - `include', and `bin'. The directories, if not existing, will be created - automatically, provided the mkdir command supports the -p option. + The HDF5 library, include files, and support programs can be + installed by specifying `make install'. The files are installed under the + directory specified with `--prefix=DIR' (or if not specified, in 'hdf5' + in the top directory of the HDF5 source code). They will be + placed in directories named `lib', `include', and `bin'. The directories, + if not existing, will be created automatically, provided the mkdir command + supports the -p option. If `make install' fails because the install command at your site somehow fails, you may use the install-sh that comes with the @@ -544,78 +543,35 @@ CONTENTS The configuration information: ./src/H5pubconf.h - The support programs that are useful are: - ./tools/h5ls/h5ls (list file contents) - ./tools/h5dump/h5dump (dump file contents) - ./tools/misc/h5repart (repartition file families) - ./tools/misc/h5debug (low-level file debugging) - ./tools/h5import/h5import (imports data to HDF5 file) - ./tools/h5diff/h5diff (compares two HDF5 files) - ./tools/gifconv/h52gif (HDF5 to GIF converter) - ./tools/gifconv/gif2h5 (GIF to HDF5 converter) + Useful support programs installed in bin and built in + subdirectories of tools/: + h5ls/h5ls (list file contents) + h5dump/h5dump (dump file contents) + h5copy/h5copy (copy objects to another file) + h5repack/h5repack (copy file changing compression/chunking) + h5jam/h5jam (add user block to front of HDF5 file) + h5jam/h5unjam (splits user block to separate file) + misc/h5repart (repartition file families) + misc/h5debug (low-level file debugging) + misc/h5mkgrp (create a new HDF5 group in a file) + h5import/h5import (imports data to HDF5 file) + h5diff/h5diff (compares two HDF5 files) + h5stat/h5stat (reports HDF5 file and object statistics) + gifconv/h52gif (HDF5 to GIF converter) + gifconv/gif2h5 (GIF to HDF5 converter) + misc/h5redeploy (update HDF5 compiler tools after + installing HDF5 in a new location) + 5. Using the Library - Please see the "HDF5 User's Guide" and the "HDF5 Reference Manual": - - http://www.hdfgroup.org/HDF5/doc/ - - Most programs will include and link with -lhdf5. - Additional libraries may also be necessary depending on whether - support for compression, etc., was compiled into the HDF5 Library. - - A summary of the HDF5 installation can be found in the - libhdf5.settings file in the same directory as the static and/or - shared HDF5 Libraries. - - -6. Support - Support is described in the README file. - - -***************************************************************************** - APPENDICES -***************************************************************************** - -A. Building and testing with other compilers -A.1. Building and testing with Intel compilers - When Intel compilers are used (icc or ecc), you will need to modify - the generated "libtool" program after configuration is finished. - On or around line 104 of the libtool file, there are lines which - look like: - - # How to pass a linker flag through the compiler. - wl="" - - Change these lines to this: - - # How to pass a linker flag through the compiler. - wl="-Wl," - - UPDATE: This is now done automatically by the configure script. - However, if you still experience a problem, you may want to check this - line in the libtool file and make sure that it has the correct value. - -A.2. Building and testing with PGI compilers - When PGI C and C++ compilers are used (pgcc or pgCC), you will need to - modify the generated "libtool" program after configuration is finished. - On or around line 104 of the libtool file, there are lines which - look like this: - - # How to pass a linker flag through the compiler. - wl="" - - Change these lines to this: - # How to pass a linker flag through the compiler. - wl="-Wl," + For information on using HDF5 see the documentation, tutorials and examples + found here: - UPDATE: This is now done automatically by the configure script. However, - if you still experience a problem, you may want to check this line in - the libtool file and make sure that it has the correct value. + https://portal.hdfgroup.org/display/HDF5/HDF5 - To build the HDF5 C++ Library with pgCC (version 4.0 and later), set - the environment variable CXX to "pgCC -tlocal" - setenv CXX "pgCC -tlocal" - before running the configure script. + A summary of the features included in the built HDF5 installation can be found + in the libhdf5.settings file in the same directory as the static and/or + shared HDF5 libraries. diff --git a/release_docs/INSTALL_Cygwin.txt b/release_docs/INSTALL_Cygwin.txt index 4c92e41..5ebb503 100644 --- a/release_docs/INSTALL_Cygwin.txt +++ b/release_docs/INSTALL_Cygwin.txt @@ -66,12 +66,11 @@ Preconditions: 2.2.2 Szip The HDF5 library has a predefined compression filter that uses the extended-Rice lossless compression algorithm for chunked - datatsets. For more information about Szip compression and - license terms see - http://hdfgroup.org/HDF5/doc_resource/SZIP/index.html. + datatsets. For more information on Szip compression, license terms, + and obtaining the Szip source code, see: + + https://portal.hdfgroup.org/display/HDF5/Szip+Compression+in+HDF+Products - The latest supported public release of SZIP is available from - ftp://ftp.hdfgroup.org/lib-external/szip/2.1. 2.3 Additional Utilities @@ -260,5 +259,7 @@ Build, Test and Install HDF5 on Cygwin with cygwin on Windows. ----------------------------------------------------------------------- +For further assistance, contact: -Need Further assistance, email help@hdfgroup.org + HDF Forum: https://forum.hdfgroup.org/ + HDF Helpdesk: https://portal.hdfgroup.org/display/support/The+HDF+Help+Desk diff --git a/release_docs/INSTALL_parallel b/release_docs/INSTALL_parallel index e4c540c..bf14b2e 100644 --- a/release_docs/INSTALL_parallel +++ b/release_docs/INSTALL_parallel @@ -1,6 +1,18 @@ Installation instructions for Parallel HDF5 ------------------------------------------- +0. Use Build Scripts +-------------------- +The HDF Group is accumulating build scripts to handle building parallel HDF5 +on various platforms (Cray, IBM, SGI, etc...). These scripts are being +maintained and updated continuously for current and future systems. The reader +is strongly encouraged to consult the repository at, + +https://github.com/HDFGroup/build_hdf5 + +for building parallel HDF5 on these system. All contributions, additions +and fixes to the repository are welcomed and encouraged. + 1. Overview ----------- @@ -28,9 +40,11 @@ and the parallel file system. 1.2. Further Help ----------------- -If you still have difficulties installing PHDF5 in your system, please send -mail to - help@hdfgroup.org + +For help with installing, questions can be posted to the HDF Forum or sent to the HDF Helpdesk: + + HDF Forum: https://forum.hdfgroup.org/ + HDF Helpdesk: https://portal.hdfgroup.org/display/support/The+HDF+Help+Desk In your mail, please include the output of "uname -a". If you have run the "configure" command, attach the output of the command and the content of @@ -75,12 +89,8 @@ The following steps are for building HDF5 for the Hopper compute nodes. They would probably work for other Cray systems but have not been verified. -Obtain a copy from the HDF ftp server: -http://www.hdfgroup.org/ftp/HDF5/current/src/ -(link might change, so always double check the HDF group website). - -$ wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-x.x.x.tar.gz -unpack the tarball +Obtain the HDF5 source code: + https://portal.hdfgroup.org/display/support/Downloads The entire build process should be done on a MOM node in an interactive allocation and on a file system accessible by all compute nodes. Request an interactive allocation with qsub: diff --git a/release_docs/USING_HDF5_CMake.txt b/release_docs/USING_HDF5_CMake.txt index e5be666..0261cae 100644 --- a/release_docs/USING_HDF5_CMake.txt +++ b/release_docs/USING_HDF5_CMake.txt @@ -216,7 +216,7 @@ adjust the forward slash to double backslashes, except for the HDF_DIR environment variable. NOTE: this file is available at the HDF web site: - http://www.hdfgroup.org/HDF5/release/cmakebuild.html + https://portal.hdfgroup.org/display/support/Building+HDF5+with+CMake HDF518_Examples.cmake HDF5_Examples_options.cmake @@ -313,7 +313,7 @@ endif() set(ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DHDF5_PACKAGE_NAME:STRING=@HDF5_PACKAGE@@HDF_PACKAGE_EXT@") ############################################################################################################### -# For any comments please contact cdashhelp@hdfgroup.org +# For any comments please contact help@hdfgroup.org # ############################################################################################################### diff --git a/release_docs/USING_HDF5_VS.txt b/release_docs/USING_HDF5_VS.txt index 41d6a99..b098f12 100644 --- a/release_docs/USING_HDF5_VS.txt +++ b/release_docs/USING_HDF5_VS.txt @@ -54,11 +54,11 @@ Using Visual Studio with HDF5 Libraries Many other common questions and hints are located online and being updated in the HDF5 FAQ. For Windows-specific questions, please see: - http://www.hdfgroup.org/HDF5/faq/windows.html + http://support.hdfgroup.org/HDF5/faq/windows.html For all other general questions, you can look in the general FAQ: - http://hdfgroup.org/HDF5-FAQ.html + http://support.hdfgroup.org/HDF5/HDF5-FAQ.html ************************************************************************ Please send email to help@hdfgroup.org for further assistance. -- cgit v0.12 From 0db9e890a84d428a4b62a51fa73480cb54f915f4 Mon Sep 17 00:00:00 2001 From: lrknox Date: Mon, 4 Jun 2018 14:41:28 -0500 Subject: Update INSTALL and INSTALL_parallel files to remove references to ancient systems and add generic steps for building HDF5 on HPC clusters. --- release_docs/INSTALL | 8 -------- release_docs/INSTALL_parallel | 25 ++++++++++++++----------- 2 files changed, 14 insertions(+), 19 deletions(-) diff --git a/release_docs/INSTALL b/release_docs/INSTALL index 124dba9..f84ddc2 100644 --- a/release_docs/INSTALL +++ b/release_docs/INSTALL @@ -14,7 +14,6 @@ CONTENTS 2. Quick installation 2.1. Windows and Cygwin - 2.2. RedStorm (Cray XT3) 3. HDF5 dependencies 3.1. Make @@ -97,13 +96,6 @@ CONTENTS Users of Microsoft Windows should see the INSTALL_Windows files for detailed instructions. INSTALL_Cygwin also exists for those platforms. -2.2. RedStorm (Cray XT3) - Users of the Red Storm machine, after reading this file, should read - the Red Storm section in the INSTALL_parallel file for specific - instructions for the Red Storm machine. The same instructions would - probably work for other Cray XT3 systems, but they have not been - verified. - 3. HDF5 dependencies 3.1. Make diff --git a/release_docs/INSTALL_parallel b/release_docs/INSTALL_parallel index bf14b2e..23dc2a0 100644 --- a/release_docs/INSTALL_parallel +++ b/release_docs/INSTALL_parallel @@ -82,19 +82,22 @@ This allows for >2GB sized files on Linux systems and is only available with Linux kernels 2.4 and greater. -2.3. Hopper (Cray XE6) (for v1.8 and later) +2.3. Unix HPC Clusters (for v1.8 and later) ------------------------- -The following steps are for building HDF5 for the Hopper compute -nodes. They would probably work for other Cray systems but have -not been verified. +The following steps are generic instructions for building HDF5 on +several current HPC systems. The exact commands and scripts to use +will vary according to the scheduling software on the individual +system. Consult the system documentation to determine the details. Obtain the HDF5 source code: https://portal.hdfgroup.org/display/support/Downloads -The entire build process should be done on a MOM node in an interactive allocation and on a file system accessible by all compute nodes. -Request an interactive allocation with qsub: -qsub -I -q debug -l mppwidth=8 +In general HDF5 can be built on a login/front-end node provided it is +installed on a file system accessible by all compute nodes. If parallel +tests run by "make check" or "make check-p" will be run on compute +nodes in a batch job, the HDF5 build directory should also exist on a +file system accessible by all compute nodes. - create a build directory build-hdf5: mkdir build-hdf5; cd build-hdf5/ @@ -102,12 +105,12 @@ qsub -I -q debug -l mppwidth=8 - configure HDF5: RUNSERIAL="aprun -q -n 1" RUNPARALLEL="aprun -q -n 6" FC=ftn CC=cc /path/to/source/configure --enable-fortran --enable-parallel --disable-shared - RUNSERIAL and RUNPARALLEL tells the library how it should launch programs that are part of the build procedure. + RUNSERIAL and RUNPARALLEL tells the library how it should launch programs that are part of the build procedure. Note that the command names and the specific options will vary according to the batch system. - Compile HDF5: gmake -- Check HDF5 +- Check HDF5: on most systems this should be run as a batch job on compute nodes. gmake check - Install HDF5 @@ -117,8 +120,8 @@ The build will be in build-hdf5/hdf5/ (or whatever you specify in --prefix). To compile other HDF5 applications use the wrappers created by the build (build-hdf5/hdf5/bin/h5pcc or h5fc) -3. Detail explanation ---------------------- +3. Detailed explanation +----------------------- 3.1. Installation steps (Uni/Multiple processes modes) ----------------------- -- cgit v0.12