diff options
Diffstat (limited to 'doc/html/Performance.html')
| -rw-r--r-- | doc/html/Performance.html | 260 |
1 files changed, 0 insertions, 260 deletions
diff --git a/doc/html/Performance.html b/doc/html/Performance.html deleted file mode 100644 index 25db8f4..0000000 --- a/doc/html/Performance.html +++ /dev/null @@ -1,260 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> -<html> - <head> - <title>Performance</title> - -<!-- #BeginLibraryItem "/ed_libs/styles_UG.lbi" --> -<!-- - * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * - * Copyright by the Board of Trustees of the University of Illinois. * - * All rights reserved. * - * * - * This file is part of HDF5. The full HDF5 copyright notice, including * - * terms governing use, modification, and redistribution, is contained in * - * the files COPYING and Copyright.html. COPYING can be found at the root * - * of the source code distribution tree; Copyright.html can be found at the * - * root level of an installed copy of the electronic HDF5 document set and * - * is linked from the top-level documents page. It can also be found at * - * http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have * - * access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. * - * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * - --> - -<link href="ed_styles/UGelect.css" rel="stylesheet" type="text/css"> -<!-- #EndLibraryItem --></head> - - <body bgcolor="#FFFFFF"> - - -<!-- #BeginLibraryItem "/ed_libs/NavBar_UG.lbi" --><hr> -<center> -<table border=0 width=98%> -<tr><td valign=top align=left> - <a href="index.html">HDF5 documents and links</a> <br> - <a href="H5.intro.html">Introduction to HDF5</a> <br> - <a href="RM_H5Front.html">HDF5 Reference Manual</a> <br> - <a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide for Release 1.6</a> <br> - <!-- - <a href="Glossary.html">Glossary</a><br> - --> -</td> -<td valign=top align=right> - And in this document, the - <a href="H5.user.html"><strong>HDF5 User's Guide from Release 1.4.5:</strong></a> - <br> - <a href="Files.html">Files</a> - <a href="Datasets.html">Datasets</a> - <a href="Datatypes.html">Datatypes</a> - <a href="Dataspaces.html">Dataspaces</a> - <a href="Groups.html">Groups</a> - <br> - <a href="References.html">References</a> - <a href="Attributes.html">Attributes</a> - <a href="Properties.html">Property Lists</a> - <a href="Errors.html">Error Handling</a> - <br> - <a href="Filters.html">Filters</a> - <a href="Caching.html">Caching</a> - <a href="Chunking.html">Chunking</a> - <a href="MountingFiles.html">Mounting Files</a> - <br> - <a href="Performance.html">Performance</a> - <a href="Debugging.html">Debugging</a> - <a href="Environment.html">Environment</a> - <a href="ddl.html">DDL</a> -</td></tr> -</table> -</center> -<hr> -<!-- #EndLibraryItem --><h1>Performance Analysis and Issues</h1> - - <h2>1. Introduction</h2> - - <p>This section includes brief discussions of performance issues - in HDF5 and performance analysis tools for HDF5 or pointers to - such discussions. - - <h2>2. Dataset Chunking</h2> - - Appropriate dataset chunking can make a siginificant difference - in HDF5 performance. This topic is discussed in - <a href="Chunking.html">Dataset Chunking Issues</a> elsewhere - in this <cite>User's Guide</cite>. - - <a name="Freespace"> - <h2>3. Freespace Management</h2> - </a> - - <p>HDF5 does not yet manage freespace as effectively as it might. - While a file is opened, the library actively tracks and re-uses - <em>freespace</em>, i.e., space that is freed (or released) - during the run. - But the library does not yet manage freespace across the - closing and reopening of a file; when a file is closed, - all knowledge of available freespace is lost. - What was freespace becomes an unusable <em>hole</em> in the file. - - <p>There are several circumstances that can result in freespace - in an HDF5 file: - <ul> - <li>Reading then rewriting a dataset or compressed dataset - chunk.<sup><a href="#footcchunk">1</a></sup> - <ul> - <li>If the rewritten dataset or compressed chunk is the same - size as or smaller than the original, it will be written - to the same file location. - <li>If, however, the dataset or compressed chunk is larger - than the original, it will be written contiguously elsewhere - in the file, leaving freespace at the original location. - <li>If the rewritten dataset or compressed chunk is - substantially smaller than the original, the remaining - space will be released and identified as freespace. - </ul> - <li>Deleting (or unlinking) a dataset or group. - <ul> - <li>If an object, such as a dataset, group, or named datatype, - is deleted (normally with <code>H5Gunlink</code>), - the space previously occupied by the object is released - and identified as freespace. - </ul> - </ul> - - <p>As stated above, freespace is not managed across the - closing and reopening of an HDF5 file; file space that was - known freespace while the file remained open becomes an - inaccessible hole when the file is closed. - Thus, if a file is often closed and reopened, datasets - frequently rewritten, or groups and/or datasets frequently - added and deleted, that file can develop large numbers of - holes and grow unnecessarily large. This can, in turn, - seriously impair application or library performance - as the file ages. - - <p>An <code>h5pack</code> utility would enable <em>packing</em> - a file to remove the holes, but writing such a utility to - universally pack the file correctly is a complex task and the - HDF5 development team has not to date had the resources to - complete the task. - - <p>For application developers or researchers who find themselves - working with files that become bloated in this manner, there - are, at this time, two remedies: - <ul> - <li><code>H5view</code>, an HDF5 Java tool, allows the user - to open a file and, using the <code>Save As...</code> feature, - save the file under a new filename. The new file can then - be closed and will be a packed version of the original file. - This approach is reasonably reliable, but with two caveats: - <ul> - <li>It is not automated. - <li>This ability is a side-effect of the tool's design; - it was not designed for this purpose and this approach - to file packing has not been exhaustively tested. - </ul> - <li>An application developer or researcher can write a utility - that is tuned to their data and file structures. This - untility can then read in a file, copy the structures and - datasets to a new file, and write the new file to storage. - This will eliminate the holes, making the new file a - fully-packed version of the original file. - </ul> - - <a name="footcchunk"> - <p></a> - <sup>1</sup> - <font size=-1> - This is a problem only with compressed chunks. - The compression ratio of data is highly dependent on the data - itself; regardless of whether the <em>size</em> of the data - changes, the size of the compressed data change substantially - as the data changes. Uncompressed chunks do not vary in size, - so this issue does not arise. - </font> - - <h2>4. Use of the Pablo Instrumentation of HDF5</h2> - - Pablo HDF5 Trace software provides a means of measuring the - performance of programs using HDF5. - - <p>The Pablo software consists - of an instrumented copy of the HDF5 library, the Pablo Trace and - Trace Extensions libraries, and some utilities for processing the - output. The instrumented version of the HDF5 library has hooks - inserted into the HDF5 code which call routines in the Pablo Trace - library just after entry to each instrumented HDF5 routine and - just prior to exit from the routine. The Pablo Trace Extension - library has programs that track the I/O activity between the - entry and exit of the HDF5 routine during execution. - - <p>A few lines of code must be inserted in the user's main program - to enable tracing and to specify which HDF5 procedures are to be - traced. The program is linked with the special HDF5 and Pablo - libraries to produce an executable. Running this executable on - a single processor produces an output file called the trace file - which contains records, called Pablo Self-Defining Data Format - (SDDF) records, which can later be analyzed using the - HDF5 Analysis Utilities. The HDF5 Analysis Utilites can be used - to interpret the SDDF records in the trace files to produce a - report describing the HDF5 IO activity that occurred during - execution. - - <p>For further instructions, see the file <code>READ_ME</code> - in the <code> $(toplevel)/hdf5/pablo/ </code> subdirectory of - the HDF5 source code distribution. - - <p>For further information about Pablo and the - Self-Defining Data Format, visit the Pablo website at - <code><a href="http://www-pablo.cs.uiuc.edu/">http://www-pablo.cs.uiuc.edu/</a></code>.</p> - - -<!-- #BeginLibraryItem "/ed_libs/NavBar_UG.lbi" --><hr> -<center> -<table border=0 width=98%> -<tr><td valign=top align=left> - <a href="index.html">HDF5 documents and links</a> <br> - <a href="H5.intro.html">Introduction to HDF5</a> <br> - <a href="RM_H5Front.html">HDF5 Reference Manual</a> <br> - <a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide for Release 1.6</a> <br> - <!-- - <a href="Glossary.html">Glossary</a><br> - --> -</td> -<td valign=top align=right> - And in this document, the - <a href="H5.user.html"><strong>HDF5 User's Guide from Release 1.4.5:</strong></a> - <br> - <a href="Files.html">Files</a> - <a href="Datasets.html">Datasets</a> - <a href="Datatypes.html">Datatypes</a> - <a href="Dataspaces.html">Dataspaces</a> - <a href="Groups.html">Groups</a> - <br> - <a href="References.html">References</a> - <a href="Attributes.html">Attributes</a> - <a href="Properties.html">Property Lists</a> - <a href="Errors.html">Error Handling</a> - <br> - <a href="Filters.html">Filters</a> - <a href="Caching.html">Caching</a> - <a href="Chunking.html">Chunking</a> - <a href="MountingFiles.html">Mounting Files</a> - <br> - <a href="Performance.html">Performance</a> - <a href="Debugging.html">Debugging</a> - <a href="Environment.html">Environment</a> - <a href="ddl.html">DDL</a> -</td></tr> -</table> -</center> -<hr> -<!-- #EndLibraryItem --><!-- #BeginLibraryItem "/ed_libs/Footer.lbi" --><address> -<a href="mailto:hdfhelp@ncsa.uiuc.edu">HDF Help Desk</a> -<br> -Describes HDF5 Release 1.4.5, February 2003 -</address><!-- #EndLibraryItem --><!-- Created: Thu Oct 14 16:46:00 CDT 1999 --> -<!-- hhmts start --> -Last modified: 2 August 2001 -<!-- hhmts end --> - -</body> -</html> |