From 365dac33e385affcb57a6b8a5cf53f8d03ac2510 Mon Sep 17 00:00:00 2001 From: Robb Matzke Date: Mon, 20 Jul 1998 08:45:25 -0500 Subject: [svn-r515] Changes since 19980715 ---------------------- ./doc/html/H5.format.html ./src/H5Gent.c ./src/H5Gprivate.h ./src/H5Oattr.c ./src/H5Oprivate.h ./src/H5Oshared.c ./src/H5HG.c ./src/H5HGprivate.h Added padding fields in symbol table entries, attribute messages, shared messages, and global heap objects to insure that things are aligned on 8-byte boundaries in the file, and thus in memory. Otherwise some little endian machines complain (DEC Alpha) during encoding/decoding of file meta data. I chose to add alignment to the file rather than rewriting the ENCODE/DECODE macros for the little endian case. Completely rewrote the section on attribute messages. More alignment stuff will follow. ./src/H5detect.c Fixed a typo `nd'->`dn' ./test/dtypes.c Commented out conversion tests to/from `long double' on machines where it's the same size as `double' to get rid of compiler warnings. ./doc/html/Big.html Fixed a couple typos. --- README | 2 +- doc/html/Big.html | 49 +++--- doc/html/H5.format.html | 402 +++++++++++++++++++----------------------------- src/H5Gent.c | 5 + src/H5Gprivate.h | 3 +- src/H5HG.c | 28 ++-- src/H5HGprivate.h | 15 +- src/H5Oattr.c | 334 +++++++++++----------------------------- src/H5Oprivate.h | 2 +- src/H5Oshared.c | 22 ++- src/H5detect.c | 2 +- test/.distdep | 68 ++++---- test/dtypes.c | 34 +++- 13 files changed, 402 insertions(+), 564 deletions(-) diff --git a/README b/README index 1983095..9b4cfd7 100644 --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is hdf5-1.0.24a released on 1998-06-17 14:14 UTC +This is hdf5-1.0.24a released on 1998-06-19 14:51 UTC Please refer to the INSTALL file for installation instructions. ------------------------------------------------------------------------------ diff --git a/doc/html/Big.html b/doc/html/Big.html index 080f786..fe00ff8 100644 --- a/doc/html/Big.html +++ b/doc/html/Big.html @@ -24,10 +24,18 @@

2. File Size Limits

-

Some 32-bit operating systems have special file systems that - can support large (>2GB) files and HDF5 will detect these and - use them automatically. If this is the case, the output from - configure will show: +

Systems that have 64-bit file addresses will be able to access + those files automatically. One should see the following output + from configure: + +

+checking size of off_t... 8
+    
+ +

Also, some 32-bit operating systems have special file systems + that can support large (>2GB) files and HDF5 will detect + these and use them automatically. If this is the case, the + output from configure will show:

 checking for lseek64... yes
@@ -42,25 +50,28 @@ checking for fseek64... yes
     

 hid_t plist, file;
 plist = H5Pcreate (H5P_FILE_ACCESS);
-H5Pset_family (plist, 1<<30, H5P_DEFAULT);
+H5Pset_family (plist, 1<<30, H5P_DEFAULT);
 file = H5Fcreate ("big%03d.h5", H5F_ACC_TRUNC, H5P_DEFAULT, plist);
     
-

The second argument (30) to +

The second argument (1<<30) to H5Pset_family() indicates that the family members - are to be 2^30 bytes (1GB) each. In general, family members - cannot be 2GB because writes to byte number 2,147,483,647 will - fail, so the largest safe value for a family member is - 2,147,483,647. HDF5 will create family members on demand as the - HDF5 address space increases, but since most Unix systems limit - the number of concurrently open files the effective maximum size - of the HDF5 address space will be limited. + are to be 2^30 bytes (1GB) each although we could have used any + reasonably large value. In general, family members cannot be + 2GB because writes to byte number 2,147,483,647 will fail, so + the largest safe value for a family member is 2,147,483,647. + HDF5 will create family members on demand as the HDF5 address + space increases, but since most Unix systems limit the number of + concurrently open files the effective maximum size of the HDF5 + address space will be limited (the system on which this was + developed allows 1024 open files, so if each family member is + approx 2GB then the largest HDF5 file is approx 2TB).

If the effective HDF5 address space is limited then one may be able to store datasets as external datasets each spanning multiple files of any length since HDF5 opens external dataset - files one at a time. To arrange storage for a 5TB dataset one - could say: + files one at a time. To arrange storage for a 5TB dataset split + among 1GB files one could say:

 hid_t plist = H5Pcreate (H5P_DATASET_CREATE);
@@ -73,9 +84,9 @@ for (i=0; i<5*1024; i++) {
     

3. Dataset Size Limits

The second limit which must be overcome is that of - sizeof(size_t). HDF5 defines a new data type - called hsize_t which is used for sizes of datasets - and is, by default, defined as unsigned long long. + sizeof(size_t). HDF5 defines a data type called + hsize_t which is used for sizes of datasets and is, + by default, defined as unsigned long long.

To create a dataset with 8*2^30 4-byte integers for a total of 32GB one first creates the dataspace. We give two examples @@ -105,7 +116,7 @@ hid_t space2 = H5Screate_simple (1, size2, size2};

Robb Matzke
-Last modified: Wed May 13 12:36:47 EDT 1998 +Last modified: Sun Jul 19 11:37:25 EDT 1998 diff --git a/doc/html/H5.format.html b/doc/html/H5.format.html index a3c9a7c..0b4da0c 100644 --- a/doc/html/H5.format.html +++ b/doc/html/H5.format.html @@ -56,8 +56,8 @@ Name: Reserved - not assigned yet
  • Name: Data Storage - Compressed -
  • - Name: Attribute List +
  • + Name: Attribute
  • Name: Object Name
  • @@ -179,9 +179,7 @@ -
    - Symbol-Table Entry of the "Root Object" -

    +
    Root Group Symbol Table Entry

    @@ -392,7 +390,7 @@ - Symbol-Table Entry of the Root Object + Root Group Symbol Table Entry This symbol-table entry (described later in this document) refers to the entry point into the group graph. If the file contains a single object, then that @@ -688,18 +686,27 @@ byte byte byte + Name Offset (<size> bytes) + Object Header Address + Symbol-Type + + + + Reserved +

    Scratch-pad Space (24 bytes)


    + @@ -759,6 +766,13 @@ + Reserved + These for bytes are present so that the scratch pad + space is aligned on an eight-byte boundary. They are + always set to zero. + + + Scratch-Pad Space This space is used for different purposes, depending on the value of the Symbol Type field. Any meta-data @@ -1101,7 +1115,11 @@ Object ID Reference Count - + + + Reserved + + Object Total Size @@ -1139,6 +1157,12 @@ + Reserved + Zero padding to align next field on an 8-byte + boundary. + + + Object Total Size This is the total size in bytes of the object. It includes all fields listed in this table. @@ -1283,7 +1307,7 @@ Version # of Object Header - Alignment of Object Header Messages + Reserved Number of Header Messages @@ -1301,7 +1325,7 @@ Reserved - Header Message Data #1 (variable size) +
    Header Message Data #1

    .
    .
    .
    @@ -1315,7 +1339,7 @@ Reserved - Header Message Data #n (variable) +
    Header Message Data #n

    @@ -1338,11 +1362,8 @@ - Alignment of object header messages - This value is used to determine the byte-alignment of - messagesin the object header. Typically set to 4, which - aligns new messages on a 4-byte boundary in the object - header. + Reserved + Always set to zero. @@ -1384,7 +1405,9 @@ Size of Header Message Data This value specifies the number of bytes of header message data following the header message type and length - information for the current message. + information for the current message. The size includes + padding bytes to make the message a multiple of eight + bytes. @@ -1410,7 +1433,8 @@ header message type and size respectively. Some header message types do not require any data and this information can be eliminated by setting the length of the message to - zero. + zero. The data is padded with enough zeros to make the + size a multiple of eight. @@ -1439,7 +1463,7 @@
    -

    Name: Simple Data Space/a>

    +

    Name: Simple Data Space

    Type: 0x0001
    Length: varies
    @@ -1517,7 +1541,7 @@ - Dimension Size #n (<size&rt; bytes) + Dimension Size #n (<size> bytes) This value is the current size of the dimension of the data as stored in the file. The first dimension stored in the list of dimensions is the slowest changing dimension @@ -1526,14 +1550,14 @@ - Dimension Maximum #n (<size&rt; bytes) + Dimension Maximum #n (<size> bytes) This value is the maximum size of the dimension of the data as stored in the file. This value may be the special - value <UNLIMITED> (0xffffffff) which indicates that - the data may expand along this dimension indefinitely. If - these values are not stored, the maximum value of each - dimension is assumed to be the same as the current size - value. + value <UNLIMITED> (all bits set) which indicates + that the data may expand along this dimension + indefinitely. If these values are not stored, the maximum + value of each dimension is assumed to be the same as the + current size value. @@ -2072,7 +2096,7 @@

    - @@ -2105,7 +2129,7 @@

    + Bit Field for Compound Types (Class 6)
    - @@ -2643,121 +2667,29 @@ library's discretion, chunks which fail the compression can also be stored in their raw format. -
    -

    Name: Attribute List

    +

    Name: Attribute

    Type: 0x000C
    Length: varies
    Status: Optional, may be repeated.
    -

    Purpose and Description: The Attribute List +

    Purpose and Description: The Attribute message is used to list objects in the HDF file which are used - as attributes, or "meta-data" about the current object. Other - objects can be used as attributes for either the entire object - or portions of the current object. The attribute list is - composed of two lists of objects, the first being simple - attributes about the entire dataset, and the second being - pointers attribute objects about the entire dataset. Partial - dataset pointers are currently unspecified and - unimplemented. + as attributes, or "meta-data" about the current object. An + attribute is a small dataset; it has a name, a data type, a data + space, and raw data. Since attributes are stored in the object + header they must be relatively small (<64kb) and can be + associated with any type of object which has an object header + (groups, datasets, named types and spaces, etc.).

    Format of Data:

    -
    + Properties for Compound Types (Class 6)
    - - + + + + + + @@ -2980,13 +2890,19 @@ - + @@ -3177,7 +3093,7 @@ data-type.
    Quincey Koziol
    Robb Matzke
    -Last modified: Mon Jun 1 21:44:38 EDT 1998 +Last modified: Mon Jul 20 09:16:11 EDT 1998 diff --git a/src/H5Gent.c b/src/H5Gent.c index 93fb1d7..1c0e981 100644 --- a/src/H5Gent.c +++ b/src/H5Gent.c @@ -142,6 +142,8 @@ H5G_ent_decode_vec(H5F_t *f, const uint8 **pp, H5G_entry_t *ent, intn n) * Jul 18 1997 * * Modifications: + * Robb Matzke, 17 Jul 1998 + * Added a 4-byte padding field for alignment and future expansion. * *------------------------------------------------------------------------- */ @@ -164,6 +166,7 @@ H5G_ent_decode(H5F_t *f, const uint8 **pp, H5G_entry_t *ent) H5F_decode_length(f, *pp, ent->name_off); H5F_addr_decode(f, pp, &(ent->header)); UINT32DECODE(*pp, tmp); + *pp += 4; /*reserved*/ ent->type=(H5G_type_t)tmp; /* decode scratch-pad */ @@ -275,6 +278,7 @@ H5G_ent_encode(H5F_t *f, uint8 **pp, const H5G_entry_t *ent) H5F_encode_length(f, *pp, ent->name_off); H5F_addr_encode(f, pp, &(ent->header)); UINT32ENCODE(*pp, ent->type); + UINT32ENCODE(*pp, 0); /*reserved*/ /* encode scratch-pad */ switch (ent->type) { @@ -300,6 +304,7 @@ H5G_ent_encode(H5F_t *f, uint8 **pp, const H5G_entry_t *ent) H5F_addr_undef(&undef); H5F_addr_encode(f, pp, &undef); UINT32ENCODE(*pp, H5G_NOTHING_CACHED); + UINT32ENCODE(*pp, 0); /*reserved*/ } /* fill with zero */ diff --git a/src/H5Gprivate.h b/src/H5Gprivate.h index ae3064a..03f0348 100644 --- a/src/H5Gprivate.h +++ b/src/H5Gprivate.h @@ -43,7 +43,8 @@ (H5F_SIZEOF_SIZE(F) + /*offset of name into heap */ \ H5F_SIZEOF_ADDR(F) + /*address of object header */ \ 4 + /*entry type */ \ - H5G_SIZEOF_SCRATCH) /*scratch pad space */ + 4 + /*reserved */ \ + H5G_SIZEOF_SCRATCH) /*scratch pad space */ /* * Various types of object header information can be cached in a symbol diff --git a/src/H5HG.c b/src/H5HG.c index eaca1f9..bc8c936 100644 --- a/src/H5HG.c +++ b/src/H5HG.c @@ -102,6 +102,7 @@ H5HG_create (H5F_t *f, size_t size) /* Check args */ assert (f); if (sizeobj[0].size = size - H5HG_SIZEOF_HDR (f); heap->obj[0].begin = p; - UINT16ENCODE (p, 0); /*object ID*/ - UINT16ENCODE (p, 0); /*reference count*/ + UINT16ENCODE(p, 0); /*object ID*/ + UINT16ENCODE(p, 0); /*reference count*/ + UINT32ENCODE(p, 0); /*reserved*/ H5F_encode_length (f, p, heap->obj[0].size); HDmemset (p, 0, (size_t)((heap->chunk+heap->nalloc) - p)); @@ -289,6 +291,7 @@ H5HG_load (H5F_t *f, const haddr_t *addr, const void __unused__ *udata1, assert (idxnalloc); assert (NULL==heap->obj[idx].begin); UINT16DECODE (p, heap->obj[idx].nrefs); + p += 4; /*reserved*/ H5F_decode_length (f, p, heap->obj[idx].size); heap->obj[idx].begin = begin; p = begin + heap->obj[idx].size; @@ -440,8 +443,9 @@ H5HG_alloc (H5F_t *f, H5HG_heap_t *heap, int cwfsno, size_t size) heap->obj[idx].size = size; heap->obj[idx].begin = heap->obj[0].begin; p = heap->obj[idx].begin; - UINT16ENCODE (p, idx); - UINT16ENCODE (p, 0); /*nrefs*/ + UINT16ENCODE(p, idx); + UINT16ENCODE(p, 0); /*nrefs*/ + UINT32ENCODE(p, 0); /*reserved*/ H5F_encode_length (f, p, size); /* Fix the free space object */ @@ -466,8 +470,9 @@ H5HG_alloc (H5F_t *f, H5HG_heap_t *heap, int cwfsno, size_t size) heap->obj[0].size -= size; heap->obj[0].begin += size; p = heap->obj[0].begin; - UINT16ENCODE (p, 0); /*id*/ - UINT16ENCODE (p, 0); /*nrefs*/ + UINT16ENCODE(p, 0); /*id*/ + UINT16ENCODE(p, 0); /*nrefs*/ + UINT32ENCODE(p, 0); /*reserved*/ H5F_encode_length (f, p, heap->obj[0].size); } else { @@ -528,7 +533,7 @@ H5HG_insert (H5F_t *f, size_t size, void *obj, H5HG_t *hobj/*out*/) } /* Find a large enough collection on the CWFS list */ - need = size + H5HG_SIZEOF_OBJHDR (f); + need = H5HG_ALIGN(size + H5HG_SIZEOF_OBJHDR(f)); for (cwfsno=0; cwfsnoshared->ncwfs; cwfsno++) { if (f->shared->cwfs[cwfsno]->obj[0].size>=need) { /* @@ -565,7 +570,9 @@ H5HG_insert (H5F_t *f, size_t size, void *obj, H5HG_t *hobj/*out*/) assert (idx>0); /* Copy data into the heap */ - HDmemcpy (heap->obj[idx].begin+H5HG_SIZEOF_OBJHDR(f), obj, size); + HDmemcpy(heap->obj[idx].begin+H5HG_SIZEOF_OBJHDR(f), obj, size); + HDmemset(heap->obj[idx].begin+H5HG_SIZEOF_OBJHDR(f)+size, 0, + need-(H5HG_SIZEOF_OBJHDR(f)+size)); heap->dirty = TRUE; /* Return value */ @@ -820,8 +827,9 @@ H5HG_remove (H5F_t *f, H5HG_t *hobj) heap->size-((obj_start+size)-heap->chunk)); if (heap->obj[0].size>=H5HG_SIZEOF_OBJHDR (f)) { p = heap->obj[0].begin; - UINT32ENCODE (p, 0); /*id*/ - UINT32ENCODE (p, 0); /*nrefs*/ + UINT16ENCODE(p, 0); /*id*/ + UINT16ENCODE(p, 0); /*nrefs*/ + UINT32ENCODE(p, 0); /*reserved*/ H5F_encode_length (f, p, size); } HDmemset (heap->obj+hobj->idx, 0, sizeof(H5HG_obj_t)); diff --git a/src/H5HGprivate.h b/src/H5HGprivate.h index c6ec1cd..8f66ef7 100644 --- a/src/H5HGprivate.h +++ b/src/H5HGprivate.h @@ -24,6 +24,16 @@ #define H5HG_VERSION 1 /* + * Pad all global heap messages to a multiple of eight bytes so we can load + * the entire collection into memory and operate on it there. Eight should + * be sufficient for machines that have alignment constraints because our + * largest data type is eight bytes. + */ +#define H5HG_ALIGNMENT 8 +#define H5HG_ALIGN(X) (H5HG_ALIGNMENT*(((X)+H5HG_ALIGNMENT-1)/ \ + H5HG_ALIGNMENT)) + +/* * All global heap collections are at least this big. This allows us to read * most collections with a single read() since we don't have to read a few * bytes of header to figure out the size. If the heap is larger than this @@ -53,8 +63,9 @@ /* * The overhead associated with each object in the heap. */ -#define H5HG_SIZEOF_OBJHDR(f) (2 + /*reference count */ \ - 2 + /*reserved */ \ +#define H5HG_SIZEOF_OBJHDR(f) (2 + /*object id number */ \ + 2 + /*reference count */ \ + 4 + /*reserved */ \ H5F_SIZEOF_SIZE(f)) /*object data size */ /* diff --git a/src/H5Oattr.c b/src/H5Oattr.c index 5be08b9..6021802 100644 --- a/src/H5Oattr.c +++ b/src/H5Oattr.c @@ -72,6 +72,10 @@ static hbool_t interface_initialize_g = FALSE; This function decodes the "raw" disk form of a attribute message into a struct in memory native format. The struct is allocated within this function using malloc() and is returned to the caller. + * + * Modifications: + * Robb Matzke, 17 Jul 1998 + * Added padding for alignment. --------------------------------------------------------------------------*/ static void * H5O_attr_decode(H5F_t *f, const uint8 *p, H5O_shared_t __unused__ *sh) @@ -90,24 +94,30 @@ H5O_attr_decode(H5F_t *f, const uint8 *p, H5O_shared_t __unused__ *sh) HRETURN_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed"); } + + /* + * Decode the sizes of the parts of the attribute. The sizes stored in + * the file are exact but the parts are aligned on 8-byte boundaries. + */ + UINT16DECODE(p, name_len); /*including null*/ + UINT16DECODE(p, attr->dt_size); + UINT16DECODE(p, attr->ds_size); + p += 2; /*reserved*/ /* Decode and store the name */ - UINT16DECODE(p, name_len); - if (NULL==(attr->name=H5MM_malloc(name_len+1))) { + if (NULL==(attr->name=H5MM_malloc(name_len))) { HRETURN_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed"); } HDmemcpy(attr->name,p,name_len); - attr->name[name_len]='\0'; - p+=name_len; /* advance the memory pointer */ + p += H5O_ALIGN(name_len); /* advance the memory pointer */ /* decode the attribute datatype */ if((attr->dt=(H5O_DTYPE->decode)(f,p,NULL))==NULL) { HRETURN_ERROR(H5E_ATTR, H5E_CANTDECODE, NULL, "can't decode attribute datatype"); } - attr->dt_size=(H5O_DTYPE->raw_size)(f,attr->dt); - p+=attr->dt_size; + p += H5O_ALIGN(attr->dt_size); /* decode the attribute dataspace */ if (NULL==(attr->ds = H5MM_calloc(sizeof(H5S_t)))) { @@ -120,9 +130,8 @@ H5O_attr_decode(H5F_t *f, const uint8 *p, H5O_shared_t __unused__ *sh) H5MM_xfree(simple); } else { attr->ds->extent.type = H5S_SCALAR; - } /* end else */ - attr->ds_size=(H5O_SDSPACE->raw_size)(f,&(attr->ds->extent.u.simple)); - p+=attr->ds_size; + } + p += H5O_ALIGN(attr->ds_size); /* Compute the size of the data */ attr->data_size=H5S_extent_npoints(attr->ds)*H5T_get_size(attr->dt); @@ -162,6 +171,10 @@ H5O_attr_decode(H5F_t *f, const uint8 *p, H5O_shared_t __unused__ *sh) DESCRIPTION This function encodes the native memory form of the attribute message in the "raw" disk form. + * + * Modifications: + * Robb Matzke, 17 Jul 1998 + * Added padding for alignment. --------------------------------------------------------------------------*/ static herr_t H5O_attr_encode(H5F_t *f, uint8 *p, const void *mesg) @@ -176,25 +189,40 @@ H5O_attr_encode(H5F_t *f, uint8 *p, const void *mesg) assert(p); assert(attr); - /* encode the attribute name */ - name_len=HDstrlen(attr->name); + /* + * Encode the lengths of the various parts of the attribute message. The + * encoded lengths are exact but we pad each part except the data to be a + * multiple of eight bytes. + */ + name_len = HDstrlen(attr->name)+1; UINT16ENCODE(p, name_len); - HDmemcpy(p,attr->name,name_len); - p+=name_len; + UINT16ENCODE(p, attr->dt_size); + UINT16ENCODE(p, attr->ds_size); + UINT16ENCODE(p, 0); /*reserved*/ + + /* + * Write the name including null terminator padded to the correct number + * of bytes. + */ + HDmemcpy(p, attr->name, name_len); + HDmemset(p+name_len, 0, H5O_ALIGN(name_len)-name_len); + p += H5O_ALIGN(name_len); /* encode the attribute datatype */ if((H5O_DTYPE->encode)(f,p,attr->dt)<0) { HRETURN_ERROR(H5E_ATTR, H5E_CANTENCODE, FAIL, "can't encode attribute datatype"); } - p+=attr->dt_size; + HDmemset(p+attr->dt_size, 0, H5O_ALIGN(attr->dt_size)-attr->dt_size); + p += H5O_ALIGN(attr->dt_size); /* encode the attribute dataspace */ if((H5O_SDSPACE->encode)(f,p,&(attr->ds->extent.u.simple))<0) { HRETURN_ERROR(H5E_ATTR, H5E_CANTENCODE, FAIL, "can't encode attribute dataspace"); } - p+=attr->ds_size; + HDmemset(p+attr->ds_size, 0, H5O_ALIGN(attr->ds_size)-attr->ds_size); + p += H5O_ALIGN(attr->ds_size); /* Store attribute data */ HDmemcpy(p,attr->data,attr->data_size); @@ -256,32 +284,36 @@ H5O_attr_copy(const void *_src, void *_dst) This function returns the size of the raw attribute message on success. (Not counting the message type or size fields, only the data portion of the message). It doesn't take into account alignment. + * + * Modified: + * Robb Matzke, 17 Jul 1998 + * Added padding between message parts for alignment. --------------------------------------------------------------------------*/ static size_t H5O_attr_size(H5F_t __unused__ *f, const void *mesg) { - size_t ret_value = 0; - const H5A_t *attr = (const H5A_t *) mesg; + size_t ret_value = 0; + size_t name_len; + const H5A_t *attr = (const H5A_t *) mesg; FUNC_ENTER(H5O_attr_size, 0); assert(attr); - /* Get size of name */ - ret_value=2; /* Size to store length of name */ - ret_value+=HDstrlen(attr->name); /* Add length of name (non-zero terminated) */ + name_len = HDstrlen(attr->name)+1; - /* Get size of datatype information */ - ret_value+=attr->dt_size; - - /* Get size of [simple] dataspace information */ - ret_value+=attr->ds_size; - - /* Get size of attribute data */ - ret_value+=attr->data_size; + ret_value = 2 + /*name size inc. null */ + 2 + /*type size */ + 2 + /*space size */ + 2 + /*reserved */ + H5O_ALIGN(name_len) + /*attribute name */ + H5O_ALIGN(attr->dt_size) + /*data type */ + H5O_ALIGN(attr->ds_size) + /*data space */ + attr->data_size; /*the data itself */ FUNC_LEAVE(ret_value); } + /*------------------------------------------------------------------------- * Function: H5O_attr_reset @@ -339,15 +371,10 @@ H5O_attr_reset(void *_mesg) parameter. --------------------------------------------------------------------------*/ static herr_t -H5O_attr_debug(H5F_t __unused__ *f, const void __unused__ *mesg, - FILE __unused__ * stream, intn __unused__ indent, - intn __unused__ fwidth) +H5O_attr_debug(H5F_t *f, const void *_mesg, FILE * stream, intn indent, + intn fwidth) { -#ifdef LATER - const char *s; - char buf[256]; - intn i, j; -#endif /* LATER */ + const H5A_t *mesg = (const H5A_t *)_mesg; FUNC_ENTER(H5O_attr_debug, FAIL); @@ -357,218 +384,29 @@ H5O_attr_debug(H5F_t __unused__ *f, const void __unused__ *mesg, assert(indent >= 0); assert(fwidth >= 0); -#ifdef LATER - switch (dt->type) { - case H5T_INTEGER: - s = "integer"; - break; - case H5T_FLOAT: - s = "floating-point"; - break; - case H5T_TIME: - s = "date and time"; - break; - case H5T_STRING: - s = "text string"; - break; - case H5T_BITFIELD: - s = "bit field"; - break; - case H5T_OPAQUE: - s = "opaque"; - break; - case H5T_COMPOUND: - s = "compound"; - break; - default: - sprintf(buf, "H5T_CLASS_%d", (int) (dt->type)); - s = buf; - break; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "Type class:", - s); - - fprintf(stream, "%*s%-*s %lu byte%s\n", indent, "", fwidth, - "Size:", - (unsigned long) (dt->size), 1 == dt->size ? "" : "s"); - - if (H5T_COMPOUND == dt->type) { - fprintf(stream, "%*s%-*s %d\n", indent, "", fwidth, - "Number of members:", - dt->u.compnd.nmembs); - for (i = 0; i < dt->u.compnd.nmembs; i++) { - sprintf(buf, "Member %d:", i); - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - buf, - dt->u.compnd.memb[i].name); - fprintf(stream, "%*s%-*s %lu\n", indent + 3, "", MAX(0, fwidth-3), - "Byte offset:", - (unsigned long) (dt->u.compnd.memb[i].offset)); - fprintf(stream, "%*s%-*s %d%s\n", indent + 3, "", MAX(0, fwidth-3), - "Dimensionality:", - dt->u.compnd.memb[i].ndims, - 0 == dt->u.compnd.memb[i].ndims ? " (scalar)" : ""); - if (dt->u.compnd.memb[i].ndims > 0) { - fprintf(stream, "%*s%-*s {", indent + 3, "", MAX(0, fwidth-3), - "Size:"); - for (j = 0; j < dt->u.compnd.memb[i].ndims; j++) { - fprintf(stream, "%s%lu", j ? ", " : "", - (unsigned long) (dt->u.compnd.memb[i].dim[j])); - } - fprintf(stream, "}\n"); - fprintf(stream, "%*s%-*s {", indent + 3, "", MAX(0, fwidth-3), - "Permutation:"); - for (j = 0; j < dt->u.compnd.memb[i].ndims; j++) { - fprintf(stream, "%s%lu", j ? ", " : "", - (unsigned long) (dt->u.compnd.memb[i].perm[j])); - } - fprintf(stream, "}\n"); - } - H5O_dtype_debug(f, dt->u.compnd.memb[i].type, stream, - indent + 3, MAX(0, fwidth - 3)); - } - } else { - switch (dt->u.atomic.order) { - case H5T_ORDER_LE: - s = "little endian"; - break; - case H5T_ORDER_BE: - s = "big endian"; - break; - case H5T_ORDER_VAX: - s = "VAX"; - break; - case H5T_ORDER_NONE: - s = "none"; - break; - default: - sprintf(buf, "H5T_ORDER_%d", dt->u.atomic.order); - s = buf; - break; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "Byte order:", - s); - - fprintf(stream, "%*s%-*s %lu bit%s\n", indent, "", fwidth, - "Precision:", - (unsigned long) (dt->u.atomic.prec), - 1 == dt->u.atomic.prec ? "" : "s"); - - fprintf(stream, "%*s%-*s %lu bit%s\n", indent, "", fwidth, - "Offset:", - (unsigned long) (dt->u.atomic.offset), - 1 == dt->u.atomic.offset ? "" : "s"); - - switch (dt->u.atomic.lsb_pad) { - case H5T_PAD_ZERO: - s = "zero"; - break; - case H5T_PAD_ONE: - s = "one"; - break; - default: - s = "pad?"; - break; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "Low pad type:", s); - - switch (dt->u.atomic.msb_pad) { - case H5T_PAD_ZERO: - s = "zero"; - break; - case H5T_PAD_ONE: - s = "one"; - break; - default: - s = "pad?"; - break; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "High pad type:", s); - - if (H5T_FLOAT == dt->type) { - switch (dt->u.atomic.u.f.pad) { - case H5T_PAD_ZERO: - s = "zero"; - break; - case H5T_PAD_ONE: - s = "one"; - break; - default: - if (dt->u.atomic.u.f.pad < 0) { - sprintf(buf, "H5T_PAD_%d", -(dt->u.atomic.u.f.pad)); - } else { - sprintf(buf, "bit-%d", dt->u.atomic.u.f.pad); - } - s = buf; - break; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "Internal pad type:", s); - - switch (dt->u.atomic.u.f.norm) { - case H5T_NORM_IMPLIED: - s = "implied"; - break; - case H5T_NORM_MSBSET: - s = "msb set"; - break; - case H5T_NORM_NONE: - s = "none"; - break; - default: - sprintf(buf, "H5T_NORM_%d", (int) (dt->u.atomic.u.f.norm)); - s = buf; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "Normalization:", s); - - fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, - "Sign bit location:", - (unsigned long) (dt->u.atomic.u.f.sign)); - - fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, - "Exponent location:", - (unsigned long) (dt->u.atomic.u.f.epos)); - - fprintf(stream, "%*s%-*s 0x%08lx\n", indent, "", fwidth, - "Exponent bias:", - (unsigned long) (dt->u.atomic.u.f.ebias)); - - fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, - "Exponent size:", - (unsigned long) (dt->u.atomic.u.f.esize)); - - fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, - "Mantissa location:", - (unsigned long) (dt->u.atomic.u.f.mpos)); - - fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, - "Mantissa size:", - (unsigned long) (dt->u.atomic.u.f.msize)); - - } else if (H5T_INTEGER == dt->type) { - switch (dt->u.atomic.u.i.sign) { - case H5T_SGN_NONE: - s = "none"; - break; - case H5T_SGN_2: - s = "2's comp"; - break; - default: - sprintf(buf, "H5T_SGN_%d", (int) (dt->u.atomic.u.i.sign)); - s = buf; - break; - } - fprintf(stream, "%*s%-*s %s\n", indent, "", fwidth, - "Sign scheme:", s); - - } - } -#endif /* LATER */ + fprintf(stream, "%*s%-*s \"%s\"\n", indent, "", fwidth, + "Name:", + mesg->name); + fprintf(stream, "%*s%-*s %u\n", indent, "", fwidth, + "Initialized:", + (unsigned int)mesg->initialized); + fprintf(stream, "%*s%-*s %u\n", indent, "", fwidth, + "Opened:", + (unsigned int)mesg->ent_opened); + fprintf(stream, "%*sSymbol table entry...\n", indent, ""); + H5G_ent_debug(f, &(mesg->ent), stream, indent+3, MAX(0, fwidth-3), NULL); + + fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, + "Data type size:", + (unsigned long)(mesg->dt_size)); + fprintf(stream, "%*sData type...\n", indent, ""); + (H5O_DTYPE->debug)(f, mesg->dt, stream, indent+3, MAX(0, fwidth-3)); + + fprintf(stream, "%*s%-*s %lu\n", indent, "", fwidth, + "Data space size:", + (unsigned long)(mesg->ds_size)); + fprintf(stream, "%*sData space...\n", indent, ""); + (H5O_SDSPACE->debug)(f, mesg->ds, stream, indent+3, MAX(0, fwidth-3)); FUNC_LEAVE(SUCCEED); } diff --git a/src/H5Oprivate.h b/src/H5Oprivate.h index 677241e..7f6cabb 100644 --- a/src/H5Oprivate.h +++ b/src/H5Oprivate.h @@ -184,7 +184,7 @@ typedef struct H5O_compress_t { /* * Attribute Message. */ -#define H5O_ATTR_ID 0x000C +#define H5O_ATTR_ID 0x000c extern const H5O_class_t H5O_ATTR[1]; /* operates on an H5A_t struct */ diff --git a/src/H5Oshared.c b/src/H5Oshared.c index b5c810b..f485143 100644 --- a/src/H5Oshared.c +++ b/src/H5Oshared.c @@ -64,6 +64,7 @@ static void * H5O_shared_decode (H5F_t *f, const uint8 *buf, H5O_shared_t __unused__ *sh) { H5O_shared_t *mesg; + uintn flags; FUNC_ENTER (H5O_shared_decode, NULL); @@ -77,7 +78,11 @@ H5O_shared_decode (H5F_t *f, const uint8 *buf, H5O_shared_t __unused__ *sh) HRETURN_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed"); } - UINT32DECODE (buf, mesg->in_gh); + + flags = *buf++; + mesg->in_gh = (flags & 0x01); + buf += 7; /*reserved*/ + if (mesg->in_gh) { H5F_addr_decode (f, &buf, &(mesg->u.gh.addr)); INT32DECODE (buf, mesg->u.gh.idx); @@ -109,6 +114,7 @@ static herr_t H5O_shared_encode (H5F_t *f, uint8 *buf/*out*/, const void *_mesg) { const H5O_shared_t *mesg = (const H5O_shared_t *)_mesg; + uintn flags; FUNC_ENTER (H5O_shared_encode, FAIL); @@ -118,7 +124,16 @@ H5O_shared_encode (H5F_t *f, uint8 *buf/*out*/, const void *_mesg) assert (mesg); /* Encode */ - INT32ENCODE (buf, mesg->in_gh); + flags = mesg->in_gh ? 0x01 : 0x00; + *buf++ = flags; + *buf++ = 0; /*reserved 1*/ + *buf++ = 0; /*reserved 2*/ + *buf++ = 0; /*reserved 3*/ + *buf++ = 0; /*reserved 4*/ + *buf++ = 0; /*reserved 5*/ + *buf++ = 0; /*reserved 6*/ + *buf++ = 0; /*reserved 7*/ + if (mesg->in_gh) { H5F_addr_encode (f, &buf, &(mesg->u.gh.addr)); INT32ENCODE (buf, mesg->u.gh.idx); @@ -153,7 +168,8 @@ H5O_shared_size (H5F_t *f, const void __unused__ *_mesg) FUNC_ENTER (H5O_shared_size, 0); - size = 4 + /*the flags field */ + size = 1 + /*the flags field */ + 7 + /*reserved */ MAX (H5F_SIZEOF_ADDR(f)+4, /*sharing via global heap */ H5G_SIZEOF_ENTRY(f)); /*sharing by another obj hdr */ diff --git a/src/H5detect.c b/src/H5detect.c index 7398cc1..1236a5b 100644 --- a/src/H5detect.c +++ b/src/H5detect.c @@ -953,7 +953,7 @@ main(void) * systems. */ DETECT_I(long, LLONG, d[nd]); nd++; - DETECT_I(unsigned long, ULLONG, d[nd]); dn++; + DETECT_I(unsigned long, ULLONG, d[nd]); nd++; #else DETECT_I(long long, LLONG, d[nd]); nd++; DETECT_I(unsigned long long, ULLONG, d[nd]); nd++; diff --git a/test/.distdep b/test/.distdep index 349087e..acce30e 100644 --- a/test/.distdep +++ b/test/.distdep @@ -100,34 +100,6 @@ tstab.o: \ ../src/H5Tpublic.h \ ../src/H5Sprivate.h \ ../src/H5Spublic.h -dtypes.o: \ - dtypes.c \ - ../src/hdf5.h \ - ../src/H5public.h \ - ../src/H5config.h \ - ../src/H5Ipublic.h \ - ../src/H5Apublic.h \ - ../src/H5ACpublic.h \ - ../src/H5Bpublic.h \ - ../src/H5Dpublic.h \ - ../src/H5Epublic.h \ - ../src/H5Fpublic.h \ - ../src/H5Gpublic.h \ - ../src/H5HGpublic.h \ - ../src/H5HLpublic.h \ - ../src/H5MFpublic.h \ - ../src/H5MMpublic.h \ - ../src/H5Opublic.h \ - ../src/H5Ppublic.h \ - ../src/H5Zpublic.h \ - ../src/H5Spublic.h \ - ../src/H5Tpublic.h \ - ../src/H5Tpkg.h \ - ../src/H5HGprivate.h \ - ../src/H5Fprivate.h \ - ../src/H5private.h \ - ../src/H5Tprivate.h \ - ../src/H5Gprivate.h hyperslab.o: \ hyperslab.c \ ../src/H5private.h \ @@ -440,12 +412,6 @@ big.o: \ ../src/H5Spublic.h \ ../src/H5Tpublic.h \ ../src/H5private.h -testhdf5.o: \ - testhdf5.c \ - testhdf5.h \ - ../src/H5private.h \ - ../src/H5public.h \ - ../src/H5config.h tselect.o: \ tselect.c \ testhdf5.h \ @@ -470,3 +436,37 @@ tselect.o: \ ../src/H5Ppublic.h \ ../src/H5Zpublic.h \ ../src/H5Spublic.h +testhdf5.o: \ + testhdf5.c \ + testhdf5.h \ + ../src/H5private.h \ + ../src/H5public.h \ + ../src/H5config.h +dtypes.o: \ + dtypes.c \ + ../src/hdf5.h \ + ../src/H5public.h \ + ../src/H5config.h \ + ../src/H5Ipublic.h \ + ../src/H5Apublic.h \ + ../src/H5ACpublic.h \ + ../src/H5Bpublic.h \ + ../src/H5Dpublic.h \ + ../src/H5Epublic.h \ + ../src/H5Fpublic.h \ + ../src/H5Gpublic.h \ + ../src/H5HGpublic.h \ + ../src/H5HLpublic.h \ + ../src/H5MFpublic.h \ + ../src/H5MMpublic.h \ + ../src/H5Opublic.h \ + ../src/H5Ppublic.h \ + ../src/H5Zpublic.h \ + ../src/H5Spublic.h \ + ../src/H5Tpublic.h \ + ../src/H5Tpkg.h \ + ../src/H5HGprivate.h \ + ../src/H5Fprivate.h \ + ../src/H5private.h \ + ../src/H5Tprivate.h \ + ../src/H5Gprivate.h diff --git a/test/dtypes.c b/test/dtypes.c index 8898adc..4963d35 100644 --- a/test/dtypes.c +++ b/test/dtypes.c @@ -29,6 +29,10 @@ # define __unused__ __attribute__((unused)) #endif +#if SIZEOF_DOUBLE != SIZEOF_LONG_DOUBLE +# define USE_LDOUBLE +#endif + #ifndef MAX # define MAX(X,Y) ((X)>(Y)?(X):(Y)) # define MIN(X,Y) ((X)<(Y)?(X):(Y)) @@ -769,10 +773,12 @@ my_isnan(flt_t type, void *val) retval = (*((double*)val)!=*((double*)val)); break; +#ifdef USE_LDOUBLE case FLT_LDOUBLE: retval = (*((long double*)val)!=*((long double*)val)); break; - +#endif + default: return 0; } @@ -791,9 +797,11 @@ my_isnan(flt_t type, void *val) sprintf(s, "%g", *((double*)val)); break; +#ifdef USE_LDOUBLE case FLT_LDOUBLE: sprintf(s, "%Lg", *((long double*)val)); break; +#endif default: return 0; @@ -843,7 +851,9 @@ test_conv_flt_1 (const char *name, hid_t src, hid_t dst) char str[256]; /*hello string */ float hw_f; /*hardware-converted */ double hw_d; /*hardware-converted */ +#ifdef USE_LDOUBLE long double hw_ld; /*hardware-converted */ +#endif unsigned char *hw=NULL; /*ptr to hardware-conv'd*/ size_t i, j, k; /*counters */ int endian; /*machine endianess */ @@ -889,9 +899,11 @@ test_conv_flt_1 (const char *name, hid_t src, hid_t dst) } else if (H5Tequal(src, H5T_NATIVE_DOUBLE)) { src_type_name = "double"; src_type = FLT_DOUBLE; +#ifdef USE_LDOUBLE } else if (H5Tequal(src, H5T_NATIVE_LDOUBLE)) { src_type_name = "long double"; src_type = FLT_LDOUBLE; +#endif } else { src_type_name = "UNKNOWN"; src_type = FLT_OTHER; @@ -903,9 +915,11 @@ test_conv_flt_1 (const char *name, hid_t src, hid_t dst) } else if (H5Tequal(dst, H5T_NATIVE_DOUBLE)) { dst_type_name = "double"; dst_type = FLT_DOUBLE; +#ifdef USE_LDOUBLE } else if (H5Tequal(dst, H5T_NATIVE_LDOUBLE)) { dst_type_name = "long double"; dst_type = FLT_LDOUBLE; +#endif } else { dst_type_name = "UNKNOWN"; dst_type = FLT_OTHER; @@ -956,7 +970,9 @@ test_conv_flt_1 (const char *name, hid_t src, hid_t dst) for (j=0; j=max_fails) { -- cgit v0.12
    - HDF5 Attribute-List Message Layout + + @@ -2765,131 +2697,105 @@ - - - - - - - - - - - - - - - - - - - - - - - - + - + + + - + + + - + + - + + - + + - + - - +
    + Attribute Message
    byte byte byte
    Attribute List Flags
    # of Simple Attributes
    Simple Attribute #1 Name Offset
    Simple Attribute #1 Data-Type
    Simple Attribute #1 Rank
    Simple Attribute #1 Dim #1 Size
    Simple Attribute #1 Dim #2 Size
    Simple Attribute #1 Dim #3 Size
    Simple Attribute #1 Dim #4 SizeName SizeType Size
    Simple Attribute #1 Data OffsetSpace SizeReserved
    .
    .
    .

    Name

    Simple Attribute #n Name Offset
    Type

    Simple Attribute #n Data-Type
    Space

    Simple Attribute #n Rank
    Data

    Simple Attribute #n Dim #1 Size
    + + +

    +

    + + + + - - + + + + - - + + + + - - + + + + - - + + + + - - + + + + - - + + + + - - + + + + - - + + + +
    Field NameDescription
    Simple Attribute #n Dim #2 Size
    Name SizeThe length of the attribute name in bytes including the + null terminator. Note that the Name field below may + contain additional padding not represented by this + field.
    Simple Attribute #n Dim #3 Size
    Type SizeThe length of the data type description in the Type + field below. Note that the Type field may contain + additional padding not represented by this field.
    Simple Attribute #n Dim #4 Size
    Space SizeThe length of the data space description in the Space + field below. Note that the Space field may contain + additional padding not represented by this field.
    Simple Attribute #n Data Offset
    ReservedThis field is reserved for later use and is set to + zero.
    # of Complex Attributes
    NameThe null-terminated attribute name. This field is + padded with additional null characters to make it a + multiple of eight bytes.
    Pointer to Complex Attribute #1
    TypeThe data type description follows the same format as + described for the data type object header message. This + field is padded with additional zero bytes to make it a + multiple of eight bytes.
    .
    .
    .
    SpaceThe data space description follows the same format as + described for the data space object header message. This + field is padded with additional zero bytes to make it a + multiple of eight bytes.
    Pointer to Complex Attribute #n
    DataThe raw data for the attribute. The size is determined + from the data type and data space descriptions. This + field is not padded with additional zero + bytes.
    -

    -

    -
    The elements of the attribute list message are described below: -
    -
    -
    Attribute List Flags: (unsigned 32-bit integer) -
    These flags indicate the presence of simple and complex - lists of attributes for this dataset. Bit 0 indicates the - presence of a list of simple attributes and Bit 1 - indicates the presence of a list of complex attributes. - -
    # of Simple Attributes: (unsigned 32-bit integer) -
    This indicates the number of simple attributes for this - dataset. - -
    Simple Attribute #n Name Offset: (unsigned 32-bit integer) -
    This is the offset of the simple attribute's name in the - global small-data heap. - -
    Simple Attribute #n Data-type: (unsigned 32-bit integer) -
    This is a simple data-type, which indicates the type of - data used for the attribute. - -
    Simple Attribute #n Rank: (unsigned 32-bit integer) -
    This is the number of dimensions of the attribute, - limited to four or less. - -
    Simple Attribute #n Dim #n Size: (unsigned 32-bit integer) -
    This is the size of the attribute's n'th dimension, - which is stored in the canonical order for dimensions - (i.e. no permutations of the indices are allowed). - -
    Simple Attribute #n Data Offset: (unsigned 32-bit integer) -
    This is the offset of the simple attribute's data in the - global small-data. - -
    # of Complex Attributes: (unsigned 32-bit integer) -
    This indicates the number of complex attributes for this - dataset. - -
    Pointer to Complex Attribute #n: (unsigned 32-bit integer) -
    This is the small-data heap offset of the name of the - attribute object in the file. -
    -
    - -

    [Note: It has been suggested that each attribute have an - additional "units" field, so this is being considered.] - -

    Examples:

    - [Comment: need examples.] -

    Name: Object Name

    Type: 0x000D
    @@ -2933,10 +2839,9 @@ by writing that message in the global heap and having the object headers all point to it. The pointing is accomplished with a Shared Object message which is understood directly by the object - header layer of the library and never actually appears as a - message in the file. It is also possible to have a message of - one object header point to a message in some other object - header, but care must be exercised to prevent cycles. + header layer of the library. It is also possible to have a + message of one object header point to a message in some other + object header, but care must be exercised to prevent cycles.

    If a message is shared, then the message appears in the global heap and its message ID appears in the Header Message Type @@ -2961,7 +2866,12 @@

    FlagsFlagsReserved
    Reserved
    FlagsThe Shared Message Message is a pointer to a Shared - Message. The actual shared message can appear in either - the global heap or in some other object header and this - field specifies which form is used. If the value is zero - then the actual message is the first such message in some - other object header; otherwise the actual message is - stored in the global heap.The Shared Message message points to a message which is + shared among multiple object headers. The Flags field + describes the type of sharing: + +
    +
    Bit 0 +
    If this bit is clear then the actual message is the + first message in some other object header; otherwise + the actual message is stored in the global heap. + +
    Bits 2-7 +
    Reserved (always zero) +