diff options
author | Robb Matzke <matzke@llnl.gov> | 1999-08-10 20:21:32 (GMT) |
---|---|---|
committer | Robb Matzke <matzke@llnl.gov> | 1999-08-10 20:21:32 (GMT) |
commit | cbf68fc824f69dcdc0fcd38a83a3fee8c7093c28 (patch) | |
tree | b7075c9f11db1b343593b5e0619a285ccc8418af /src/H5Fsplit.c | |
parent | 7d949c9da91b33955d4e253c1093a4f23fb63523 (diff) | |
download | hdf5-cbf68fc824f69dcdc0fcd38a83a3fee8c7093c28.zip hdf5-cbf68fc824f69dcdc0fcd38a83a3fee8c7093c28.tar.gz hdf5-cbf68fc824f69dcdc0fcd38a83a3fee8c7093c28.tar.bz2 |
[svn-r1568] Changes since 19990730
----------------------
This extensive change is the virtual file layer implementation. I've
ported and tested the sec2, family, and core drivers and only ported
the mpio driver (Albert will test it). So if you need MPIO I would
recommend sticking with the previous version for a while.
You will get a few compile warnings about split and stdio drivers not
being implemented and possibly tracing information not inserted in
some of the drivers. You can safely ignore them but I plan to fix
them.
I'm still working on the split driver because I just realized that it
needs a part of the VFL that isn't written yet.
Documentation is being updated also because there were some minor
changes (mostly just name changes). It should be available on my web
site later this week.
./MANIFEST
./src/Makefile.in
./src/hdf5.h
./src/H5Flow.c [REMOVED]
./src/H5Fstdio.c [REMOVED]
./src/H5Fsec2.c [REMOVED]
./src/H5Fsplit.c [REMOVED]
./src/H5Fmpio.c [REMOVED]
./src/H5Ffamily.c [REMOVED]
./src/H5Fcore.c [REMOVED]
./src/H5MFpublic.h [REMOVED]
./src/H5FD.c [NEW]
./src/H5FDcore.c [NEW]
./src/H5FDcore.h [NEW]
./src/H5FDfamily.c [NEW]
./src/H5FDfamily.h [NEW]
./src/H5FDmpio.c [NEW]
./src/H5FDmpio.h [NEW]
./src/H5FDprivate.h [NEW]
./src/H5FDpublic.h [NEW]
./src/H5FDsec2.c [NEW]
./src/H5FDsec2.h [NEW]
Removed/added files for virtual file layer.
./bin/trace
./src/H5.c
Removed unused public datatypes and added new VFL public
datatypes.
Changed an error message.
./config/BlankForm
./config/dec-flags
./config/gnu-flags
./config/hpux10.20
./config/hpux9.03
./config/irix5.x
./config/irix6.x
./config/solaris2.x
./config/unicosmk
Removed the H5F_OPT_SEEK and H5F_LOW_DFLT constants from the
configuration since they're no longer applicable. The default
file driver is always the sec2 driver and it always optimizes
calls to lseek() or lseek64().
./config/depend.in
C preprocessor errors generated during automatic dependency
building are sent to /dev/null to prevent them from appearing
twice in the make output.
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oshared.c
./src/H5T.c
./src/H5detect.c
./test/ohdr.c
Changed H5F_ADDR_UNDEF to HADDR_UNDEF to be more consistent
with the `haddr_t' datatype which is now a public type.
./src/H5D.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Tconv.c
./test/cmpd_dset.c
./test/dsets.c
./test/overhead.c
./test/tselect.c
./test/tvltypes.c
The H5P_DATASET_XFER constant was changed to H5P_DATA_XFER
because the properties apply to all types of I/O operations,
not just datasets.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5R.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/big.c
./test/h5test.c
./test/istore.c
./testpar/t_dset.c
./testpar/t_file.c
./tools/h5debug.c
./tools/h5ls.c
Modified to work with the virtual file layer by calling H5FD_*
functions instead of H5F_low_* functions and by passing file
access and data transfer properties by object ID instead of
pointer.
Changed H5D_transfer_t to H5FD_mpio_xfer_t since the
COLLECTIVE vs. INDEPENDENT transfer mode is specific to the
MPIO file driver.
Moved MPIO-specific stuff into the MPIO driver.
./src/H5B.c
./src/H5D.c
./src/H5Fprivate.h
The H5F_mpio_* private functions were renamed and placed in
the H5FDmpio driver except those which appeared in H5Smpio.c.
./src/H5E.c
./src/H5Epublic.h
Added major error number H5E_VFL for virtual file layer
related errors.
./src/H5F.c
./src/H5Fprivate.h
Changed the logic that controls whether the boot block is
written. Instead of assuming that the first call to write the
boot block is only to allocate space, I've added a function
argument which makes this explicit.
Changed the way files are compared so that a driver-defined
comparison function can be called. Files which belong to
different drivers are always considered different.
Removed H5F_driver_t since file drivers are now identified by
object ID instead of a special non-user-extendible datatype.
Removed all the hard-coded low-level file properties which
have been replaced by the various file drivers.
./src/H5I.c
./src/H5Iprivate.h
Added the H5I_inc_ref() which was removed a few months ago
since we finally have a use for it.
./src/H5Ipublic.h
Added the H5I_VFL object ID type to identify file drivers in
the virtual file layer.
./src/H5MF.c
./src/H5MFprivate.h
Moved all the allocation/deallocation code into the virtual
file layer which allows file drivers to override much of it.
./src/H5P.c
./src/H5Ppublic.h
Moved file driver-specific code into the various file driver
files.
The H5Pcopy() and H5Pclose() functions make calls into the
virtual file driver to manage the memory for driver-specific
file access and data transfer properties.
./src/H5private.h
./src/H5public.h
The `haddr_t' type is now public.
./test/tfile.c
Added a few more comments.
Diffstat (limited to 'src/H5Fsplit.c')
-rw-r--r-- | src/H5Fsplit.c | 523 |
1 files changed, 0 insertions, 523 deletions
diff --git a/src/H5Fsplit.c b/src/H5Fsplit.c deleted file mode 100644 index d18b113..0000000 --- a/src/H5Fsplit.c +++ /dev/null @@ -1,523 +0,0 @@ -/* - * Copyright (C) 1997 NCSA - * All rights reserved. - * - * Programmer: Robb Matzke <matzke@llnl.gov> - * Thursday, November 13, 1997 - * - * Purpose: A driver that splits the meta data and raw data into two - * separate files. The high-order bit of the file address - * determines whether the address refers to the meta data file - * (high order bit is clear) or the raw data file (high order bit - * is set). - */ -#include <H5private.h> -#include <H5Eprivate.h> -#include <H5Fprivate.h> -#include <H5MFprivate.h> -#include <H5MMprivate.h> - -/* Default file name extensions */ -#define H5F_SPLIT_META_EXT ".meta" -#define H5F_SPLIT_RAW_EXT ".raw" - -#define PABLO_MASK H5Fsplit_mask -static intn interface_initialize_g = 0; -#define INTERFACE_INIT NULL - -static htri_t H5F_split_access(const char *name, - const H5F_access_t *access_parms, int mode, - H5F_search_t *key/*out*/); -static H5F_low_t *H5F_split_open(const char *name, - const H5F_access_t *access_parms, uintn flags, - H5F_search_t *key/*out*/); -static herr_t H5F_split_close(H5F_low_t *lf, const H5F_access_t *access_parms); -static herr_t H5F_split_read(H5F_low_t *lf, const H5F_access_t *access_parms, - const H5F_xfer_t *xfer_parms, haddr_t addr, - size_t size, uint8_t *buf/*out*/); -static herr_t H5F_split_write(H5F_low_t *lf, const H5F_access_t *access_parms, - const H5F_xfer_t *xfer_parms, - haddr_t addr, size_t size, const uint8_t *buf); -static herr_t H5F_split_flush(H5F_low_t *lf, const H5F_access_t *access_parms); -static herr_t H5F_split_extend(H5F_low_t *lf, const H5F_access_t *access_parms, - intn op, hsize_t size, haddr_t *addr/*out*/); -static intn H5F_split_alloc (H5F_low_t *lf, intn op, hsize_t alignment, - hsize_t threshold, hsize_t size, H5MF_free_t *blk, - haddr_t *addr/*out*/); - -const H5F_low_class_t H5F_LOW_SPLIT_g[1] = {{ - H5F_split_access, /*access method */ - H5F_split_open, /*open method */ - H5F_split_close, /*close method */ - H5F_split_read, /*read method */ - H5F_split_write, /*write method */ - H5F_split_flush, /*flush method */ - H5F_split_extend, /*extend method */ - H5F_split_alloc, /*alloc method */ -}}; - -/* - * This is the bit that determines whether the address is part of the meta - * data file or part of the raw data file. Eventually we'll want to pass - * this kind of thing down to this function from above... - */ -#define H5F_SPLIT_MASK 0x80000000 - - -/*------------------------------------------------------------------------- - * Function: H5F_split_open - * - * Purpose: Opens a split meta data/raw data family with the specified - * base name. The name of the meta data file will be created by - * appending `.h5' while the name of the raw data file will be - * created by appending `.raw'. - * - * Return: Success: Low-level file pointer - * - * Failure: NULL - * - * Programmer: Robb Matzke - * Monday, November 13, 1997 - * - * Modifications: - * - *------------------------------------------------------------------------- - */ -static H5F_low_t * -H5F_split_open(const char *name, const H5F_access_t *access_parms, - uintn flags, H5F_search_t *key/*out*/) -{ - H5F_low_t *lf = NULL, *ret_value = NULL; - char fullname[4096]; - const char *ext; /*file name extension*/ - const H5F_low_class_t *meta_type = NULL; - const H5F_low_class_t *raw_type = NULL; - - FUNC_ENTER(H5F_split_open, NULL); - - assert(name && *name); - assert (access_parms); - assert (H5F_LOW_SPLIT==access_parms->driver); - assert (access_parms->u.split.meta_access); - assert (access_parms->u.split.raw_access); - - /* Get member types */ - meta_type = H5F_low_class (access_parms->u.split.meta_access->driver); - raw_type = H5F_low_class (access_parms->u.split.raw_access->driver); - - /* Create the file descriptor */ - if (NULL==(lf = H5MM_calloc(sizeof(H5F_low_t)))) { - HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, - "memory allocation failed"); - } - lf->u.split.name = H5MM_xstrdup(name); - lf->u.split.mask = H5F_SPLIT_MASK; - - /* Open the meta data file */ - ext = access_parms->u.split.meta_ext ? - access_parms->u.split.meta_ext : H5F_SPLIT_META_EXT; - if (HDstrlen(name)+HDstrlen(ext) >= sizeof fullname) { - HGOTO_ERROR (H5E_IO, H5E_CANTINIT, NULL, "file name is too long"); - } - HDstrcpy(fullname, name); - HDstrcat(fullname, ext); - - lf->u.split.meta = H5F_low_open(meta_type, fullname, - access_parms->u.split.meta_access, - flags, key/*out*/); - if (NULL == lf->u.split.meta) { - HGOTO_ERROR(H5E_IO, H5E_CANTOPENFILE, NULL, "can't open meta file"); - } - - /* Open the raw data file */ - ext = access_parms->u.split.raw_ext ? - access_parms->u.split.raw_ext : H5F_SPLIT_RAW_EXT; - if (HDstrlen(name)+HDstrlen(ext) >= sizeof fullname) { - HGOTO_ERROR (H5E_IO, H5E_CANTINIT, NULL, "file name is too long"); - } - HDstrcpy(fullname, name); - HDstrcat(fullname, ext); - - lf->u.split.raw = H5F_low_open(raw_type, fullname, - access_parms->u.split.raw_access, - flags, NULL); - if (NULL == lf->u.split.raw) { - HGOTO_ERROR(H5E_IO, H5E_CANTOPENFILE, NULL, "can't open raw file"); - } - - /* Initialize the file size */ - H5F_low_size(lf->u.split.raw, &(lf->eof)); - lf->eof |= lf->u.split.mask; - - HRETURN(lf); - - done: - if (!ret_value) { - if (lf) { - H5F_split_close(lf, access_parms); - H5MM_xfree(lf); - } - } - FUNC_LEAVE(ret_value); -} - -/*------------------------------------------------------------------------- - * Function: H5F_split_close - * - * Purpose: Closes a split file. - * - * Return: Non-negative on success/Negative on failure - * - * Programmer: Robb Matzke - * Monday, November 13, 1997 - * - * Modifications: - * - *------------------------------------------------------------------------- - */ -static herr_t -H5F_split_close(H5F_low_t *lf, const H5F_access_t *access_parms) -{ - FUNC_ENTER(H5F_split_close, FAIL); - - assert(lf); - - H5F_low_close(lf->u.split.meta, access_parms->u.split.meta_access); - H5F_low_close(lf->u.split.raw, access_parms->u.split.raw_access); - H5MM_xfree(lf->u.split.name); - - FUNC_LEAVE(SUCCEED); -} - -/*------------------------------------------------------------------------- - * Function: H5F_split_read - * - * Purpose: Reads a chunk of contiguous data from a split file. We - * assume that the data being read never crosses the meta - * data/raw data boundary. Reading past the end of a file - * returns zeros instead of failing. - * - * Return: Non-negative on success/Negative on failure - * - * Programmer: Robb Matzke - * Monday, November 13, 1997 - * - * Modifications: - * Albert Cheng, 1998-06-02 - * Added XFER_MODE argument. - * - * Robb Matzke, 1999-07-28 - * The ADDR argument is passed by value. - *------------------------------------------------------------------------- - */ -static herr_t -H5F_split_read(H5F_low_t *lf, const H5F_access_t *access_parms, - const H5F_xfer_t *xfer_parms, haddr_t addr, size_t size, - uint8_t *buf/*out*/) -{ - haddr_t tmp_addr; - H5F_low_t *sub = NULL; - herr_t status; - const H5F_access_t *sub_parms = NULL; - - FUNC_ENTER(H5F_split_read, FAIL); - - assert(lf); - assert(H5F_addr_defined(addr)); - assert(buf); - /* no collective support */ - assert(xfer_parms->xfer_mode != H5D_XFER_COLLECTIVE); - - /* Which file to we actually read from? */ - if (addr & lf->u.split.mask) { - sub = lf->u.split.raw; - sub_parms = access_parms->u.split.raw_access; - tmp_addr = addr & (lf->u.split.mask - 1); - } else { - sub = lf->u.split.meta; - sub_parms = access_parms->u.split.meta_access; - tmp_addr = addr; - } - - /* Read the data */ - status = H5F_low_read(sub, sub_parms, xfer_parms, tmp_addr, size, - buf/*out*/); - FUNC_LEAVE(status); -} - - -/*------------------------------------------------------------------------- - * Function: H5F_split_write - * - * Purpose: Writes BUF to either the meta data file or the raw data file. - * - * Return: Non-negative on success/Negative on failure - * - * Programmer: Robb Matzke - * Monday, November 13, 1997 - * - * Modifications: - * Albert Cheng, 1998-06-02 - * Added XFER_MODE argument. - * - * Robb Matzke, 1999-07-28 - * The ADDR argument is passed by value. - *------------------------------------------------------------------------- - */ -static herr_t -H5F_split_write(H5F_low_t *lf, const H5F_access_t *access_parms, - const H5F_xfer_t *xfer_parms, haddr_t addr, size_t size, - const uint8_t *buf) -{ - haddr_t tmp_addr; - H5F_low_t *sub = NULL; - herr_t status; - const H5F_access_t *sub_parms = NULL; - - FUNC_ENTER(H5F_split_write, FAIL); - - assert(lf); - assert(H5F_addr_defined(addr)); - assert(buf); - /* no collective support */ - assert(xfer_parms->xfer_mode != H5D_XFER_COLLECTIVE); - - /* Which file to we actually write to? */ - if (addr & lf->u.split.mask) { - sub = lf->u.split.raw; - sub_parms = access_parms->u.split.raw_access; - tmp_addr = addr & (lf->u.split.mask - 1); - } else { - sub = lf->u.split.meta; - sub_parms = access_parms->u.split.meta_access; - tmp_addr = addr; - } - - /* Write the data */ - status = H5F_low_write(sub, sub_parms, xfer_parms, tmp_addr, size, buf); - FUNC_LEAVE(status); -} - -/*------------------------------------------------------------------------- - * Function: H5F_split_flush - * - * Purpose: Flushes all data to disk. - * - * Return: Non-negative on success/Negative on failure - * - * Programmer: Robb Matzke - * Monday, November 13, 1997 - * - * Modifications: - * - *------------------------------------------------------------------------- - */ -static herr_t -H5F_split_flush(H5F_low_t *lf, const H5F_access_t *access_parms) -{ - herr_t ret_value = SUCCEED; - - FUNC_ENTER(H5F_split_flush, FAIL); - - assert(lf); - - ret_value = (H5F_low_flush(lf->u.split.meta, - access_parms->u.split.meta_access) >= 0 && - H5F_low_flush(lf->u.split.raw, - access_parms->u.split.raw_access) >= 0); - - FUNC_LEAVE(ret_value); -} - -/*------------------------------------------------------------------------- - * Function: H5F_split_access - * - * Purpose: Determines if both members of the split data file family can - * be accessed and returns the key for the first member of the - * family. - * - * Return: Success: TRUE or FALSE - * - * Failure: FAIL - * - * Programmer: Robb Matzke - * Monday, November 13, 1997 - * - * Modifications: - * - *------------------------------------------------------------------------- - */ -static htri_t -H5F_split_access(const char *name, const H5F_access_t *access_parms, - int mode, H5F_search_t *key/*out*/) -{ - char fullname[4096]; - htri_t status; - const char *ext; /*file extension*/ - const H5F_low_class_t *meta_type = NULL; - const H5F_low_class_t *raw_type = NULL; - - FUNC_ENTER(H5F_split_access, FAIL); - - assert(name && *name); - assert (access_parms); - assert (H5F_LOW_SPLIT==access_parms->driver); - assert (access_parms->u.split.meta_access); - assert (access_parms->u.split.raw_access); - - /* The meta data member */ - meta_type = H5F_low_class (access_parms->u.split.meta_access->driver); - ext = access_parms->u.split.meta_ext ? - access_parms->u.split.meta_ext : H5F_SPLIT_META_EXT; - if (HDstrlen(name)+HDstrlen(ext) >= sizeof fullname) { - HRETURN_ERROR (H5E_IO, H5E_CANTINIT, FAIL, "file name is too long"); - } - HDstrcpy(fullname, name); - HDstrcat(fullname, ext); - - status = H5F_low_access(meta_type, fullname, - access_parms->u.split.meta_access, - mode, key/*out*/); - if (status < 0) { - HRETURN_ERROR(H5E_IO, H5E_CANTINIT, FAIL, - "access call failed for meta data member"); - } - if (!status) HRETURN(FALSE); - - /* The raw data member */ - raw_type = H5F_low_class (access_parms->u.split.raw_access->driver); - ext = access_parms->u.split.raw_ext ? - access_parms->u.split.raw_ext : H5F_SPLIT_RAW_EXT; - if (HDstrlen(name)+HDstrlen(ext) >= sizeof fullname) { - HRETURN_ERROR (H5E_IO, H5E_CANTINIT, FAIL, "file name is too long"); - } - HDstrcpy(fullname, name); - HDstrcat(fullname, ext); - - status = H5F_low_access(raw_type, fullname, - access_parms->u.split.raw_access, - mode, NULL/*out*/); - if (status < 0) { - HRETURN_ERROR(H5E_IO, H5E_CANTINIT, FAIL, - "access call failed for raw data member"); - } - FUNC_LEAVE(status); -} - -/*------------------------------------------------------------------------- - * Function: H5F_split_extend - * - * Purpose: Allocates memory from the end of the meta data file or raw - * data file. - * - * Return: Non-negative on success (with the address of the allocated - * memory returned through the ADDR_P argument.) /Negative - * on failure - * - * Programmer: Robb Matzke - * Thursday, November 13, 1997 - * - * Modifications: - * - *------------------------------------------------------------------------- - */ -static herr_t -H5F_split_extend(H5F_low_t *lf, const H5F_access_t *access_parms, intn op, - hsize_t size, haddr_t *addr_p/*out*/) -{ - FUNC_ENTER(H5F_split_extend, FAIL); - - assert(lf); - assert(H5MF_META == op || H5MF_RAW == op); - assert(size > 0); - assert(addr_p); - - if (H5MF_META == op) { - if (H5F_low_extend(lf->u.split.meta, access_parms->u.split.meta_access, - op, size, addr_p/*out*/)<0) { - HRETURN_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, - "meta data allocation failed"); - } - if (*addr_p + size > lf->eof) { - lf->eof = *addr_p + size; - } - } else { - if (H5F_low_extend(lf->u.split.raw, access_parms->u.split.raw_access, - op, size, addr_p/*out*/)<0) { - HRETURN_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, - "raw data allocation failed"); - } - *addr_p |= lf->u.split.mask; - lf->eof = lf->u.split.raw->eof; - lf->eof |= lf->u.split.mask; - } - - FUNC_LEAVE(SUCCEED); -} - - -/*------------------------------------------------------------------------- - * Function: H5F_split_alloc - * - * Purpose: Determines if free block BLK in file LF can be used to - * satisfy the request for SIZE bytes. This function is - * actually the same as H5F_low_alloc() except it returns - * failure if the OP is not compatible with the block address, - * insuring that meta data is allocated from one half of the - * address space and raw data from the other half. - * - * Return: Success: Positive if the free block satisfies the - * request exactly, zero if the free block - * over-satisfies the request. The ADDR_P will - * contain the address within the free block - * where the request starts. - * - * Failure: Negative - * - * Programmer: Robb Matzke - * Tuesday, June 9, 1998 - * - * Modifications: - * - *------------------------------------------------------------------------- - */ -static intn -H5F_split_alloc (H5F_low_t *lf, intn op, hsize_t alignment, hsize_t threshold, - hsize_t size, H5MF_free_t *blk, haddr_t *addr_p/*out*/) -{ - intn ret_value = FAIL; - hsize_t wasted; - - FUNC_ENTER (H5F_split_alloc, FAIL); - assert (lf); - assert (alignment>0); - assert (size>0); - assert (blk); - assert (addr_p); - - switch (op) { - case H5MF_META: - if (blk->addr & lf->u.split.mask) HRETURN(FAIL); - break; - case H5MF_RAW: - if (0==(blk->addr & lf->u.split.mask)) HRETURN(FAIL); - break; - } - - if (size>=threshold) { - wasted = blk->addr % alignment; - } else { - wasted = 0; - } - if (0==wasted && size==blk->size) { - /* exact match */ - *addr_p = blk->addr; - ret_value = 1; - } else if (blk->size>wasted && blk->size-wasted>=size) { - /* over-satisfied */ - *addr_p = blk->addr + wasted; - ret_value = 0; - } - - FUNC_LEAVE (ret_value); -} |