Funtools Programming

FunLib: the Funtools Programming Interface

Summary

A description of the Funtools library.

Introduction to the Funtools Programming Interface

To create a Funtools application, you need to include the funtools.h definitions file in your code:

  #include <funtools.h>

You then call Funtools subroutines and functions to access Funtools data. The most important routines are:

FunOpen: open a Funtools file
FunInfoGet: get info about an image or table
FunImageGet: retrieve image data
FunImageRowGet: retrieve image data by row
FunImagePut: output image data
FunImageRowPut: output image data by row
FunColumnSelect: select columns in a table for access
FunTableRowGet: retrieve rows from a table
FunTableRowPut: output rows to a table
FunClose: close a Funtools file

Your program must be linked against the libfuntools.a library, along with the math library. The following libraries also might be required on your system:

-lsocket -lnsl for socket support
-ldl for dynamic loading

For example, on a Solaris system using gcc, use the following link line:

  gcc -o foo foo.c -lfuntools -lsocket -lnsl -ldl -lm

On a Solaris system using Solaris cc, use the following link line:

  gcc -o foo foo.c -lfuntools -lsocket -lnsl -lm

On a Linux system using gcc, use the following link line:

  gcc -o foo foo.c -lfuntools -ldl -lm

Once configure has built a Makefile on your platform, the required "extra" libraries (aside from -lm, which always is required) are specified in that file's EXTRA_LIBS variable. For example, under Linux you will find:

  grep EXTRA_LIBS Makefile
  EXTRA_LIBS      =  -ldl
  ...

The Funtools library contains both the zlib library (http://www.gzip.org/zlib/) and Doug Mink's WCS library (http://tdc-www.harvard.edu/software/wcstools/). It is not necessary to put these libraries on a Funtools link line. Include files necessary for using these libraries are installed in the Funtools include directory.

Funtools Programming Tutorial

The FunOpen() function is used to open a FITS file, an array, or a raw event file:

  /* open the input FITS file for reading */
  ifun = FunOpen(iname, "r", NULL);
  /* open the output FITS file for writing, and connect it to the input file */
  ofun = FunOpen(iname, "w", ifun);

A new output file can inherit header parameters automatically from existing input file by passing the input Funtools handle as the last argument to the new file's FunOpen() call as shown above.

For image data, you then can call FunImageGet() to read an image into memory.

  float buf=NULL;
  /* extract and bin the data section into an image buffer */
  buf = FunImageGet(fun, NULL, "bitpix=-32");

If the (second) buf argument to this call is NULL, buffer space is allocated automatically. The (third) plist argument can be used to specify the return data type of the array. If NULL is specified, the data type of the input file is used.

To process an image buffer, you would generally make a call to FunInfoGet() to determine the dimensions of the image (which may have been changed from the original file dimensions due to Funtools image sectioning on the command line). In a FITS image, the index along the dim1 axis varies most rapidly, followed by the dim2 axis, etc. Thus, to access each pixel in an 2D image, use a double loop such as: buf = FunImageGet(fun, NULL, "bitpix=-32"); FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); for(i=1; i<=dim2; i++){ for(j=1; j<=dim1; j++){ ... process buf[((i-1)*dim1)+(j-1)] ... } } or:

  buf = FunImageGet(fun, NULL, "bitpix=-32");
  FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0);
  for(i=0; i<(dim1*dim2); i++){
    ... process buf[i] ...
  }

Finally, you can write the resulting image to disk using FunImagePut():

  FunImagePut(fun2, buf, dim1, dim2, -32, NULL);

Note that Funtools automatically takes care of book-keeping tasks such as reading and writing FITS headers (although you can, of course, write your own header or add your own parameters to a header).

For binary tables and raw event files, a call to FunOpen() will be followed by a call to the FunColumnSelect() routine to select columns to be read from the input file and/or written to the output file:

  typedef struct evstruct{
    double time;
    int time2;
  } *Ev, EvRec;
  FunColumnSelect(fun, sizeof(EvRec), NULL,
                  "time",      "D",     "rw",  FUN_OFFSET(Ev, time),
                  "time2",     "J",     "w",   FUN_OFFSET(Ev, time2),
                  NULL);

Columns whose (third) mode argument contains an "r" are "readable", i.e., columns will be read from the input file and converted into the data type specified in the call's second argument. These columns values then are stored in the specified offset of the user record structure. Columns whose mode argument contains a "w" are "writable", i.e., these values will be written to the output file. The FunColumnSelect() routine also offers the option of automatically merging user columns with the original input columns when writing the output rows.

Once a set of columns has been specified, you can retrieve rows using FunTableRowGet(), and write the rows using FunTableRowPut():

  Ev ebuf, ev;
  /* get rows -- let routine allocate the array */
  while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* process all rows */
    for(i=0; i<got; i++){
      /* point to the i'th row */
      ev = ebuf+i;
      /* time2 is generated here */
      ev->time2 = (int)(ev->time+.5);
      /* change the input time as well */
      ev->time = -(ev->time/10.0);
    }
    /* write out this batch of rows with the new column */
    FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL);
    /* free row data */
    if( ebuf ) free(ebuf);
  }

The input rows are retrieved into an array of user structs, which can be accessed serially as shown above. Once again, Funtools automatically takes care of book-keeping tasks such as reading and writing FITS headers (although you can, of course, write your own header or add your own parameters to a header).

When all processing is done, you can call FunClose() to close the file(s):

  FunClose(fun2);
  FunClose(fun);

These are the basics of processing FITS files (and arrays or raw event data) using Funtools. The routines in these examples are described in more detail below, along with a few other routines that support parameter access, data flushing, etc.

Compiling and Linking

To create a Funtools application, a software developer will include the funtools.h definitions file in Funtools code:

  #include <funtools.h>

The program is linked against the libfuntools.a library, along with the math library (and the dynamic load library, if the latter is available on your system):

  gcc -o foo foo.c -lfuntools -ldl -lm

If gcc is used, Funtools filtering can be performed using dynamically loaded shared objects that are built at run-time. Otherwise, filtering is performed using a slave process.

Funtools has been built on the following systems:

Sun/Solaris 5.X
Linux/RedHat Linux 5.X,6.X,7.X
Dec Alpha/OSF1 V4.X
WindowsNT/Cygwin 1.0
SGI/IRIX64 6.5

A Short Digression on Subroutine Order

There is a natural order for all I/O access libraries. You would not think of reading a file without first opening it, or writing a file after closing it. A large part of the experiment in funtools is to use the idea of "natural order" as a means of making programming easier. We do this by maintaining the state of processing for a given funtools file, so that we can do things like write headers and flush extension padding at the right time, without you having to do it.

For example, if you open a new funtools file for writing using FunOpen(), then generate an array of image data and call FunImagePut(), funtools knows to write the image header automatically. There is no need to think about writing a standard header. Of course, you can add parameters to the file first by calling one of the FunParamPut() routines, and these parameters will automatically be added to the header when it is written out. There still is no need to write the header explicitly.

Maintaining state in this way means that there are certain rules of order which should be maintained in any funtools program. In particular, we strongly recommend the following ordering rules be adhered to:

When specifying that input extensions be copied to an output file via a reference handle, open the output file before reading the input file. (Otherwise the initial copy will not occur).
Always write parameters to an output file using one of the FunParamPut() calls before writing any data. (This is a good idea for all FITS libraries, to avoid having to recopy data is the FITS header needs to be extended by adding a single parameter.)
If you retrieve an image, and need to know the data type, use the FUN_SECT_BITPIX option of FunInfoGet(), after calling FunImageGet(), since it is possible to change the value of BITPIX from the latter.
When specifying that input extensions be copied to an output file via a reference handle, close the output file before closing input file, or else use FunFlush() explicitly on the output file before closing the input file. (Otherwise the final copy will not occur).

We believe that these are the natural rules that are implied in most FITS programming tasks. However, we recognize that making explicit use of "natural order" to decide what automatic action to take on behalf of the programmer is experimental. Therefore, if you find that your needs are not compatible with our preferred order, please let us know -- it will be most illuminating for us as we evaluate this experiment.

Funtools Programming Examples

The following complete coding examples are provided to illustrate the simplicity of Funtools applications. They can be found in the funtest subdirectory of the Funtools distribution. In many cases, you should be able to modify one of these programs to generate your own Funtools program:

evread.c: read and write binary tables
evcols.c: add column and rows to binary tables
evmerge.c: merge new columns with existing columns
evnext.c: manipulate raw data pointers
imblank.c: blank out image values below a threshold
asc2fits.c: convert a specific ASCII table to FITS binary table

The Funtools Programming Reference Manual

#include <funtools.h>

Fun FunOpen(char *name, char *mode, Fun ref)

void *FunImageGet(Fun fun, void *buf, char *plist)

int FunImagePut(Fun fun, void *buf, int dim1, int dim2, int bitpix, char *plist)

void * FunImageRowGet(Fun fun, void *buf, int rstart, int rstop, char *plist)

void * FunImageRowPut(Fun fun, void *buf, int rstart, int rstop, int dim1, int dim2, int bitpix, char *plist)

int FunColumnSelect(Fun fun, int size, char *plist, ...)

void FunColumnActivate(Fun fun, char *s, char *plist)

int FunColumnLookup(Fun fun, char *s, int which, char **name, int *type, int *mode, int *offset, int *n, int *width)

void *FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, int *nrow)

int FunTableRowPut(Fun fun, void *rows, int nev, int idx, char *plist)

int FunParamGetb(Fun fun, char *name, int n, int defval, int *got)

int FunParamGeti(Fun fun, char *name, int n, int defval, int *got)

double FunParamGetd(Fun fun, char *name, int n, double defval, int *got)

char *FunParamGets(Fun fun, char *name, int n, char *defval, int *got)

int FunParamPutb(Fun fun, char *name, int n, int value, char *comm, int append)

int FunParamPuti(Fun fun, char *name, int n, int value, char *comm, int append)

int FunParamPutd(Fun fun, char *name, int n, double value, int prec, char *comm, int append)

int FunParamPuts(Fun fun, char *name, int n, char *value, char *comm, int append)

int FunInfoGet(Fun fun, int type, ...)

int FunInfoPut(Fun fun, int type, ...)

void FunFlush(Fun fun, char *plist)

void FunClose(Fun fun)

FunOpen - open a Funtools data file

#include <funtools.h> Fun FunOpen(char *name, char *mode, Fun ref);

The FunOpen() routine opens a Funtools data file for reading or appending, or creates a new FITS file for writing. The name argument specifies the name of the Funtools data file to open. You can use IRAF-style bracket notation to specify Funtools Files, Extensions, and Filters. A separate call should be made each time a different FITS extension is accessed:

  Fun fun;
  char *iname;
  ...
  if( !(fun = FunOpen(iname, "r", NULL)) ){
    fprintf(stderr, "could not FunOpen input file: %s\n", iname);
    exit(1);
  }

If mode is "r", the file is opened for reading, and processing is set up to begin at the specified extension. For reading, name can be stdin, in which case the standard input is read.

If mode is "w", the file is created if it does not exist, or opened and truncated for writing if it does exist. Processing starts at the beginning of the file. The name can be stdout, in which case the standard output is readied for processing.

If mode is "a", the file is created if it does not exist, or opened if it does exist. Processing starts at the end of the file. The name can be stdout, in which case the standard output is readied for processing.

When a Funtools file is opened for writing or appending, a previously opened Funtools reference handle can be specified as the third argument. This handle typically is associated with the input Funtools file that will be used to generate the data for the output data. When a reference file is specified in this way, the output file will inherit the (extension) header parameters from the input file:

  Fun fun, fun2;
  ...
  /* open input file */
  if( !(fun = FunOpen(argv[1], "r", NULL)) )
    gerror(stderr, "could not FunOpen input file: %s\n", argv[1]);
  /* open the output FITS image, inheriting params from input */
  if( !(fun2 = FunOpen(argv[2], "w", fun)) )
    gerror(stderr, "could not FunOpen output file: %s\n", argv[2]);

Thus, in the above example, the output FITS binary table file will inherit all of the parameters associated with the input binary table extension.

A file opened for writing with a Funtools reference handle also inherits the selected columns (i.e. those columns chosen for processing using the FunColumnSelect() routine) from the reference file as its default columns. This makes it easy to open an output file in such a way that the columns written to the output file are the same as the columns read in the input file. Of course, column selection can easily be tailored using the FunColumnSelect() routine. In particular, it is easy to merge user-defined columns with the input columns to generate a new file. See the evmerge for a complete example.

In addition, when a Funtools reference handle is supplied in a FunOpen() call, it is possible also to specify that all other extensions from the reference file (other than the input extension being processed) should be copied from the reference file to the output file. This is useful, for example, in a case where you are processing a FITS binary table or image and you want to copy all of the other extensions to the output file as well. Copy of other extensions is controlled by adding a "C" or "c" to the mode string of the FunOpen() call of the input reference file. If "C" is specified, then other extensions are always copied (i.e., copy is forced by the application). If "c" is used, then other extensions are copied if the user requests copying by adding a plus sign "+" to the extension name in the bracket specification. For example, the funtable program utilizes "c" mode, giving users the option of copying all other extensions:

  /* open input file -- allow user copy of other extensions */
  if( !(fun = FunOpen(argv[1], "rc", NULL)) )
    gerror(stderr, "could not FunOpen input file: %s\n", argv[1]);
  /* open the output FITS image, inheriting params from input */
  if( !(fun2 = FunOpen(argv[2], "w", fun)) )
    gerror(stderr, "could not FunOpen output file: %s\n", argv[2]);

Thus, funtable supports either of these command lines:

  # copy only the EVENTS extension
  csh> funtable "test.ev[EVENTS,circle(512,512,10)]" foo.ev
  # copy ALL extensions
  csh> funtable "test.ev[EVENTS+,circle(512,512,10)]" foo.ev

Use of a Funtools reference handle implies that the input file is opened before the output file. However, it is important to note that if copy mode ("c" or "C") is specified for the input file, the actual input file open is delayed until just after the output file is opened, since the copy of prior extensions to the output file takes place while Funtools is seeking to the specified input extension. This implies that the output file should be opened before any I/O is done on the input file or else the copy will fail. Note also that the copy of subsequent extension will be handled automatically by FunClose() if the output file is closed before the input file. Alternatively, it can be done explicitly by FunFlush(), but again, this assumes that the input file still is open.

Upon success FunOpen() returns a Fun handle that is used in subsequent Funtools calls. On error, NULL is returned.

FunImageGet - get an image or image section

#include <funtools.h> void *FunImageGet(Fun fun, void *buf, char *plist)

The FunImageGet() routine returns an binned image array of the specified section of a Funtools data file. If the input data are already of type image, the array is generated by extracting the specified image section and then binning it according to the specified bin factor. If the input data are contained in a binary table or raw event file, the rows are binned on the columns specified by the bincols= keyword (using appropriate default columns as necessary), after which the image section and bin factors are applied. In both cases, the data is automatically converted from FITS to native format, if necessary.

The first argument is the Funtools handle returned by FunOpen(). The second buf argument is a pointer to a data buffer to fill. If NULL is specified, FunImageGet will allocate a buffer of the appropriate size. Generally speaking, you always want Funtools to allocate the buffer because the image dimensions will be determined by Funtools image sectioning on the command line.

The third plist (i.e., parameter list) argument is a string containing one or more comma-delimited keyword=value parameters. It can be used to specify the return data type using the bitpix= keyword. If no such keyword is specified in the plist string, the data type of the returned image is the same as the data type of the original input file, or is of type int for FITS binary tables.

If the bitpix= keyword is supplied in the plist string, the data type of the returned image will be one of the supported FITS image data types:

8 unsigned char
16 short
32 int
-32 float
-64 double

For example:

  void *buf;
  /* extract data section into an image buffer */
  if( !(buf = FunImageGet(fun, NULL, NULL)) )
    gerror(stderr, "could not FunImageGet: %s\n", iname);

will allocate buf and retrieve the image in the file data format. In this case, you will have to determine the data type (using the FUN_SECT_BITPIX value in the FunInfoGet() routine) and then use a switch statement to process each data type:

  int bitpix;
  void *buf;
  unsigned char *cbuf;
  short *sbuf;
  int *ibuf;
  ...
  buf = FunImageGet(fun, NULL, NULL);
  FunInfoGet(fun, FUN_SECT_BITPIX,  &bitpix, 0);
  /* set appropriate data type buffer to point to image buffer */
  switch(bitpix){
  case 8:
    cbuf = (unsigned char *)buf; break;
  case 16:
    sbuf = (short *)buf; break;
  case 32:
    ibuf = (int *)buf; break;
 ...

See the imblank example code for more details on how to process an image when the data type is not specified beforehand.

It often is easier to specify the data type directly:

  double *buf;
  /* extract data section into a double image buffer */
  if( !(buf = FunImageGet(fun, NULL, "bitpix=-64")) )
    gerror(stderr, "could not FunImageGet: %s\n", iname);

will extract the image while converting to type double.

On success, a pointer to the image buffer is returned. (This will be the same as the second argument, if NULL is not passed to the latter.) On error, NULL is returned.

In summary, to retrieve image or row data into a binned image, you simply call FunOpen() followed by FunImageGet(). Generally, you then will want to call FunInfoGet() to retrieve the axis dimensions (and data type) of the section you are processing (so as to take account of sectioning and blocking of the original data):

  double *buf;
  int i, j;
  int dim1, dim2;
  ... other declarations, etc.

  /* open the input FITS file */
  if( !(fun = FunOpen(argv[1], "rc", NULL)) )
    gerror(stderr, "could not FunOpen input file: %s\n", argv[1]);

  /* extract and bin the data section into a double float image buffer */
  if( !(buf = FunImageGet(fun, NULL, "bitpix=-64")) )
    gerror(stderr, "could not FunImageGet: %s\n", argv[1]);

  /* get dimension information from funtools structure */
  FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0);

  /* loop through pixels and reset values below limit to value */
  for(i=0; i<dim1*dim2; i++){
    if( buf[i] <= blimit ) buf[i] = bvalue;
  }

Another useful plist string value is "mask=all", which returns an image populated with regions id values. Image pixels within a region will contain the associated region id (region values start at 1), and otherwise will contain a 0 value. Thus, the returned image is a region mask which can be used to process the image data (which presumably is retrieved by a separate call to FunImageGet) pixel by pixel.

If a FITS binary table or a non-FITS raw event file is being binned into an image, it is necessary to specify the two columns that will be used in the 2D binning. This usually is done on the command line using the bincols=(x,y) keyword:

  funcnts "foo.ev[EVENTS,bincols=(detx,dety)]"

The full form of the bincols= specifier is:

  bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]])

where the tlmin, tlmax, and binsiz specifiers determine the image binning dimensions:

  dim = (tlmax - tlmin)/binsiz     (floating point data)
  dim = (tlmax - tlmin)/binsiz + 1 (integer data)

These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, TLMAX, and TDBIN header parameters (respectively) are present in the FITS binary table header for the column in question. Note that if only one parameter is specified, it is assumed to be tlmax, and tlmin defaults to 1. If two parameters are specified, they are assumed to be tlmin and tlmax.

If bincols is not specified on the command line, Funtools tries to use appropriate defaults: it looks for the environment variable FITS_BINCOLS (or FITS_BINKEY). Then it looks for the Chandra parameters CPREF (or PREFX) in the FITS binary table header. Failing this, it looks for columns named "X" and "Y" and if these are not found, it looks for columns containing the characters "X" and "Y".

See Binning FITS Binary Tables and Non-FITS Event Files for more information.

FunImagePut - put an image to a Funtools file

#include <funtools.h> int FunImagePut(Fun fun, void *buf, int dim1, int dim2, int bitpix, char *plist)
The FunImagePut() routine outputs an image array to a FITS file. The image is written either as a primary header/data unit or as an image extension, depending on whether other data have already been written to the file. That is, if the current file position is at the beginning of the file, a primary HDU is written. Otherwise, an image extension is written.

The first argument is the Funtools handle returned by FunOpen(). The second buf argument is a pointer to a data buffer to write. The dim1and dim2 arguments that follow specify the dimensions of the image, where dim1 corresponds to naxis1 and dim2 corresponds to naxis2. The bitpix argument specifies the data type of the image and can have the following FITS-standard values:

8 unsigned char
16 short
32 int
-32 float
-64 double

When FunTableRowPut() is first called for a given image, Funtools checks to see if the primary header has already been written (by having previously written an image or a binary table.) If not, this image is written to the primary HDU. Otherwise, it is written to an image extension.

Thus, a simple program to generate a FITS image might look like this:

  int i;
  int dim1=512, dim2=512;
  double *dbuf;
  Fun fun;
  dbuf = malloc(dim1*dim2*sizeof(double));
  /* open the output FITS image, preparing to copy input params */
  if( !(fun = FunOpen(argv[1], "w", NULL)) )
    gerror(stderr, "could not FunOpen output file: %s\n", argv[1]);
  for(i=0; i<(dim1*dim2); i++){
    ... fill dbuf ...
  }
  /* put the image (header will be generated automatically */
  if( !FunImagePut(fun, buf, dim1, dim2, -64, NULL) )
    gerror(stderr, "could not FunImagePut: %s\n", argv[1]);
  FunClose(fun);
  free(dbuf);

In addition, if a Funtools reference handle was specified when this table was opened, the parameters from this Funtools reference handle are merged into the new image header. Furthermore, if a reference image was specified during FunOpen(), the values of dim1, dim2, and bitpix in the calling sequence can all be set to 0. In this case, default values are taken from the reference image section. This is useful if you are reading an image section in its native data format, processing it, and then writing that section to a new FITS file. See the imblank example code.

The data are assumed to be in the native machine format and will automatically be swapped to FITS big-endian format if necessary. This behavior can be over-ridden with the convert=[true|false] keyword in the plist param list string.

When you are finished writing the image, you should call FunFlush() to write out the FITS image padding. However, this is not necessary if you subsequently call FunClose() without doing any other I/O to the FITS file.

FunImageRowGet - get row(s) of an image

#include <funtools.h> void *FunImageRowGet(Fun fun, void *buf, int rstart, int rstop, char *plist)

The FunImageRowGet() routine returns one or more image rows from the specified section of a Funtools data file. If the input data are of type image, the array is generated by extracting the specified image rows and then binning them according to the specified bin factor. If the input data are contained in a binary table or raw event file, the rows are binned on the columns specified by the bincols= keyword (using appropriate default columns as needed), after which the image section and bin factors are applied.

The first argument is the Funtools handle returned by FunOpen(). The second buf argument is a pointer to a data buffer to fill. If NULL is specified, FunImageGet() will allocate a buffer of the appropriate size.

The third and fourth arguments specify the first and last row to retrieve. Rows are counted starting from 1, up to the value of FUN_YMAX(fun). The final plist (i.e., parameter list) argument is a string containing one or more comma-delimited keyword=value parameters. It can be used to specify the return data type using the bitpix= keyword. If no such keyword is specified in the plist string, the data type of the image is the same as the data type of the original input file, or is of type int for FITS binary tables.

If the bitpix=value is supplied in the plist string, the data type of the returned image will be one of the supported FITS image data types:

8 unsigned char
16 short
32 int
-32 float
-64 double

For example:

  double *drow;
  Fun fun;
  ... open files ...
  /* get section dimensions */
  FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0);
  /* allocate one line's worth */
  drow = malloc(dim1*sizeof(double));
  /* retrieve and process each input row (starting at 1) */
  for(i=1; i <= dim2; i++){
    if( !FunImageRowGet(fun, drow, i, i, "bitpix=-64") )
      gerror(stderr, "can't FunImageRowGet: %d %s\n", i, iname);
      /* reverse the line */
      for(j=1; j<=dim1; j++){
        ... process drow[j-1] ...
      }
  }
  ...

On success, a pointer to the image buffer is returned. (This will be the same as the second argument, if NULL is not passed to the latter.) On error, NULL is returned. Note that the considerations described above for specifying binning columns in FunImageGet() also apply to FunImageRowGet().

FunImageRowPut - put row(s) of an image

#include <funtools.h> void *FunImageRowPut(Fun fun, void *buf, int rstart, int rstop, int dim1, int dim2, int bitpix, char *plist)

The FunImageRowPut() routine writes one or more image rows to the specified FITS image file. The first argument is the Funtools handle returned by FunOpen(). The second buf argument is a pointer to the row data buffer, while the third and fourth arguments specify the starting and ending rows to write. Valid rows values range from 1 to dim2, i.e., row is one-valued.

The dim1and dim2 arguments that follow specify the dimensions, where dim1 corresponds to naxis1 and dim2 corresponds to naxis2. The bitpix argument data type of the image and can have the following FITS-standard values:

8 unsigned char
16 short
32 int
-32 float
-64 double

For example:

  double *drow;
  Fun fun, fun2;
  ... open files ...
  /* get section dimensions */
  FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0);
  /* allocate one line's worth */
  drow = malloc(dim1*sizeof(double));
  /* retrieve and process each input row (starting at 1) */
  for(i=1; i <= dim2; i++){
    if( !FunImageRowGet(fun, drow, i, i, "bitpix=-64") )
      gerror(stderr, "can't FunImageRowGet: %d %s\n", i, iname);
    ... process drow ...
    if( !FunImageRowPut(fun2, drow, i, i, 64, NULL) )
      gerror(stderr, "can't FunImageRowPut: %d %s\n", i, oname);
  }
  ...

The data are assumed to be in the native machine format and will automatically be swapped to big-endian FITS format if necessary. This behavior can be over-ridden with the convert=[true|false] keyword in the plist param list string.

FunColumnSelect - select Funtools columns

#include <funtools.h> int FunColumnSelect(Fun fun, int size, char *plist, char *name1, char *type1, char *mode1, int offset1, char *name2, char *type2, char *mode2, int offset2, ..., NULL) int FunColumnSelectArr(Fun fun, int size, char *plist, char **names, char **types, char **modes, int *offsets, int nargs);
The FunColumnSelect() routine is used to select the columns from a Funtools binary table extension or raw event file for processing. This routine allows you to specify how columns in a file are to be read into a user record structure or written from a user record structure to an output FITS file.

The first argument is the Fun handle associated with this set of columns. The second argument specifies the size of the user record structure into which columns will be read. Typically, the sizeof() macro is used to specify the size of a record structure. The third argument allows you to specify keyword directives for the selection and is described in more detail below.

Following the first three required arguments is a variable length list of column specifications. Each column specification will consist of four arguments:

name: the name of the column
type: the data type of the column as it will be stored in the user record struct (not the data type of the input file). The following basic data types are recognized:
- A: ASCII characters
- B: unsigned 8-bit char
- I: signed 16-bit int
- U: unsigned 16-bit int (not standard FITS)
- J: signed 32-bit int
- V: unsigned 32-bit int (not standard FITS)
- E: 32-bit float
- D: 64-bit float
The syntax used is similar to that which defines the TFORM parameter in FITS binary tables. That is, a numeric repeat value can precede the type character, so that "10I" means a vector of 10 short ints, "E" means a single precision float, etc. Note that the column value from the input file will be converted to the specified data type as the data is read by FunTableRowGet().
[ A short digression regarding bit-fields: Special attention is required when reading or writing the FITS bit-field type ("X"). Bit-fields almost always have a numeric repeat character preceding the 'X' specification. Usually this value is a multiple of 8 so that bit-fields fit into an integral number of bytes. For all cases, the byte size of the bit-field B is (N+7)/8, where N is the numeric repeat character.
A bit-field is most easily declared in the user struct as an array of type char of size B as defined above. In this case, bytes are simply moved from the file to the user space. If, instead, a short or int scalar or array is used, then the algorithm for reading the bit-field into the user space depends on the size of the data type used along with the value of the repeat character. That is, if the user data size is equal to the byte size of the bit-field, then the data is simply moved (possibly with endian-based byte-swapping) from one to the other. If, on the other hand, the data storage is larger than the bit-field size, then a data type cast conversion is performed to move parts of the bit-field into elements of the array. Examples will help make this clear:
- If the file contains a 16X bit-field and user space specifies a 2B char array[2], then the bit-field is moved directly into the char array.
- If the file contains a 16X bit-field and user space specifies a 1I scalar short int, then the bit-field is moved directly into the short int.
- If the file contains a 16X bit-field and user space specifies a 1J scalar int, then the bit-field is type-cast to unsigned int before being moved (use of unsigned avoids possible sign extension).
- If the file contains a 16X bit-field and user space specifies a 2J int array[2], then the bit-field is handled as 2 chars, each of which are type-cast to unsigned int before being moved (use of unsigned avoids possible sign extension).
- If the file contains a 16X bit-field and user space specifies a 1B char, then the bit-field is treated as a char, i.e., truncation will occur.
- If the file contains a 16X bit-field and user space specifies a 4J int array[4], then the results are undetermined.
For all user data types larger than char, the bit-field is byte-swapped as necessary to convert to native format, so that bits in the resulting data in user space can be tested, masked, etc. in the same way regardless of platform.]
In addition to setting data type and size, the type specification allows a few ancillary parameters to be set, using the full syntax for type:
```
 [@][n]<type>[[['B']poff]][:[tlmin[:tlmax[:binsiz]]]]
```
The special character "@" can be prepended to this specification to indicated that the data element is a pointer in the user record, rather than an array stored within the record.
The [n] value is an integer that specifies the number of elements that are in this column (default is 1). TLMIN, TLMAX, and BINSIZ values also can be specified for this column after the type, separated by colons. If only one such number is specified, it is assumed to be TLMAX, and TLMIN and BINSIZ are set to 1.
The [poff] value can be used to specify the offset into an array. By default, this offset value is set to zero and the data specified starts at the beginning of the array. The offset usually is specified in terms of the data type of the column. Thus an offset specification of [5] means a 20-byte offset if the data type is a 32-bit integer, and a 40-byte offset for a double. If you want to specify a byte offset instead of an offset tied to the column data type, precede the offset value with 'B', e.g. [B6] means a 6-bye offset, regardless of the column data type. The [poff] is especially useful in conjunction with the pointer @ specification, since it allows the data element to anywhere stored anywhere in the allocated array. For example, a specification such as "@I[2]" specifies the third (i.e., starting from 0) element in the array pointed to by the pointer value. A value of "@2I[4]" specifies the fifth and sixth values in the array. For example, consider the following specification:
```
  typedef struct EvStruct{
    short x[4], *atp;
  } *Event, EventRec;
  /* set up the (hardwired) columns */
  FunColumnSelect( fun, sizeof(EventRec), NULL,
                   "2i",    "2I  ",    "w", FUN_OFFSET(Event, x),
                   "2i2",   "2I[2]",   "w", FUN_OFFSET(Event, x),
                   "at2p",  "@2I",     "w", FUN_OFFSET(Event, atp),
                   "at2p4", "@2I[4]",  "w", FUN_OFFSET(Event, atp),
                   "atp9",  "@I[9]",   "w", FUN_OFFSET(Event, atp),
                   "atb20", "@I[B20]", "w", FUN_OFFSET(Event, atb),
                   NULL);
```
Here we have specified the following columns:
- 2i: two short ints in an array which is stored as part the record
- 2i2: the 3rd and 4th elements of an array which is stored as part of the record
- an array of at least 10 elements, not stored in the record but allocated elsewhere, and used by three different columns:
  - at2p: 2 short ints which are the first 2 elements of the allocated array
  - at2p4: 2 short ints which are the 5th and 6th elements of the allocated array
  - atp9: a short int which is the 10th element of the allocated array
- atb20: a short int which is at byte offset 20 of another allocated array
In this way, several columns can be specified, all of which are in a single array. NB: it is the programmer's responsibility to ensure that specification of a positive value for poff does not point past the end of valid data.
read/write mode: "r" means that the column is read from an input file into user space by FunTableRowGet(), "w" means that the column is written to an output file. Both can specified at the same time.
offset: the offset into the user data to store this column. Typically, the macro FUN_OFFSET(recname, colname) is used to define the offset into a record structure.

When all column arguments have been specified, a final NULL argument must added to signal the column selection list.

As an alternative to the varargs FunColumnSelect() routine, a non-varargs routine called FunColumnSelectArr() also is available. The first three arguments (fun, size, plist) of this routine are the same as in FunColumnSelect(). Instead of a variable argument list, however, FunColumnSelectArr() takes 5 additional arguments. The first 4 arrays arguments contain the names, types, modes, and offsets, respectively, of the columns being selected. The final argument is the number of columns that are contained in these arrays. It is the user's responsibility to free string space allocated in these arrays.

Consider the following example:

  typedef struct evstruct{
    int status;
    float pi, pha, *phas;
    double energy;
  } *Ev, EvRec;

  FunColumnSelect(fun, sizeof(EvRec), NULL,
    "status",  "J",     "r",   FUN_OFFSET(Ev, status),
    "pi",      "E",     "r",  FUN_OFFSET(Ev, pi),
    "pha",     "E",     "r",  FUN_OFFSET(Ev, pha),
    "phas",    "@9E",   "r",  FUN_OFFSET(Ev, phas),
    NULL);

Each time a row is read into the Ev struct, the "status" column is converted to an int data type (regardless of its data type in the file) and stored in the status value of the struct. Similarly, "pi" and "pha", and the phas vector are all stored as floats. Note that the "@" sign indicates that the "phas" vector is a pointer to a 9 element array, rather than an array allocated in the struct itself. The row record can then be processed as required:

  /* get rows -- let routine allocate the row array */
  while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* process all rows */
    for(i=0; i<got; i++){
      /* point to the i'th row */
      ev = ebuf+i;
      ev->pi = (ev->pi+.5);
      ev->pha = (ev->pi-.5);
    }

FunColumnSelect() can also be called to define "writable" columns in order to generate a FITS Binary Table, without reference to any input columns. For example, the following will generate a 4-column FITS binary table when FunTableRowPut() is used to write Ev records:

  typedef struct evstruct{
    int status;
    float pi, pha
    double energy;
  } *Ev, EvRec;

  FunColumnSelect(fun, sizeof(EvRec), NULL,
    "status",  "J",     "w",   FUN_OFFSET(Ev, status),
    "pi",      "E",     "w",   FUN_OFFSET(Ev, pi),
    "pha",     "E",     "w",   FUN_OFFSET(Ev, pha),
    "energy",  "D",       "w",   FUN_OFFSET(Ev, energy),
    NULL);

All columns are declared to be write-only, so presumably the column data is being generated or read from some other source.

In addition, FunColumnSelect() can be called to define both "readable" and "writable" columns. In this case, the "read" columns are associated with an input file, while the "write" columns are associated with the output file. Of course, columns can be specified as both "readable" and "writable", in which case they are read from input and (possibly modified data values are) written to the output. The FunColumnSelect() call itself is made by passing the input Funtools handle, and it is assumed that the output file has been opened using this input handle as its Funtools reference handle.

Consider the following example:

  typedef struct evstruct{
    int status;
    float pi, pha, *phas;
    double energy;
  } *Ev, EvRec;

  FunColumnSelect(fun, sizeof(EvRec), NULL,
    "status",  "J",     "r",   FUN_OFFSET(Ev, status),
    "pi",      "E",     "rw",  FUN_OFFSET(Ev, pi),
    "pha",     "E",     "rw",  FUN_OFFSET(Ev, pha),
    "phas",    "@9E",   "rw",  FUN_OFFSET(Ev, phas),
    "energy",  "D",     "w",   FUN_OFFSET(Ev, energy),
    NULL);

As in the "read" example above, each time an row is read into the Ev struct, the "status" column is converted to an int data type (regardless of its data type in the file) and stored in the status value of the struct. Similarly, "pi" and "pha", and the phas vector are all stored as floats. Since the "pi", "pha", and "phas" variables are declared as "writable" as well as "readable", they also will be written to the output file. Note, however, that the "status" variable is declared as "readable" only, and hence it will not be written to an output file. Finally, the "energy" column is declared as "writable" only, meaning it will not be read from the input file. In this case, it can be assumed that "energy" will be calculated in the program before being output along with the other values.

In these simple cases, only the columns specified as "writable" will be output using FunTableRowPut(). However, it often is the case that you want to merge the user columns back in with the input columns, even in cases where not all of the input column names are explicitly read or even known. For this important case, the merge=[type] keyword is provided in the plist string.

The merge=[type] keyword tells Funtools to merge the columns from the input file with user columns on output. It is normally used when an input and output file are opened and the input file provides the Funtools reference handle for the output file. In this case, each time FunTableRowGet() is called, the raw input rows are saved in a special buffer. If FunTableRowPut() then is called (before another call to FunTableRowGet()), the contents of the raw input rows are merged with the user rows according to the value of type as follows:

update: add new user columns, and update value of existing ones (maintaining the input data type)
replace: add new user columns, and replace the data type and value of existing ones. (Note that if tlmin/tlmax values are not specified in the replacing column, but are specified in the original column being replaced, then the original tlmin/tlmax values are used in the replacing column.)
append: only add new columns, do not "replace" or "update" existing ones

Consider the example above. If merge=update is specified in the plist string, then "energy" will be added to the input columns, and the values of "pi", "pha", and "phas" will be taken from the user space (i.e., the values will be updated from the original values, if they were changed by the program). The data type for "pi", "pha", and "phas" will be the same as in the original file. If merge=replace is specified, both the data type and value of these three input columns will be changed to the data type and value in the user structure. If merge=append is specified, none of these three columns will be updated, and only the "energy" column will be added. Note that in all cases, "status" will be written from the input data, not from the user record, since it was specified as read-only.

Standard applications will call FunColumnSelect() to define user columns. However, if this routine is not called, the default behavior is to transfer all input columns into user space. For this purpose a default record structure is defined such that each data element is properly aligned on a valid data type boundary. This mechanism is used by programs such as fundisp and funtable to process columns without needing to know the specific names of those columns. It is not anticipated that users will need such capabilities (contact us if you do!)

By default, FunColumnSelect() reads/writes rows to/from an "array of structs", where each struct contains the column values for a single row of the table. This means that the returned values for a given column are not contiguous. You can set up the IO to return a "struct of arrays" so that each of the returned columns are contiguous by specifying org=structofarrays (abbreviation: org=soa) in the plist. (The default case is org=arrayofstructs or org=aos.)

For example, the default setup to retrieve rows from a table would be to define a record structure for a single event and then call FunColumnSelect() as follows:

  typedef struct evstruct{
    short region;
    double x, y;
    int pi, pha;
    double time;
  } *Ev, EvRec;

  got = FunColumnSelect(fun, sizeof(EvRec), NULL,
                        "x",       "D:10:10", mode, FUN_OFFSET(Ev, x),
                        "y",       "D:10:10", mode, FUN_OFFSET(Ev, y),
                        "pi",      "J",       mode, FUN_OFFSET(Ev, pi),
                        "pha",     "J",       mode, FUN_OFFSET(Ev, pha),
                        "time",    "1D",      mode, FUN_OFFSET(Ev, time),
                        NULL);

Subsequently, each call to FunTableRowGet() will return an array of structs, one for each returned row. If instead you wanted to read columns into contiguous arrays, you specify org=soa:

  typedef struct aevstruct{
    short region[MAXROW];
    double x[MAXROW], y[MAXROW];
    int pi[MAXROW], pha[MAXROW];
    double time[MAXROW];
  } *AEv, AEvRec;

  got = FunColumnSelect(fun, sizeof(AEvRec), "org=soa",
                      "x",       "D:10:10", mode, FUN_OFFSET(AEv, x),
                      "y",       "D:10:10", mode, FUN_OFFSET(AEv, y),
                      "pi",      "J",       mode, FUN_OFFSET(AEv, pi),
                      "pha",     "J",       mode, FUN_OFFSET(AEv, pha),
                      "time",    "1D",      mode, FUN_OFFSET(AEv, time),
                      NULL);

Note that the only modification to the call is in the plist string.

Of course, instead of using staticly allocated arrays, you also can specify dynamically allocated pointers:

  /* pointers to arrays of columns (used in struct of arrays) */
  typedef struct pevstruct{
    short *region;
    double *x, *y;
    int *pi, *pha;
    double *time;
  } *PEv, PEvRec;

  got = FunColumnSelect(fun, sizeof(PEvRec), "org=structofarrays",
                      "$region", "@I",       mode, FUN_OFFSET(PEv, region),
                      "x",       "@D:10:10", mode, FUN_OFFSET(PEv, x),
                      "y",       "@D:10:10", mode, FUN_OFFSET(PEv, y),
                      "pi",      "@J",       mode, FUN_OFFSET(PEv, pi),
                      "pha",     "@J",       mode, FUN_OFFSET(PEv, pha),
                      "time",    "@1D",      mode, FUN_OFFSET(PEv, time),
                      NULL);

Here, the actual storage space is either allocated by the user or by the FunColumnSelect() call).

In all of the above cases, the same call is made to retrieve rows, e.g.:

    buf = (void *)FunTableRowGet(fun, NULL, MAXROW, NULL, &got);

However, the individual data elements are accessed differently. For the default case of an "array of structs", the individual row records are accessed using:

  for(i=0; i<got; i++){
    ev = (Ev)buf+i;
    fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n",
            ev->x, ev->y, ev->pi, ev->pha, ev->dx, ev->dy, ev->time);
  }

For a struct of arrays or a struct of array pointers, we have a single struct through which we access individual columns and rows using:

  aev = (AEv)buf;
  for(i=0; i<got; i++){
    fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n",
            aev->x[i], aev->y[i], aev->pi[i], aev->pha[i], 
            aev->dx[i], aev->dy[i], aev->time[i]);
  }

Support for struct of arrays in the FunTableRowPut() call is handled analogously.

See the evread example code and evmerge example code for working examples of how FunColumnSelect() is used.

FunColumnActivate - activate Funtools columns

#include <funtools.h> void FunColumnActivate(Fun fun, char *s, char *plist)

The FunColumnActivate() routine determines which columns (set up by FunColumnSelect()) ultimately will be read and/or written. By default, all columns that are selected using FunColumnSelect() are activated. The FunColumnActivate() routine can be used to turn off/off activation of specific columns.

The first argument is the Fun handle associated with this set of columns. The second argument is a space-delimited list of columns to activate or de-activate. Columns preceded by "+" are activated and columns preceded by a "-" are de-activated. If a column is named without "+" or "-", it is activated. The reserved strings "$region" and '$n' are used to activate a special columns containing the filter region value and row value, respectively, associated with this row. For example, if a filter containing two circular regions is specified as part of the Funtools file name, this column will contain a value of 1 or 2, depending on which region that row was in. The reserved strings "$x" and "$y" are used to activate the current binning columns. Thus, if the columns DX and DY are specified as binning columns:

  [sh $] fundisp foo.fits[bincols=(DX,DY)]

then "$x" and "$y" will refer to these columns in a call to FunColumnActivate().

In addition, if the activation string contains only columns to be activated, then the routine will de-activate all other columns. Similarly, if the activation string contains only columns to de-activate, then the routine will activate all other columns before activating the list. This makes it simple to change the activation state of all columns without having to know all of the column names. For example:

"pi pha time" # only these three columns will be active
"-pi -pha -time" # all but these columns will be active
"pi -pha" # only pi is active, pha is not, others are not
"+pi -pha" # same as above
"pi -pha -time" # only pi is active, all others are not
"pi pha" # pha and pi are active, all others are not
"pi pha -x -y" # pha and pi are active, all others are not

You can use the column activation list to reorder columns, since columns are output in the order specified. For example:

  # default output order
  fundisp snr.ev'[cir 512 512 .1]' 
         X        Y      PHA       PI                  TIME       DX       DY
  -------- -------- -------- -------- --------------------- -------- --------
       512      512        6        7     79493997.45854475      578      574
       512      512        8        9     79494575.58943175      579      573
       512      512        5        6     79493631.03866175      578      575
       512      512        5        5     79493290.86521725      578      575
       512      512        8        9     79493432.00990875      579      573

  # re-order the output by specifying explicit order
  fundisp snr.ev'[cir 512 512 .1]' "time x y dy dx pi pha"
                   TIME        X        Y       DY       DX       PI      PHA
  --------------------- -------- -------- -------- -------- -------- --------
      79493997.45854475      512      512      574      578        7        6
      79494575.58943175      512      512      573      579        9        8
      79493631.03866175      512      512      575      578        6        5
      79493290.86521725      512      512      575      578        5        5
      79493432.00990875      512      512      573      579        9        8

A "+" sign by itself means to activate all columns, so that you can reorder just a few columns without specifying all of them:

  # reorder 3 columns and then output the rest
  fundisp snr.ev'[cir 512 512 .1]' "time pi pha +"
                   TIME       PI      PHA        Y        X       DX       DY
  --------------------- -------- -------- -------- -------- -------- --------
      79493997.45854475        7        6      512      512      578      574
      79494575.58943175        9        8      512      512      579      573
      79493631.03866175        6        5      512      512      578      575
      79493290.86521725        5        5      512      512      578      575
      79493432.00990875        9        8      512      512      579      573

The column activation/deactivation is performed in the order of the specified column arguments. This means you can mix "+", "-" (which de-activates all columns) and specific column names to reorder and select columns in one command. For example, consider the following:

  # reorder and de-activate
  fundisp snr.ev'[cir 512 512 .1]' "time pi pha + -x -y"
                   TIME       PI      PHA       DX       DY
  --------------------- -------- -------- -------- --------
      79493997.45854475        7        6      578      574
      79494575.58943175        9        8      579      573
      79493631.03866175        6        5      578      575
      79493290.86521725        5        5      578      575
      79493432.00990875        9        8      579      573

We first activate "time", "pi", and "pha" so that they are output first. We then activate all of the other columns, and then de-activate "x" and "y". Note that this is different from:

  # probably not what you want ...
  fundisp snr.ev'[cir 512 512 .1]' "time pi pha -x -y +"
                   TIME       PI      PHA        Y        X       DX       DY
  --------------------- -------- -------- -------- -------- -------- --------
      79493997.45854475        7        6      512      512      578      574
      79494575.58943175        9        8      512      512      579      573
      79493631.03866175        6        5      512      512      578      575
      79493290.86521725        5        5      512      512      578      575
      79493432.00990875        9        8      512      512      579      573

Here, "x" and "y" are de-activated, but then all columns including "x" and "y" are again re-activated.

Typically, FunColumnActivate() uses a list of columns that are passed into the program from the command line. For example, the code for funtable contains the following:

  char *cols=NULL;

  /* open the input FITS file */
  if( !(fun = FunOpen(argv[1], "rc", NULL)) )
    gerror(stderr, "could not FunOpen input file: %s\n", argv[1]);

  /* set active flag for specified columns */
  if( argc >= 4 ) cols = argv[3];
  FunColumnActivate(fun, cols, NULL);

The FunOpen() call sets the default columns to be all columns in the input file. The FunColumnActivate() call then allows the user to control which columns ultimately will be activated (i.e., in this case, written to the new file). For example:

  funtable test.ev foo.ev "pi pha time"

will process only the three columns mentioned, while:

  funtable test.ev foo.ev "-time"

will process all columns except "time".

If FunColumnActivate() is called with a null string, then the environment variable FUN_COLUMNS will be used to provide a global value, if present. This is the reason why we call the routine even if no columns are specified on the command line (see example above), instead of calling it this way:

  /* set active flag for specified columns */
  if( argc >= 4 ){
    FunColumnActivate(fun, argv[3], NULL);
  }

FunColumnLookup - lookup a Funtools column

#include <funtools.h> int FunColumnLookup(Fun fun, char *s, int which, char **name, int *type, int *mode, int *offset, int *n, int *width)

The FunColumnLookup() routine returns information about a named (or indexed) column. The first argument is the Fun handle associated with this set of columns. The second argument is the name of the column to look up. If the name argument is NULL, the argument that follows is the zero-based index into the column array of the column for which information should be returned. The next argument is a pointer to a char *, which will contain the name of the column. The arguments that follow are the addresses of int values into which the following information will be returned:

type: data type of column:
- A: ASCII characters
- B: unsigned 8-bit char
- I: signed 16-bit int
- U: unsigned 16-bit int (not standard FITS)
- J: signed 32-bit int
- V: unsigned 32-bit int (not standard FITS)
- E: 32-bit float
- D: 64-bit float
mode: bit flag status of column, including:
- COL_ACTIVE 1 is column activated?
- COL_IBUF 2 is column in the raw input data?
- COL_PTR 4 is column a pointer to an array?
- COL_READ 010 is read mode selected?
- COL_WRITE 020 is write mode selected?
- COL_REPLACEME 040 is this column being replaced by user data?
offset: byte offset in struct
n: number of elements (i.e. size of vector) in this column
width: size in bytes of this column

If the named column exists, the routine returns a positive integer, otherwise zero is returned. (The positive integer is the index+1 into the column array where this column was located.) If NULL is passed as the return address of one (or more) of these values, no data is passed back for that information. For example:

  if( !FunColumnLookup(fun, "phas", 0, NULL NULL, NULL, NULL, &npha, NULL) )
    gerror(stderr, "can't find phas column\n");

only returns information about the size of the phas vector.

FunTableRowGet - get Funtools rows

#include <funtools.h> void *FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, int *nrow)

The FunTableRowGet() routine retrieves rows from a Funtools binary table or raw event file, and places the values of columns selected by FunColumnSelect() into an array of user structs. Selected column values are automatically converted to the specified user data type (and to native data format) as necessary.

The first argument is the Fun handle associated with this row data. The second rows argument is the array of user structs into which the selected columns will be stored. If NULL is passed, the routine will automatically allocate space for this array. (This includes proper allocation of pointers within each struct, if the "@" pointer type is used in the selection of columns. Note that if you pass NULL in the second argument, you should free this space using the standard free() system call when you are finished with the array of rows.) The third maxrow argument specifies the maximum number of rows to be returned. Thus, if rows is allocated by the user, it should be at least of size maxrow*sizeof(evstruct).

The fourth plist argument is a param list string. Currently, the keyword/value pair "mask=transparent" is supported in the plist argument. If this string is passed in the call's plist argument, then all rows are passed back to the user (instead of just rows passing the filter). This is only useful when FunColumnSelect() also is used to specify "$region" as a column to return for each row. In such a case, rows found within a region have a returned region value greater than 0 (corresponding to the region id of the region in which they are located), rows passing the filter but not in a region have region value of -1, and rows not passing any filter have region value of 0. Thus, using "mask=transparent" and the returned region value, a program can process all rows and decide on an action based on whether a given row passed the filter or not.

The final argument is a pointer to an int variable that will return the actual number of rows returned. The routine returns a pointer to the array of stored rows, or NULL if there was an error. (This pointer will be the same as the second argument, if the latter is non-NULL).

  /* get rows -- let routine allocate the row array */
  while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* process all rows */
    for(i=0; i<got; i++){
      /* point to the i'th row */
      ev = buf+i;
      /* rearrange some values. etc. */
      ev->energy = (ev->pi+ev->pha)/2.0;
      ev->pha = -ev->pha;
      ev->pi = -ev->pi;
    }
    /* write out this batch of rows */
    FunTableRowPut(fun2, buf, got, 0, NULL);
    /* free row data */
    if( buf ) free(buf);
  }

As shown above, successive calls to FunTableRowGet() will return the next set of rows from the input file until all rows have been read, i.e., the routine behaves like sequential Unix I/O calls such as fread(). See evmerge example code for a more complete example.

Note that FunTableRowGet() also can be called as FunEventsGet(), for backward compatibility.

FunTableRowPut - put Funtools rows


int FunTableRowPut(Fun fun, void *rows, int nev, int idx, char *plist)

The FunTableRowPut() routine writes rows to a FITS binary table, taking its input from an array of user structs that contain column values selected by a previous call to FunColumnSelect(). Selected column values are automatically converted from native data format to FITS data format as necessary.

The first argument is the Fun handle associated with this row data. The second rows argument is the array of user structs to output. The third nrow argument specifies the number number of rows to write. The routine will write nrow records, starting from the location specified by rows.

The fourth idx argument is the index of the first raw input row to write, in the case where rows from the user buffer are being merged with their raw input row counterparts (see below). Note that this idx value is has nothing to do with the row buffer specified in argument 1. It merely matches the row being written with its corresponding (hidden) raw row. Thus, if you read a number of rows, process them, and then write them out all at once starting from the first user row, the value of idx should be 0:

  Ev ebuf, ev;
  /* get rows -- let routine allocate the row array */
  while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* process all rows */
    for(i=0; i<got; i++){
      /* point to the i'th row */
      ev = ebuf+i;
      ...
    }
    /* write out this batch of rows, starting with the first */
    FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL);
    /* free row data */
    if( ebuf ) free(ebuf);
  }

On the other hand, if you write out the rows one at a time (possibly skipping rows), then, when writing the i'th row from the input array of rows, set idx to the value of i:

  Ev ebuf, ev;
  /* get rows -- let routine allocate the row array */
  while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* process all rows */
    for(i=0; i<got; i++){
      /* point to the i'th row */
      ev = ebuf+i;
      ...
      /* write out the current (i.e., i'th) row */
      FunTableRowPut(fun2, (char *)ev, 1, i, NULL);
    }
    /* free row data */
    if( ebuf ) free(ebuf);
  }

The final argument is a param list string that is not currently used. The routine returns the number of rows output. This should be equal to the value passed in the third nrow When FunTableRowPut() is first called for a given binary table, Funtools checks to see of the primary header has already been written (either by writing a previous row table or by writing an image.) If not, a dummy primary header is written to the file specifying that an extension should be expected. After this, a binary table header is automatically written containing information about the columns that will populate this table. In addition, if a Funtools reference handle was specified when this table was opened, the parameters from this Funtools reference handle are merged into the new binary table header.

In a typical Funtools row loop, you read rows using FunTableRowGet()() and write rows using FunTableRowPut(). The columns written by FunTableRowPut()() are those defined as writable by a previous call to FunColumnSelect(). If that call to FunColumnSelect also specified merge=[update|replace|append], then the entire corresponding raw input row record will be merged with the output row according to the merge specification (see FunColumnSelect() above).

A call to write rows can either be done once, after all rows in the input batch have been processed, or it can be done (slightly less efficiently) one row at a time (or anything in between). We do recommend that you write all rows associated with a given batch of input rows before reading new rows. This is required if you are merging the output rows with the raw input rows (since the raw rows are destroyed with each successive call to get new rows). For example:

  Ev buf, ev;
  ...
  /* get rows -- let routine allocate the row array */
  while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* point to the i'th row */
    ev = buf + i;
    .... process
  }
  /* write out this batch of rows */
  FunTableRowPut(fun2, buf, got, 0, NULL);
  /* free row data */
  if( buf ) free(buf);
  }

  Ev buf, ev;
  ...
  /* get rows -- let routine allocate the row array */
  while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
    /* process all rows */
    for(i=0; i<got; i++){
      /* point to the i'th row */
      ev = buf + i;
      ... process
      /* write out this batch of rows with the new column */
      if( dowrite )
        FunTableRowPut(fun2, buf, 1, i, NULL);
    }
    /* free row data */
    if( buf ) free(buf);
  }

Note that the difference between these calls is that the first one outputs got rows all at once and therefore passes idx=0 in argument four, so that merging starts at the first raw input row. In the second case, a check it made on each row to see if it needs to be output. If so, the value of idx is passed as the value of the i variable which points to the current row being processed in the batch of input rows.

As shown above, successive calls to FunTableRowPut() will write rows sequentially. When you are finished writing all rows in a table, you should call FunFlush() to write out the FITS binary table padding. However, this is not necessary if you subsequently call FunClose() without doing any other I/O to the FITS file.

Note that FunTableRowPut() also can be called as FunEventsPut(), for backward compatibility.

FunParamGet - get a Funtools param value

#include <funtools.h> int FunParamGetb(Fun fun, char *name, int n, int defval, int *got) int FunParamGeti(Fun fun, char *name, int n, int defval, int *got) double FunParamGetd(Fun fun, char *name, int n, double defval, int *got) char *FunParamGets(Fun fun, char *name, int n, char *defval, int *got)

The four routines FunParamGetb(), FunParamGeti(), FunParamGetd(), and FunParamGets(), return the value of a FITS header parameter as a boolean, int, double, and string, respectively. The string returned by FunParamGets() is a malloc'ed copy of the header value and should be freed when no longer needed.

The first argument is the Fun handle associated with the FITS header being accessed. Normally, the header is associated with the FITS extension that you opened with FunOpen(). However, you can use FunInfoPut() to specify access of the primary header. In particular, if you set the FUN_PRIMARYHEADER parameter to 1, then the primary header is used for all parameter access until the value is reset to 0. For example:

  int val;
  FunParamGeti(fun, "NAXIS", 1, 0, &got);              # current header
  val=1;
  FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0);         # switch to ...
  FunParamGeti(fun, "NAXIS", 1, 0, &got);              # ... primary header
  FunParamGeti(fun, "NAXIS", 2, 0, &got);              # ... primary header
  val=0;
  FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0);         # switch back to ...
  FunParamGeti(fun, "NAXIS", 2, 0, &got);              # current header

Alternatively, you can use the FUN_PRIMARY macro to access parameters from the primary header on a per-parameter basis:

  FunParamGeti(fun, "NAXIS1", 0, 0, &got);              # current header
  FunParamGeti(FUN_PRIMARY(fun), "NAXIS1", 0, 0, &got); # primary header

NB: FUN_PRIMARY is deprecated. It makes use of a global parameter and therefore will not not appropriate for threaded applications, when we make funtools thread-safe. We recommend use of FunInfoPut() to switch between the extension header and the primary header.

For output data, access to the primary header is only possible until the header is written out, which usually takes place when the first data are written.

The second argument is the name of the parameter to access. The third n argument, if non-zero, is an integer that will be added as a suffix to the parameter name. This makes it easy to use a simple loop to process parameters having the same root name. For example, to gather up all values of TLMIN and TLMAX for each column in a binary table, you can use:

  for(i=0, got=1; got; i++){
    fun->cols[i]->tlmin = (int)FunParamGeti(fun, "TLMIN", i+1, 0.0, &got);
    fun->cols[i]->tlmax = (int)FunParamGeti(fun, "TLMAX", i+1, 0.0, &got);
  }

The fourth defval argument is the default value to return if the parameter does not exist. Note that the data type of this parameter is different for each specific FunParamGet() call. The final got argument will be 0 if no param was found. Otherwise the data type of the parameter is returned as follows: FUN_PAR_UNKNOWN ('u'), FUN_PAR_COMMENT ('c'), FUN_PAR_LOGICAL ('l'), FUN_PAR_INTEGER ('i'), FUN_PAR_STRING ('s'), FUN_PAR_REAL ('r'), FUN_PAR_COMPLEX ('x').

These routines return the value of the header parameter, or the specified default value if the header parameter does not exist. The returned value is a malloc'ed string and should be freed when no longer needed.

By default, FunParamGets() returns the string value of the named parameter. However, you can use FunInfoPut() to retrieve the raw 80-character FITS card instead. In particular, if you set the FUN_RAWPARAM parameter to 1, then card images will be returned by FunParamGets() until the value is reset to 0.

Alternatively, if the FUN_RAW macro is applied to the name, then the 80-character raw FITS card is returned instead. NB: FUN_RAW is deprecated. It makes use of a global parameter and therefore will not not appropriate for threaded applications, when we make funtools thread-safe. We recommend use of FunInfoPut() to switch between the extension header and the primary header.

Note that in addition to the behaviors described above, the routine FunParamGets() will return the 80 raw characters of the nth FITS card (including the comment) if name is specified as NULL and n is positive. For example, to loop through all FITS header cards in a given extension and print out the raw card, use:

  for(i=1; ;i++){
    if( (s = FunParamGets(fun, NULL, i, NULL, &got)) ){
      fprintf(stdout, "%.80s\n", s);
      free(s);
    }
    else{
      break;
    }
  }

FunParamPut - put a Funtools param value

#include <funtools.h> int FunParamPutb(Fun fun, char *name, int n, int value, char *comm, int append) int FunParamPuti(Fun fun, char *name, int n, int value, char *comm, int append) int FunParamPutd(Fun fun, char *name, int n, double value, int prec, char *comm, int append) int FunParamPuts(Fun fun, char *name, int n, char *value, char *comm, int append)

The four routines FunParamPutb(), FunParamPuti(), FunParamPutd(), and FunParamPuts(), will set the value of a FITS header parameter as a boolean, int, double, and string, respectively.

The first argument is the Fun handle associated with the FITS header being accessed. Normally, the header is associated with the FITS extension that you opened with FunOpen(). However, you can use FunInfoPut() to specify that use of the primary header. In particular, if you set the FUN_PRIMARYHEADER parameter to 1, then the primary header is used for all parameter access until the value is reset to 0. For example:

  int val;
  FunParamPuti(fun, "NAXIS1", 0, 10, NULL, 1);       # current header
  val=1;
  FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0);       # switch to ...
  FunParamPuti(fun, "NAXIS1", 0, 10, NULL, 1);       # primary header

(You also can use the deprecated FUN_PRIMARY macro, to access parameters from the primary header.)

The second argument is the name of the parameter. ( In accordance with FITS standards, the special names COMMENT and HISTORY, as well as blank names, are output without the "= " value indicator in columns 9 and 10.

The third n argument, if non-zero, is an integer that will be added as a suffix to the parameter name. This makes it easy to use a simple loop to process parameters having the same root name. For example, to set the values of TLMIN and TLMAX for each column in a binary table, you can use:

  for(i=0; i<got; i++){
    FunParamPutd(fun, "TLMIN", i+1, tlmin[i], 7, "min column val", 1);
    FunParamPutd(fun, "TLMAX", i+1, tlmax[i], 7, "max column val", 1);
  }

The fourth defval argument is the value to set. Note that the data type of this argument is different for each specific FunParamPut() call. The comm argument is the comment string to add to this header parameter. Its value can be NULL. The final append argument determines whether the parameter is added to the header if it does not exist. If set to a non-zero value, the header parameter will be appended to the header if it does not exist. If set to 0, the value will only be used to change an existing parameter.

Note that the double precision routine FunParamPutd() supports an extra prec argument after the value argument, in order to specify the precision when converting the double value to ASCII. In general a 20.[prec] format is used (since 20 characters are alloted to a floating point number in FITS) as follows: if the double value being put to the header is less than 0.1 or greater than or equal to 10**(20-2-[prec]), then %20.[prec]e format is used (i.e., scientific notation); otherwise %20.[prec]f format is used (i.e., numeric notation).

As a rule, parameters should be set before writing the table or image. It is, however, possible to update the value of an existing parameter after writing an image or table (but not to add a new one). Such updating only works if the parameter already exists and if the output file is seekable, i.e. if it is a disk file or is stdout being redirected to a disk file.

It is possible to add a new parameter to a header after the data has been written, but only if space has previously been reserved. To reserve space, add a blank parameter whose value is the name of the parameter you eventually will update. Then, when writing the new parameter, specify a value of 2 for the append flag. The parameter writing routine will first look to update an existing parameter, as usual. If an existing parameter is not found, an appropriately-valued blank parameter will be searched for and replaced. For example:

  /* add blank card to be used as a place holder for IPAR1 update */
  FunParamPuts(fun, NULL, 0, "IPAR1", "INTEGER Param", 0);
  ...
  /* write header and data */
  FunTableRowPut(fun, events, got, 0, NULL);
  ...
  /* update param in file after writing data -- note append = 2 here */
  FunParamPuti(fun, "IPAR", 1, 400, "INTEGER Param", 2);

The parameter routines return a 1 if the routine was successful and a 0 on failure. In general, the major reason for failure is that you did not set the append argument to a non-zero value and the parameter did not already exist in the file.

FunInfoGet - get information from Funtools struct

#include <funtools.h> int FunInfoGet(Fun fun, int type, char *addr, ...)

The FunInfoGet() routine returns information culled from the Funtools structure. The first argument is the Fun handle from which information is to be retrieved. This first required argument is followed by a variable length list of pairs of arguments. Each pair consists of an integer representing the type of information to retrieve and the address where the information is to be stored. The list is terminated by a 0. The routine returns the number of get actions performed.

The full list of available information is described below. Please note that only a few of these will be useful to most application developers. For imaging applications, the most important types are:

  FUN_SECT_DIM1   int  /* dim1 for section */
  FUN_SECT_DIM2   int  /* dim2 for section */
  FUN_SECT_BITPIX int  /* bitpix for section */

These would be used to determine the dimensions and data type of image data retrieved using the FunImageGet() routine. For example:

  /* extract and bin the data section into an image buffer */
  buf = FunImageGet(fun, NULL, NULL);
  /* get required information from funtools structure.
     this should come after the FunImageGet() call, in case the call
     changed sect_bitpix */
  FunInfoGet(fun,
             FUN_SECT_BITPIX,  &bitpix,
             FUN_SECT_DIM1,    &dim1,
             FUN_SECT_DIM2,    &dim2,
             0);
  /* loop through pixels and reset values below limit to value */
  for(i=0; i<dim1*dim2; i++){
    switch(bitpix){
    case 8:
      if( cbuf[i] <= blimit ) cbuf[i] = bvalue;
    ...
  }

It is important to bear in mind that the call to FunImageGet() can change the value of FUN_SECT_BITPIX (e.g. if "bitpix=n" is passed in the param list). Therefore, a call to FunInfoGet() should be made after the call to FunImageGet(), in order to retrieve the updated bitpix value. See the imblank example code for more details.

It also can be useful to retrieve the World Coordinate System information from the Funtools structure. Funtools uses the the WCS Library developed by Doug Mink at SAO, which is available here. (More information about the WCSTools project in general can be found here.) The FunOpen() routine initializes two WCS structures that can be used with this WCS Library. Applications can retrieve either of these two WCS structures using FunInfoGet():

  FUN_WCS  struct WorldCoor * /* wcs structure, for image coordinates*/
  FUN_WCS0 struct WorldCoor * /* wcs structure, for physical coordinates */

The structure retrieved by FUN_WCS is a WCS library handle containing parameters suitable for use with image coordinates, regardless of whether the data are images or tables. For this structure, the WCS reference point (CRPIX) has been converted to image coordinates if the underlying file is a table (and therefore in physical coordinates). You therefore must ensure that the positions being passed to a routine like pix2wcs are in image coordinates. The FUN_WCS0 structure has not had its WCS reference point converted to image coordinates. It therefore is useful when passing processing physical coordinates from a table.

Once a WCS structure has been retrieved, it can be used as the first argument to the WCS library routines. (If the structure is NULL, no WCS information was contained in the file.) The two important WCS routines that Funtools uses are:

  #include <wcs.h>
  void pix2wcs (wcs,xpix,ypix,xpos,ypos)
    struct WorldCoor *wcs; /* World coordinate system structure */
    double xpix,ypix;      /* x and y coordinates in pixels */
    double *xpos,*ypos;    /* RA and Dec in degrees (returned) */

which converts pixel coordinates to sky coordinates, and:

  void wcs2pix (wcs, xpos, ypos, xpix, ypix, offscl)
    struct WorldCoor *wcs; /* World coordinate system structure */
    double xpos,ypos;      /* World coordinates in degrees */
    double *xpix,*ypix;    /* coordinates in pixels */
    int *offscl;           /* 0 if within bounds, else off scale */

which converts sky coordinates to pixel coordinates. Again, please note that the wcs structure returned by FUN_WCS assumes that image coordinates are passed to the pix2wcs routine, while FUN_WCS0 assumes that physical coordinates are passed.

Note that funtools.h file automatically includes wcs.h. An example program that utilizes these WCS structure to call WCS Library routines is twcs.c.

The following is the complete list of information that can be returned:

  name            type            comment
  ---------       --------        ---------------------------------------------
  FUN_FNAME     char *            /* file name */
  FUN_GIO       GIO               /* gio handle */
  FUN_HEADER    FITSHead          /* fitsy header struct */
  FUN_TYPE      int               /* TY_TABLE,TY_IMAGE,TY_EVENTS,TY_ARRAY */
  FUN_BITPIX    int               /* bits/pixel in file */
  FUN_MIN1      int               /* tlmin of axis1 -- tables */
  FUN_MAX1      int               /* tlmax of axis1 -- tables */
  FUN_MIN2      int               /* tlmin of axis2 -- tables */
  FUN_MAX2      int               /* tlmax of axis2 -- tables */
  FUN_DIM1      int               /* dimension of axis1 */
  FUN_DIM2      int               /* dimension of axis2 */
  FUN_ENDIAN    int               /* 0=little, 1=big endian */
  FUN_FILTER    char *            /* supplied filter */
  FUN_IFUN      FITSHead          /* pointer to reference header */
  FUN_IFUN0     FITSHead          /* same as above, but no reset performed */
  /* image information */
  FUN_DTYPE     int               /* data type for images */
  FUN_DLEN      int               /* length of image in bytes */
  FUN_DPAD      int               /* padding to end of extension */
  FUN_DOBLANK   int               /* was blank keyword defined? */
  FUN_BLANK     int               /* value for blank */
  FUN_SCALED    int               /* was bscale/bzero defined? */
  FUN_BSCALE    double            /* bscale value */
  FUN_BZERO     double            /* bzero value */
  /* table information */
  FUN_NROWS     int               /* number of rows in file (naxis2) */
  FUN_ROWSIZE   int               /* size of user row struct */
  FUN_BINCOLS   char *            /* specified binning columns */
  FUN_OVERFLOW  int               /* overflow detected during binning? */
  /* array information */
  FUN_SKIP      int               /* bytes to skip in array header */
  /* section information */
  FUN_SECT_X0   int               /* low dim1 value of section */
  FUN_SECT_X1   int               /* hi dim1 value of section */
  FUN_SECT_Y0   int               /* low dim2 value of section */
  FUN_SECT_Y1   int               /* hi dim2 value of section */
  FUN_SECT_BLOCK int              /* section block factor */
  FUN_SECT_BTYPE int              /* 's' (sum), 'a' (average) for binning */
  FUN_SECT_DIM1 int               /* dim1 for section */
  FUN_SECT_DIM2 int               /* dim2 for section */
  FUN_SECT_BITPIX int             /* bitpix for section */
  FUN_SECT_DTYPE int              /* data type for section */
  FUN_RAWBUF    char *            /* pointer to raw row buffer */
  FUN_RAWSIZE   int               /* byte size of raw row records */
  /* column  information */
  FUN_NCOL      int               /* number of row columns defined */
  FUN_COLS      FunCol            /* array of row columns */
  /* WCS information */
  FUN_WCS       struct WorldCoor * /* wcs structure, converted for images*/
  FUN_WCS0      struct WorldCoor * /* wcs structure, not converted */

Row applications would not normally need any of this information. An example of how these values can be used in more complex programs is the evnext example code. In this program, the time value for each row is changed to be the value of the succeeding row. The program thus reads the time values for a batch of rows, changes the time values to be the value for the succeeding row, and then merges these changed time values back with the other columns to the output file. It then reads the next batch, etc.

This does not work for the last row read in each batch, since there is no succeeding row until the next batch is read. Therefore, the program saves that last row until it has read the next batch, then processes the former before starting on the new batch. In order to merge the last row successfully, the code uses FUN_RAWBUF to save and restore the raw input data associated with each batch of rows. Clearly, this requires some information about how funtools works internally. We are happy to help you write such programs as the need arises.

FunInfoPut - put information into a Funtools struct

#include <funtools.h> int FunInfoPut(Fun fun, int type, char *addr, ...)

The FunInfoPut() routine puts information into a Funtools structure. The first argument is the Fun handle from which information is to be retrieved. After this first required argument comes a variable length list of pairs of arguments. Each pair consists of an integer representing the type of information to store and the address of the new information to store in the struct. The variable list is terminated by a 0. The routine returns the number of put actions performed.

The full list of available information is described above with the FunInfoPut() routine. Although use of this routine is expected to be uncommon, there is one important situation in which it plays an essential part: writing multiple extensions to a single output file.

For input, multiple extensions are handled by calling FunOpen() for each extension to be processed. When opening multiple inputs, it sometimes is the case that you will want to process them and then write them (including their header parameters) to a single output file. To accomplish this, you open successive input extensions using FunOpen() and then call FunInfoPut() to set the Funtools reference handle of the output file to that of the newly opened input extension:

  /* open a new input extension */
  ifun=FunOpen(tbuf, "r", NULL)) )
  /* make the new extension the reference handle for the output file */
  FunInfoPut(ofun, FUN_IFUN, &ifun, 0);

Resetting FUN_IFUN has same effect as when a funtools handle is passed as the final argument to FunOpen(). The state of the output file is reset so that a new extension is ready to be written. Thus, the next I/O call on the output extension will output the header, as expected.

For example, in a binary table, after resetting FUN_IFUN you can then call FunColumnSelect() to select the columns for output. When you then call FunImagePut() or FunTableRowPut(), a new extension will be written that contains the header parameters from the reference extension. Remember to call FunFlush() to complete output of a given extension.

A complete example of this capability is given in the evcol example code. The central algorithm is:

open the output file without a reference handle
loop: open each input extension in turn
- set the reference handle for output to the newly opened input extension
- read the input rows or image and perform processing
- write new rows or image to the output file
- flush the output
- close input extension
close output file

Note that FunFlush() is called after processing each input extension in order to ensure that the proper padding is written to the output file. A call to FunFlush() also ensures that the extension header is written to the output file in the case where there are no rows to output.

If you wish to output a new extension without using a Funtools reference handle, you can call FunInfoPut() to reset the FUN_OPS value directly. For a binary table, you would then call FunColumnSelect() to set up the columns for this new extension.

  /* reset the operations performed on this handle */
  int ops=0;
  FunInfoPut(ofun, FUN_OPS, &ops, 0);
  FunColumnSelect(fun, sizeof(EvRec), NULL,
                  "MYCOL", "J", "w", FUN_OFFSET(Ev, mycol),
                  NULL);

Once the FUN_OPS variable has been reset, the next I/O call on the output extension will output the header, as expected.

FunFlush - flush data to output file

#include <funtools.h> void FunFlush(Fun fun, char *plist)

The FunFlush routine will flush data to a FITS output file. In particular, it can be called after all rows have been written (using the FunTableRowPut() routine) in order to add the null padding that is required to complete a FITS block. It also should be called after completely writing an image using FunImagePut() or after writing the final row of an image using FunTableRowPut().

The plist (i.e., parameter list) argument is a string containing one or more comma-delimited keyword=value parameters. If the plist string contains the parameter "copy=remainder" and the file was opened with a reference file, which, in turn, was opened for extension copying (i.e. the input FunOpen() mode also was "c" or "C"), then FunFlush also will copy the remainder of the FITS extensions from the input reference file to the output file. This normally would be done only at the end of processing.

Note that FunFlush() is called with "copy=remainder" in the mode string by FunClose(). This means that if you close the output file before the reference input file, it is not necessary to call FunFlush() explicitly, unless you are writing more than one extension. See the evmerge example code. However, it is safe to call FunFlush() more than once without fear of re-writing either the padding or the copied extensions.

In addition, if FunFlush() is called on an output file with the plist set to "copy=reference" and if the file was opened with a reference file, the reference extension is written to the output file. This mechanism provides a simple way to copy input extensions to an output file without processing the former. For example, in the code fragment below, an input extension is set to be the reference file for a newly opened output extension. If that reference extension is not a binary table, it is written to the output file:

  /* process each input extension in turn */
  for(ext=0; ;ext++){
    /* get new extension name */
    sprintf(tbuf, "%s[%d]", argv[1], ext);
    /* open input extension -- if we cannot open it, we are done */
    if( !(ifun=FunOpen(tbuf, "r", NULL)) )
      break;
    /* make the new extension the reference handle for the output file */
    FunInfoPut(ofun, FUN_IFUN, &ifun, 0);
    /* if its not a binary table, just write it out */
    if( !(s=FunParamGets(ifun, "XTENSION", 0, NULL, &got)) || 
      strcmp(s, "BINTABLE")){
      if( s ) free(s);
      FunFlush(ofun, "copy=reference");
      FunClose(ifun);
      continue;
    }
    else{
      /* process binary table */
      ....
    }
  }

FunClose - close a Funtools data file

#include <funtools.h> void FunClose(Fun fun)

The FunClose() routine closes a previously-opened Funtools data file, freeing control structures. If a Funtools reference handle was passed to the FunOpen() call for this file, and if copy mode also was specified for that file, then FunClose() also will copy the remaining extensions from the input file to the output file (if the input file still is open). Thus, we recommend always closing the output Funtools file before the input file. (Alternatively, you can call FunFlush() explicitly).

FunRef: the Funtools Reference Handle

Summary

A description of how to use a Funtools reference handle to connect a Funtools input file to an output file.

Description

The Funtools reference handle connects a Funtools input file to a Funtools output file so that parameters (or even whole extensions) can be copied from the one to the other. To make the connection, the Funtools handle of the input file is passed to the final argument of the FunOpen() call for the output file:

  if( !(ifun = FunOpen(argv[1], "r", NULL)) )
    gerror(stderr, "could not FunOpen input file: %s\n", argv[1]);
  if( !(ofun = FunOpen(argv[2], "w", ifun)) )
    gerror(stderr, "could not FunOpen output file: %s\n", argv[2]);

It does not matter what type of input or output file (or extension) is opened, or whether they are the same type. When the output image or binary table is written using FunImagePut() or FunTableRowPut() an appropriate header will be written first, with parameters copied from the input extension. Of course, invalid parameters will be removed first, e.g., if the input is a binary table and the output is an image, then binary table parameters such as TFORM, TUNIT, etc. parameters will not be copied to the output.

Use of a reference handle also allows default values to be passed to FunImagePut() in order to write out an output image with the same dimensions and data type as the input image. To use the defaults from the input, a value of 0 is entered for dim1, dim2, and bitpix. For example:

  fun = FunOpen(argv[1], "r", NULL);
  fun2 = FunOpen(argv[2], "w", fun);
  buf = FunImageGet(fun, NULL, NULL);
  ... process image data ...
  FunImagePut(fun2, buf, 0, 0, 0, NULL);

Of course, you often want to get information about the data type and dimensions of the image for processing. The above code is equivalent to the following:

  fun = FunOpen(argv[1], "r", NULL);
  fun2 = FunOpen(argv[2], "w", fun);
  buf = FunImageGet(fun, NULL, NULL);
  FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 
             FUN_SECT_BITPIX, &bitpix, 0);
  ... process image data ...
  FunImagePut(fun2, buf, dim1, dim2, bitpix, NULL);

It is possible to change the reference handle for a given output Funtools handle using the FunInfoPut() routine:

  /* make the new extension the reference handle for the output file */
  FunInfoPut(fun2, FUN_IFUN, &fun, 0);

When this is done, Funtools specially resets the output file to start a new output extension, which is connected to the new input reference handle. You can use this mechanism to process multiple input extensions into a single output file, by successively opening the former and setting the reference handle for the latter. For example:

  /* open a new output FITS file */
  if( !(fun2 = FunOpen(argv[2], "w", NULL)) )
    gerror(stderr, "could not FunOpen output file: %s\n", argv[2]);
  /* process each input extension in turn */
  for(ext=0; ;ext++){
    /* get new extension name */
    sprintf(tbuf, "%s[%d]", argv[1], ext);
    /* open it -- if we cannot open it, we are done */
    if( !(fun=FunOpen(tbuf, "r", NULL)) )
      break;
    /* make the new extension the reference handle for the output file */
    FunInfoPut(fun2, FUN_IFUN, &fun, 0);
    ... process ...
    /* flush output extension (write padding, etc.) */
    FunFlush(fun2, NULL);
    /* close the input extension */
    FunClose(fun);
  }

In this example, the output file is opened first. Then each successive input extension is opened, and the output reference handle is set to the newly opened input handle. After data processing is performed, the output extension is flushed and the input extension is closed, in preparation for the next input extension.

Finally, a reference handle can be used to copy other extensions from the input file to the output file. Copy of other extensions is controlled by adding a "C" or "c" to the mode string of the FunOpen() call of the input reference file. If "C" is specified, then other extensions are always copied (i.e., copy is forced by the application). If "c" is used, then other extensions are copied if the user requests copying by adding a plus sign "+" to the extension name in the bracket specification. For example, the funtable program utilizes user-specified "c" mode so that the second example below will copy all extensions:

  # copy only the EVENTS extension
  csh> funtable "test.ev[EVENTS,circle(512,512,10)]" foo.ev
  # copy ALL extensions
  csh> funtable "test.ev[EVENTS+,circle(512,512,10)]" foo.ev

When extension copy is specified in the input file, the call to FunOpen() on the input file delays the actual file open until the output file also is opened (or until I/O is performed on the input file, which ever happens first). Then, when the output file is opened, the input file is also opened and input extensions are copied to the output file, up to the specific extension being opened. Processing of input and output extensions then proceed.

When extension processing is complete, the remaining extensions need to be copied from input to output. This can be done explicitly, using the FunFlush() call with the "copy=remaining" plist:

  FunFlush(fun, "copy=remaining");

Alternatively, this will happen automatically, if the output file is closed before the input file:

  /* we could explicitly flush remaining extensions that need copying */
  /* FunFlush(fun2, "copy=remaining"); */
  /* but if we close output before input, end flush is done automatically  */
  FunClose(fun2);
  FunClose(fun);

Go to Funtools Help Index

Summary

Summary

Description

Last updated: December 1, 2005