diff options
| author | William Joye <wjoye@cfa.harvard.edu> | 2016-10-27 17:38:41 (GMT) |
|---|---|---|
| committer | William Joye <wjoye@cfa.harvard.edu> | 2016-10-27 17:38:41 (GMT) |
| commit | 5b44fb0d6530c4ff66a446afb69933aa8ffd014f (patch) | |
| tree | e059f66d1f612e21fe9d83f9620c8715530353ec /funtools/doc | |
| parent | da2e3d212171bbe64c1af39114fd067308656990 (diff) | |
| parent | 23c7930d27fe11c4655e1291a07a821dbbaba78d (diff) | |
| download | blt-5b44fb0d6530c4ff66a446afb69933aa8ffd014f.zip blt-5b44fb0d6530c4ff66a446afb69933aa8ffd014f.tar.gz blt-5b44fb0d6530c4ff66a446afb69933aa8ffd014f.tar.bz2 | |
Merge commit '23c7930d27fe11c4655e1291a07a821dbbaba78d' as 'funtools'
Diffstat (limited to 'funtools/doc')
90 files changed, 41661 insertions, 0 deletions
diff --git a/funtools/doc/asc2fits.c b/funtools/doc/asc2fits.c new file mode 100644 index 0000000..5be9a9e --- /dev/null +++ b/funtools/doc/asc2fits.c @@ -0,0 +1,86 @@ +/* + * + * asc2fits foo.fits < foo.ascii + * + * This is an example of generating a binary table from specific ASCII input. + * The more general case is much harder. + * + */ +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <funtools.h> + +#define SZ_LINE 1024 +#define MAXREC 30 + +typedef struct EvStruct{ + int x, y, pha; + double time; +} *Event, EventRec; + +int main(int argc, char **argv) +{ + int got, put; + char tbuf[SZ_LINE]; + Fun fun; + EventRec events[MAXREC]; + Event ev; + + /* exit on gio errors */ + setgerror(2); + + if( argc < 2 ){ + fprintf( stderr, "usage: %s oname\n", argv[0]); + exit(1); + } + + /* open output file */ + if( !(fun = FunOpen(argv[1],"w", NULL)) ) + gerror(stderr, "Could not open the output file: %s\n", argv[1]); + + /* set up the (hardwired) columns */ + FunColumnSelect( fun, sizeof(EventRec), NULL, + "x", "J", "w", FUN_OFFSET(Event, x), + "y", "J", "w", FUN_OFFSET(Event, y), + "pha", "J", "w", FUN_OFFSET(Event, pha), + "time", "D", "w", FUN_OFFSET(Event, time), + NULL); + + /* ignore first line, which is the header */ + fgets(tbuf, SZ_LINE, stdin); + + /* process data lines */ + got = 0; + /* get next line */ + while( fgets(tbuf, SZ_LINE, stdin) ){ + /* ignore comments */ + if( (*tbuf == '\n') || (*tbuf == '#') ) + continue; + /* point to next buffer to fill */ + ev = &events[got]; + /* parse data record */ + if(sscanf(tbuf, "%d %d %d %lf", &ev->x, &ev->y, &ev->pha, &ev->time) != 4) + break; + /* got another record */ + got++; + /* flush this batch of records if necessary */ + if( got == MAXREC ){ + if( (put=FunTableRowPut(fun, events, got, 0, NULL)) != got ){ + gerror(stderr, "expected to write %d rows; only wrote %d\n", + got, put); + } + /* reset record counter */ + got = 0; + } + } + /* final flush */ + if( got ){ + if( (put=FunTableRowPut(fun, events, got, 0, NULL)) != got ){ + gerror(stderr, "expected to write %d rows; only wrote %d\n", + got, put); + } + } + FunClose(fun); + return(0); +} diff --git a/funtools/doc/changelog.html b/funtools/doc/changelog.html new file mode 100644 index 0000000..a4049c8 --- /dev/null +++ b/funtools/doc/changelog.html @@ -0,0 +1,1182 @@ +<HTML> +<HEAD> +<TITLE>Funtools ChangeLog</TITLE> +</HEAD> +<BODY> +<H2>Funtools ChangeLog</H2> + +<P> +This ChangeLog covers both the Funtools library and the suite of +applications. It will be updated as we continue to develop and improve +Funtools. The up-to-date version can be found +<A HREF="http://hea-www.harvard.edu/RD/funtools/changelog.html">here</A>. +[The changelog for the initial development of Funtools, covering the +beta releases, can be found +<A HREF="http://hea-www.harvard.edu/RD/funtools/changelog_beta.html">here</A>.] + +<H2> Patch Release 1.4.6 (internal ds9 release)</H2> +<ul> +<p> +<li> Upgrade wcssubs to version 3.8.7 +<p> +<li> Port to mingw (Windows) +</ul> + +<H2> Patch Release 1.4.5 (internal ds9 release)</H2> +<ul> +<p> +<li> Fixed bug in funim.c which broke vcol functionality. +<p> +<li> Removed permission checking from Find() on cygwin systems. This was broken +by Windows 7. +<p> +<li> Removed addition of -no-cpp-precomp flag from gcc 4.2 and beyond. +</ul> + +<H2> Patch Release 1.4.4 (internal ds9 release)</H2> +<ul> +<p> +<li> Fixed -J funcone, which was not properly outputting all rows. +<p> +<li> Filter: when an image is flipped, the rotation angle must be reversed. +<p> +<li> Filter: add support for windows-based ipc communication when using tcc +compiler. +</ul> + +<H2> Patch Release 1.4.3 (internal ds9 release)</H2> +<ul> +<p> +<li> Filter: improve checks for existence of compiler, even if CC is set. +<p> +<li> Change launch.h to xlaunch.h to avoid conflict with OS X. +<p> +<li> handle flipped images in filtering +</ul> + +<H2> Patch Release 1.4.2 (internal ds9 release)</H2> +<ul> +<p> +<li> Port to gcc 4.2. +<p> +<li> Fix 1-byte filters on intel machines (missing SW1 no-op). +<p> +<li> Remove ambiguity from filt.l (and calc.l) using [A-z] in a +case-insensitive lexer. +<p> +<li> In funsky, the default unit for RA was changed from hours to degrees. +<p> +<li> Fixed bug in funtable in which TCRVL header values were output as strings. +<p> +<li> Added support for running funtools filters in Rosetta (i.e. running PPC +executables on an Intel Mac) by sensing and handling data swap requirements. +Only works with FILTER_PTYPE set to 'c' (can't link against wrong architecture +libraries). +<p> +<li> Fixed bug in FITS library to allow "-" in extension names. +<p> +<li> Code and documentation now agree that the copy extension specifier ('+') +comes after the extension name. +</ul> + +<H2> Patch Release 1.4.1 (internal ds9 release)</H2> +<ul> +<p> +<li> Modified internal Launch() routine to use posix_spawn(), if necessary. +This is required for OS X 10.5 (leopard), which frowns upon use of fork() +and exec(). Also modified zprocess routines to use Launch(). +</ul> + +<H2> Public Release 1.4.0 (15 August 2007)</H2> + +<ul> +<li> Public release of production-quality code, incorporating changes +and improvements from previous beta releases, including: +<ul> +<li> Support for access to ASCII text column files. +<li> Support for fast indexed access of binary tables. +<li> Support for database views of tables, i.e. pre-set values for the filter + specification, the columns to activate, and display format. +<li> New programs include funcone (cone search), funindex (create index files), + and funcen (calculate centroids within regions). +</ul> + +</ul> + +<H2> Release 1.3.0b[n] (mainly internal SAO beta releases)</H2> +<ul> + +<p> +<li> Added -F[c] switch to change the column delimiter to the +specified character. + +<p> +<li> Extended fundisp's format switch (-f) so that it can now handle complex +formats such as 'x=sometext%3d- y=othertest%3d.ext'. + +<p> +<li> Added support for creating and processing 1D FITS images. + +<p> +<li> Added vcol=colname and vcol=/colname to filter specifications to +support use of a third value column when binning 2D images. + +<p> +<li> Added switches to funcone to write out data rows are not within +any cone (-J, -X) and centers which have no close data rows (-L). + +<p> +<li> In funjoin, added ability to specify a numeric tolerance for when joining +two files. + +<p> +<li> shared memory support in gio now can create a shared segment if w+ is +specified as the open mode. + +<p> +<li> Changed reggeometry man page so that examples correctly show angles +going counter-clockwise from the x-axis instead of from the y-axis. + +<p> +<li> Added checks to funmerge to ensure that all files have the same columns. + +<p> +<li> Fixed bug in text support that prevented header-less files from being +processed properly. + +<p> +<li> Added support for 64-bit images (bitpix=64) and table columns (TFORM=K). + +<p> +<li> Filter code was not applying bscale/bzero to columns. + +<p> +<li> Fixed funimage bug that caused a .5/block error in WCS CRPIX values +generated from binary tables. + +<p> +<li> Added feq(a,b) and div(a,b) macros to funcalc. + +<p> +<li> Added support for single-line #define to funcalc. + +<p> +<li> Updated wcs library to 3.6.6 + +<p> +<li> Fix bug in funcen in which ra,dec was not being calculated correctly +if physical and image coords did not match up. + +<p> +<li> The filter syntax "col1 = col2" now explicitly generates an error +(you really want to do "col1 == col2"). + +<p> +<li> Added -o switch to include offset from the nominal target position. + +<p> +<li> Fundisp now displays multi-dimensional vector columns properly. + +<p> +<li> Documented support for lists of files processed as a single file +using "list: file1 ... filen" syntax. + +<p> +<li> Fixed bugs in support for pipe file type (i.e. ability to pass +commands as a filename using "pipe: cmd arg1 ... argn" syntax). + +<p> +<li> Fixed bug in funhist processing of image-based pixel histograms +(i.e using "xy" for columns) where a region was specified. All pixels +outside the region were erroneously being added to the bin containing +the 0 value. + +<p> +<li> Disabled multi-file processing in funds9, which was breaking support +for pathnames containing spaces and is not used by ds9 anyway. + +<p> +<li> Added support for Views of tables, i.e. pre-set values for the +filter specification, the columns to activate, and display format +(though the latter is for fundisp only). + +<p> +<li> Added -l switch to funimage to read x, y, val columns from a list. + +<p> +<li> Removed useless and meaningless section syntax foo'[*]' because it +breaks pointer de-referencing on string columns (i.e. foo'[*xxx=='a']'). +Use foo'[*,*]' instead, as documented. + +<p> +<li> String variables were not always being terminated properly in the +filter code because FITS 'A' data is not necessarily null-terminated. + +<p> +<li> Added funtools version number to all usage() displays. + +<p> +<li> Added explanation of switch arguments to many usage() displays. + +<p> +<li> The filter keyword row# now supports single row selection as well +as range selection, i.e., "row#=100" along with previous "row#=100:200". + +<p> +<li> fundisp now outputs "0x" before hex values. + +<p> +<li> Fixed bug in filter parser which processed rangelists incorrectly +if spaces were put into the rangelist (i.e. "pha= 1 : 3" instead of +pha=1:3). + +<p> +<li> Fixed a bug in funindex which created a wrongly named index file +if more than one "." was in the input file name. + +<p> +<li> Added support to funcone to take ra, dec, radius from a list +(i.e. columns in a FITS file or a text file). + +<p> +<li> Fixed a bug in FunColumnActivate so that if some columns are +explicitly activated while others are de-activated, only the +explicitly activated columns are activated (code was activating all +columns in this case). + +<p> +<li> Fixed a bug in funindex which prevented indexing tables containing +a column named N. + +<p> +<li> fundisp now encloses ASCII column values in single quotes (unless +-T is specified to output RDB format). + +<p> +<li> If a filter specification only involves indexed columns, then the +compiled filter is not used. + +<p> +<li> Funmerge can now be given a list of files to merge using @list syntax. +Also removed the restriction on how many files can be merged (was limited to +the max number of open files). + +<p> +<li> Added ability to edit (add, delete, modify) header parameters in funhead +by specifying an output file (editing acts as a filter) and an edit command +file (which can be stdin). + +<p> +<li> Funtools now contains preliminary code to support (fast) indexed access +of binary tables. See idx.html or "man funidx" for more details. + +<p> +<li> Funtools now contains preliminary code supporting access to ASCII +column files. See text.html or "man funtext" for more details. + +<p> +<li> Fixed bug in funcalc in which columns used in an expression were +always being replaced by new columns, with all associated parameters +(e.g. WCS) were being deleted. Now this only happens if the column +explicitly changes its data type. + +<p> +<li> Fixed bug in funcalc in which the raw data and user data became out +of sync for one row after every 8192 (FUN_MAXROW) rows. + +<p> +<li> Fixed bug in gio in which gseek returned 0 instead of the current byte +offset for disk files. + +<p> +<li> Added funcone program to perform cone search on RA, Dec columns in +a FITS binary table. + +<p> +<li> Fixed bug in polygon, pie and rotated box region filtering for +tables (nearby rows exactly in line between two non-vertical or +non-horizontal vertices were being accepted incorrectly). + +<p> +<li> Fixed pie and panda regions so that the angles now start from +positive x axis == 0 degrees and run counter-clockwise, as +documented. They were going from positive y. NB: a similar change +was made to ds9 release 4.0b3. You must be using ds9 4.0b3 or later +in order to have the correct behavior when generating regions in ds9 +and using them in funtools. + +<p> +<li> Added -p [prog] switch to funcalc to save the generated program. +instead of executing (and deleting) it. + +<p> +<li> Upgraded zlib to 1.2.3. + +</ul> + +<H2> Patch Release 1.2.4 (internal SAO and beta release only)</H2> +<ul> + +<p> +<li> In funcalc, added support for user-specified arguments via the +-a [argstr] switch. These arguments are accessed in the compiled program +using the supplied ARGC and ARGV(n) macros. + +<p> +<li> Added -n (no header display) to fundisp to skip outputting header. + +<p> +<li> Added checks for various types of blank filters. + +<p> +<li> Added macros NROW (current row number) and WRITE_ROW (write current +row to disk) to funcalc. + +<p> +<li> funcalc no longer requires that at least one data column be +specified in the compiled expression. + +<p> +<li> Added FUN_NROWS to FunInfoGet() to return the total number of rows in +an input table (i.e. value of NAXIS2). + +<p> +<li> The compiled funcalc program now includes stdlib.h and unistd.h. + +<p> +<li> The util/NaN.h header file is now modified at configure time to +contain endian status for the target architecture. References to +specific platforms have been removed. + +<P> +<li> Added -m switch to funtable to output multiple files, one for +each input region (and a separate file for events that pass the +filters but are not in any region). + +<p> +<li> Added ability to add new parameters (FunParamPutx) after writing +data if space is previously reserved in the form of a blank parameter +whose value is the name of the param to be updated. (Also requires the +append argument of FunParamPutx be set to 2). + +<p> +<li> Added ability to build shared libraries. With --enable-shared=yes, +shared library is built but not used. With --enable-shared=link, +shared library is linked against (requires proper installation and/or +use of LD_LIBRARY_PATH). + +<p> +<li> Added -v [column] support to funcnts so that counts in a table +can be accumulated using values from a specified column (instead of +the default case where an integral count is accumulated for each event +in a region). + +<p> +<li> Added funcen program to calculate centroids within regions +(binary tables only). Also added support for a funcen-based centroid +tool to funtools.ds9. + +<p> +<li> Fixed bug which prevented successful filtering of columns containing +arrays. + +<p> +<li> Added filter check to ensure that a column is not incorrectly used +as an array. + +<p> +<li> Fundisp now displays column arrays indexed from 0, not 1. + +<p> +<li> Added -i [interval] support to funcnts so that multiple intervals +can be processed in a single pass through the data. For example, +specifying -i "pha=1:5;pha=6:10;pha=11:15" will generate results in +each of 3 pha bands. + +<p> +<li> Fixed calculation of LTV quantities when binning floating point +column data (value was off by 0.5). + +<p> +<li> Added support for 'D' in floating point header values. + +<p> +<li> Added -a switch to funimage and funtable to append output image or +table to an existing FITS file (as an IMAGE or BINTABLE extension). + +<p> +<li> Added support for column scaling (TSCAL and TZERO) on input columns. +Note that the default column type is changed to accommodate scaling (e.g. +a column of type 'I' is changed to 'J', 'J' is changed to 'D') so that +the scaled values can be handled properly by programs such as fundisp +(which utilize default types). + +<p> +<li> Added support to FunColumnSelect() for handling structs of arrays +(i.e. where returned columns are contiguous) instead of the default array +of structs (returned row are contiguous). This is done by specifying +"org=structofarrays" in the plist and passing a single struct containing +the arrays. + +<p> +<li> When writing an rdb/starbase file, fundisp now outputs the full +column name, regardless of the width of the column (which ordinarily +is truncated to match). + +<p> +<li> Fixed support for large files by changing all file positions variables +from "long" declarations to "off_t. + +<p> +<li> Fixed bug in funcalc incorrectly processed multiple array +references (e.g. cur->foo[0]=cur->x;cur->foo[1]=cur->y;) within a single +line of code. + +<p> +<li> Added FILTER_CFLAGS environment variable for all filtering. Also added +--with-filter-cc and --with-filter-cflags options on configure to allow +specification of a default C compiler and associated CFLAGS for filtering. +All of this is necessary in order to support 64-bit libraries under Solaris. + +<p> +<li> Added the funtbl script to extract a table from Funtools ASCII output. + +<p> +<li> Added code to funimage to update IRAF DATASEC keyword. + +<p> +<li> Added checks to ensure that image dimensions are positive. + +<p> +<li> Fixed a bug in funimage where int data was being scaled using BSCALE and +BZERO but these keywords also were being retained in the output image header. +Now the data are not scaled unless the output data type is float (in which +case the scaling parameters are removed). + +<p> +<li> Fixed a bug in funmerge which prevented merging of files unless one of +the -f, -w, or -x switches were used. + +<p> +<li> Fixed a bug in funtable and fundisp which caused the special '$n' column +to be output incorrectly. + +<p> +<li> Fixed sort option in funtable, which previously worked only if the +record size was an even divisor of 8192 (and returned garbage otherwise). + +<p> +<li> Fixed bug in filters involving FITS data type 'X' (bitfield). + +<p> +<li> Fixed bug in funcnts in which the output angles and radii were +being displayed incorrectly when multiple panda shapes were specified. + +<p> +<li> Fixed bug in pandas and pies using n= syntax when first angle +specified was greater than second. The resulting mask was of +the correct shape but contained only a single region. + +<p> +<li> Table row access routines will now decrease maxrows if memory cannot be +allocated for maxrows*sizeof(row), i.e. if the size of a row is so large that +space for maxrows cannot be allocated. + +<p> +<li> The FUN_MAXBUFSIZE environment variable was added to limit the +max buffer size that will be allocated to hold table row data. The +default is 5Mb. + +<p> +<li> Generated PostScript and PDF versions of the help pages. + +<p> +<li> Moved OPTIONS section before (often-lengthy) DESCRIPTION section in +man pages. + +<p> +<li> All memory allocation now does error checking on the result +(except wcs library, which is external code). + +<p> +<li> Removed some compiler warnings that surfaced when using gcc -O2. + +<p> +<li> Updated wcs library to 3.5.5. + +<p> +<li> Upgraded zlib to 1.2.1. + +</ul> + +<H2> Patch Release 1.2.3 (12 January 2004)</H2> +<ul> + +<p> +<li> Generated man pages from the html pages. These are installed +automatically at build time. + +<p> +<li> Changed instances of sprintf() to snprintf() to protect +against buffer overflow. + +<p> +<li> Fixed a number of compiler warnings in non-ANSI compilers. + +<p> +<li> Increased SZ_LINE parameter value from 1024 to 4096. + +</ul> + +<H2> Patch Release 1.2.3b1 (19 August 2003)</H2> +<ul> + +<p> +<li> The rule for using comma to separate a table filter expression +and a region expression has been changed. The rule now states: +<ul> +<LI> if both expressions contain a region, the operator used is <B>or</B>. +<LI> if one (or both) expression(s) does not contain a region, the operator +used is <B>and</B>. +</ul> +This rule handles the cases of pure regions and pure column filters properly. +It unambiguously assigns the boolean <B>and</B> to all mixed cases. Thus: +<PRE> + foo.fits[circle(10,10,3),pi=1:5] +</PRE> +and +<PRE> + foo.fits[pi=1:5,circle(10,10,3)] +</PRE> +both are equivalent to: +<PRE> + foo.fits[circle(10,10,3) && pi=1:5] +</PRE> + +<p> +<li> When include files are used in filters, they now have implied +parentheses surrounding them. Thus, if a region file foo.reg contains two +regions (e.g. circle 1 2 3 and circle 4 5 6), the syntax: +<pre> + pha=4:5&&@foo.reg +</pre> +is equivalent to: +<pre> + pha=4:5 && (circle 1 2 3 || cir 4 5 6) +</pre> +instead of: +<pre> + pha=4:5 && circle 1 2 3 || cir 4 5 6 +</pre> +and the pha filter is applied to both regions. + +<p> +<li> Filters and comments now can be terminated with the string +literal "\n" as well as ";" and the new-line character. This means +that a region can have comments embedded in it: +<pre> + funcnts foo.fits "circle 512 512 10 # color=red\n circle 512 512 20" +</pre> + +<p> +<li> Added capability to update the value of an existing parameter +after writing the table or image (assuming the output image is a +disk file or is being redirected into a file). + +<p> +<li> Improved handling of parentheses in filter expressions. + +<p> +<li> Fixed a bug in image (not event) regions in which circles and +annuli with radius of 1 pixel were not being processed. No counts and +no area would be found in such regions. + +<p> +<li> Fixed a bug in funcnts in which the radii column values for out of sync +if multiple annuli were specified (instead of a single varargs or accel +annulus). + +<p> +<li> By default, fundisp will display integer image data as floats +if the BSCALE and BZERO header parameters are present. + +<p> +<li> Added -L switch to funhead to output starbase list format. + +<p> +<li> Changed the name of the routine _FunColumnSelect to +FunColumnSelectArr, in order to emphasize that it is not +a private routine. + +<p> +<li> Funcalc now checks to ensure that a column was specified as part of +the expression. + +<p> +<li> Funcalc local variables in the compiled program now use a "__" prefix +to avoid conflicts with user-defined variables. + +<p> +<li> Unofficial unsigned short (bitpix=-16) image data now is scaled +correctly using BSCALE and BZERO header parameters. + +<p> +<li> Ported to Intel icc and gcc 3.3 compilers. + +<p> +<li> Updated wcs library to 3.5.1. + +<p> +<li> Changed license from public domain to GNU GPL. + +</ul> + +<H2> Patch Release 1.2.2 (18 May 2003)</H2> +<ul> + +<p> +<li> Fixed funcalc so that it now actually compiles an expression and +runs it, instead of getting a "filter compilation error". Oops! + +<p> +<li> Fixed bug in FunOpen in which the bracket specification was being +removed from the filename if a disk file was opened for "w" or "a". + +<p> +<li> Fixed bug in FunFlush which prevented two successive calls to +FunImagePut from writing the second extension header properly. + +<p> +<li> All filter routines now use gerror(stderr, ...) call instead of +fprintf(stderr, ...) so that output to stderr can be turned off (via +setgerror(level) or GERROR environment variable). + +<p> +<li> All standard Funtools programs check for GERROR environment +variable before setting gerror flag. + +<p> +<li> Some error messages about invalid region arguments were not being +printed. + +<p> +<li> FITS parameters/headers now conform more closely to FITS standard: +<ul> +<li> Blank keywords are treated in the same way as COMMENTS and HISTORY cards +<li> XTENSION keywords are now exactly 8 characters long +<li> 'E' is output instead of 'e' in floating point param values +<li> PCOUNT and GCOUNT are output correctly for image extensions +<li> EXTEND=T is output in primary header +<li> COMMENTS and HISTORY start in column 9 +</ul> + +</ul> + +<H2> Patch Release 1.2.1 (24 April 2003)</H2> +<ul> + +<p> +<li> Varargs ellipse and box annular regions were being +processed incorrectly when the following conditions all were met: +<ul> +<li> the region was specified in physical or wcs coordinates +<li> the data file contained LTM/LTV keywords, i.e., it +was blocked with respect to the original data file +<li> the program being run was an image program (e.g. funcnts, funimage) +</ul> +Varargs ellipse and boxes are regions of the form: +<pre> + ellipse x y a1 b1 a2 b2 ... an bn [angle] + box x y l1 w1 l2 w2 ... ln wn [angle] +</pre> +where at least 2 sets of axis (length) values were specified to form +an annulus (i.e. simple ellipses and boxes worked properly). With all +of the above conditions met, a region in physical coordinates saw its +second length argument converted incorrectly from physical coordinates +to image coordinates. In simple terms, this means that funcnts did not +process elliptical or box regions in physical coords on blocked images +properly. Note that blocking on the command line (e.g. foo.fits[*,*,2]) +did work when no LTM/LTV keywords existed in the file. + +<p> +<li> The fundisp -f switch now supports specification of +column-specific display formats as well as a more convenient way to +specify datatype-specific display formats. Both use keyword=value +specifiers. For columns, use: +<pre> + fundisp -f "colname1=format1 colname2=format2 ..." ... +</pre> +e.g. +<pre> + fundisp -f "time=%13.2f pha=%3d" ... +</pre> +You also can specify display formats for individual datatypes using the FITS +binary table TFORM variables as the keywords: +<pre> + fundisp -f "D=double_format E=float_format J=int_format etc." +</pre> +e.g. +<pre> + fundisp -f "D=%13.2f I=%3d" ... +</pre> +The old position-dependent syntax is deprecated. + +<p> +<li> Fundisp will now print out a single 16-bit (or 32-bit) unsigned +int for a column whose data format is 16X (or 32X), instead of +printing 2 (or 4) unsigned chars. + +<p> +<li> Fixed bug in which fundisp was not able to display bitfield data for +raw event lists. + +<p> +<li> Previously, when binning columns used implicitly in a region +and explicitly in a filter could suffer from a case sensitivity problem. +This has been fixed. + +<p> +<li> Fixed internal mask=all switch on fundisp. + +<p> +<li> Filter include files now simply include text without changing the state +of the filter. They therefore can be used in expression. That is, if foo1 +contains "pi==1" and foo2 contains "pha==2" then the following expressions +are equivalent: +<pre> + "[@foo1&&@foo2]" is equivalent to "[pi==1&&pha==2]" + "[pha==1||@foo2]" is equivalent to "[pi==1||pha==2]" + "[@foo1,@foo2]" is equivalent to "[pi==1,pha==2]" +</pre> + +<p> +<li> Fixed bug in filter specification which caused a SEGV if a varargs-style +region was enclosed in parens. + +<p> +<li> Updated wcs library to 3.3.2. + +</ul> + +<H2> Public Release 1.2.0 (24 March 2003)</H2> +<ul> + +<p> +<li> BSCALE and BZERO are now always applied to int pixel data, instead of +only being applied if the desired output is floating point. + +</ul> + +<H2> Beta Release 1.2.b3 (4 February 2003)</H2> +<ul> + +<p> +<li> In FunColumnSelect, added the ability to specify an offset into +an array in the type specification, using the extended syntax: +<PRE> + [@][n]<type>[[poff]][:[tlmin[:tlmax[:binsiz]]]] +</PRE> +The [poff] string specifies the offset. For example, a type specification +such as "@I[2]" specifies the third (i.e., starting from 0) element in +the array pointed to by the pointer value. A value of "@2I[4]" specifies +the fifth and sixth values in the array. + +<p> +<li> Added a non-varargs version of FunColumnSelect called _FunColumnSelect: +<pre> +int _FunColumnSelect(Fun fun, int size, char *plist, + char **names, char **types, char **modes, int *offsets, + int nargs); +</pre> + +<p> +<li> Added support for sorting binary tables by column name using: +funtable -s "col1 col2 ... coln" ... + +<p> +<li> Added the FUN_RAW macro which, when applied to the "name" parameter +of FunParamGets(), returns the 80-character raw FITS card instead of +only the value. + +<p> +<li> Added support for comparing column values with binary masks of the +form 0b[01]+, e.g.: +<pre> + (status&0b111)==0b001 +</pre> +Previously, such masks had to be specified in decimal, octal, or hex. + +<p> +<li> Completed support for type 'L' (logical) in fundisp and in filtering of +binary tables. + +<p> +<li> Fixed bug in funhist that was improperly setting the number of bins +when the data was of type float. + +<p> +<li> Fixed bug in filter/Makefile where the filter OBJPATH #define was +being passed to the wrong module. + +</ul> + +<H2> Beta Release 1.2.b2 (7 October 2002)</H2> +<ul> + +<p> +<li> Updated wcs library to 3.1.3. + +<p> +<li> Added support for reading gzip'ed files via stdin. + +</ul> + +<H2> Beta Release 1.2.b1 (24 September 2002)</H2> +<ul> + +<p> +<li> Added the following accelerators to region filtering: +<pre> + shape: arguments: + ----- --------- + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + BOX xcenter ycenter xwlo yhin xwout yhhi n=[number] (angle) + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + ELLIPSE xcenter ycenter xwlo yhin xwout yhhi n=[number] (angle) +</pre> + +<p> +<li> Added the following new pandas (Pie AND Annulus) to region filtering: +<pre> + shape: arguments: + ----- --------- + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # same as panda + BPANDA xcen ycen ang1 ang2 nang ixlo iylo ixhi iyhi nrad (ang) # box + EPANDA xcen ycen ang1 ang2 nang ixlo iylo ixhi iyhi nrad (ang) # ellipse +</pre> + +<p> +<li> Added support for filtering images using simple FITS image masks, +i.e. 8-bit or 16-bit FITS images where the value of a pixel is the +region id number for that pixel (and therefore must be greater than +0). The image section being filtered must either be the same size as the +mask dimensions or else be an even multiple of the mask. This works with +image-style filtering, i.e., funcnts can utilize a mask on both +images and binary tables. + +<p> +<li> Added '$n' to fundisp column specification to allow display of +ordinal value of each row passing the filter. + +<p> +<li> Added code to support region filtering on image sections. + +<p> +<li> Fixed bugs which prevented filtering more than one ASCII region file. + +<p> +<li> Fixed bug occasionally causing filter slave processes to become zombies. + +<p> +<li> Fixed bugs in event filtering: annulus with inner radius of 0 +(i.e., a circle) was rejecting events with coordinates xcen, ycen. +Also, pie with angles of 0 and 360 was rejecting some events. +Image filtering (e.g. funcnts) did not have these problems. + +<p> +<li> Filters now accept global exclude regions without an include region. +In such a case, the field region is implied. That is, "-circle(x,y,r)" +is equivalent to "field; -circle(x,y,r)", etc. + +<p> +<li> Fixed panda so that it can be used as a global exclude. + +<p> +<li> Allow empty ds9 region file (comments and globals only) to be +a valid filter. Totally ignore zero length region or include file. + +<p> +<li> Fixed funcnts bug that was displaying 0 value as inner radius of +a circle, instead of just one radius value. + +</ul> + +<H2> Public Release 1.1.0 (22 April 2002)</H2> + +<p> +New features include: +<ul> +<p> +<li> Funtools programs now accept gzip'ed files as valid input. + +<p> +<li> Improved security via replacement of system() function. + +<p> +<li> fundisp, funcnts, funhist can output starbase/rdb format (tabs between columns, form-feeds between tables). + +<p> +<li> Improved support for Windows platform, as well as new support for Mac OSX. +</ul> + +<H2> Pre-Release 1.1.0e (10 April 2002)</H2> +<UL> + +<P> +<LI> Added enough support to skip over variable length arrays in BINTABLES. +We will add full support if this non-standard construct becomes more widely +used. + +<P> +<LI> Fixed bug in underlying fitsy _gread() routine that was returning +an arbitrary bytes-read value if the input fd was invalid. + +</UL> + +<H2> Pre-Release 1.1.0e (19 March 2002)</H2> +<UL> + +<P> +<LI> Added additional check for Windows/PC to filter/Nan.h. + +<P> +<LI> Upgraded zlib library to 1.1.4 (fix double free security hole). + +</UL> + + +<H2> Pre-Release 1.1.0e (27 February 2002)</H2> +<UL> + +<P> +<LI> Changed filter/process.[ch] to filter/zprocess.[ch] to avoid name +collision with Cygwin include file. + +<P> +<LI> Added -a switch to funhead to display all headers in a FITS file. + +</UL> + +<H2> Pre-Release 1.1.0e (11 February 2002)</H2> +<UL> + +<P> +<LI> Fixed filter parser so that it ignores ds9 "ruler" and "text" markers +only up to the first \n or ; (was ignoring to last \n). + +<P> +<LI> The NBLOCK parameter in fitsy/headdata.c was too large for Mac OS X +(max size of a declared char buf seems to be about .5 Mb). + +</UL> + +<H2> Beta Release 1.0.1b5 (31 January 2002)</H2> +<UL> + +<P> +<LI> Fixed bug introduced in calculated IRAF LTM values in 1.0.1b3. + +<P> +<LI> Fixed bug in filter parser giving wrong answers when two range +lists were combined with and explicit boolean operator: +<PRE> + $ fundisp $S"[x=512&&y=511,512]" +</PRE> +incorrectly acted like: +<PRE> + fundisp $S"[(x=512&&y=511)||(y=512)]" +</PRE> +instead of: +<PRE> + fundisp $S"[x=512&&(y=511||y=512)]" +</PRE> +In general, we recommend use of explicit parentheses. + +<P> +<LI> Fixed filter/NaN.h to recognize Compaq Alpha again (broken by their last change to cc). + +<P> +<LI> Removed redundant varargs definitions that conflicted with Alpha compiler definitions. + +<P> +<LI> Added blank line to inc.sed to work around Apple Mac OS X bug in which the +"i" (insert) command was treating final \\ as continuation \ in the text. + +<P> +<LI> Added include of mkrtemp.h to mkrtemp.c to get conditional compilation +for Mac OSX. + +<P> +<LI> Added support for --with-zlib to fitsy so that ds9 could use its own +copy of zlib (and not build the copy in fitsy). + +<P> +<LI> Removed config.cache and Makefile files from distribution tar file. + +</UL> + +<H2> Beta Release 1.0.1b4 (26 January 2002)</H2> +<UL> + +<P> +<LI> Make explicit that column filters are not permitted in an image +expression (such as the funcnts region arguments). + +<P> +<LI> Fix bug in region parser in which a region (without parens), +followed immediately by an operator: +<PRE> + circle 512 512 .5&&pi==1 +</PRE> +was not processing the final argument of the region correctly. + +<P> +<LI> Ignore new "tile" directive in filters (used by ds9). + +</UL> + +<H2> Beta Release 1.0.1b3 (4 January 2002)</H2> +<UL> + +<P> +<LI> Made modifications to Makefile.in to make releases easier. + +<P> +<LI> Added instructions Makefile.in so that funtools.h will always +have correct #defines for FUN_VERSION, FUN_MAJOR_VERSION, +FUN_MINOR_VERSION, and FUN_PATCH_LEVEL. + +<P> +<LI> Allow #include statements in funcalc program files. + +<P> +<LI> funimage now updates all 4 CDX_Y values by the block factor. + +<P> +<LI> Minor changes to make funtools work under darwin (Mac OS X). + +</UL> + +<H2> Beta Release 1.0.1b2 (14 November 2001)</H2> +<UL> + +<P> +<LI> Fixed FunOpen() bug (introduced in b1) in which filenames without +extensions SEGV'ed on open. Yikes! + +<P> +<LI> Funmerge now extends the tlmin/tlmax values of the output +binning columns so that merged events from widely separated files are +valid in the output table. + +<P> +<LI> In funhist, added -w switch to specify bin width (lo:hi:width) +instead of number of bins (lo:hi:num). Added support for this new +width option in funtools.ds9. + +<P> +<LI> If a tdbin value was set using bincols=(name:tlmin:tlmax:tdbin, ...), +the WCS parameters were not being updated properly. + +<P> +<LI> Cleaned up build support for zlib. + +</UL> + +<H2> Beta Release 1.0.1b1 (6 November 2001)</H2> +<UL> + +<P> +<LI> Added support for gzip'ed files to the underlying fitsy/gio +library. This means that all funtools programs now accept gzip'ed +files as valid input: +<PRE> + funcnts foo.fits.gz "circle 504 512 10" +</PRE> +It is no longer necessary to run gunzip and pipe the results to +stdin of a funtools program. + +<P> +<LI> Funtools tasks are now placed in a sub-menu in the DS9 Analysis +menu, instead of at the top level. + +<P> +<LI> Fixed a bug in funcnts in which the bottom-most pixel of a small +circle or annulus region could be missed when the region is only one +pixel wide for that value of y. + +<P> +<LI> Added -n switch to funhist so that table histograms could be +normalized by the width of the bin (val/(hi_edge-lo_edge)). + +<P> +<LI> Added -T switch to fundisp, funcnts, funhist to output in +starbase/rdb format (uses tabs instead of spaces between columns, +form-feeds between tables, etc.) + +<P> +<LI> Fixed a bug in which the field() region was not being properly +processed in combination with an image section. This could affect +funcnts processing of image data where an image section was specified +(though it usually resulted in a funcnts error). + +<P> +<LI> Fixed bug in display of binary table header for vector columns. + +<P> +<LI> Filters now recognize hex constants (starting with 0x) and long +constants (ending with L). + +<P> +<LI>Filenames containing a ':' are now only treated as sockets if they +actually are in the form of a valid ip:port. + +<P> +<LI>Replaced funtools.ds9 with a new version that calls a new funds9 +script, instead of calling funcnts or funhist directly. The new script +supports gzip'ed files and bracket specifications on filenames at the +same time, which the direct call could not. Also the new script has +better error reporting. + +<P> +<LI> Replaced system() call used to compile filter and funcalc +expression with a special launch() call, which performs execvp() +directly without going through sh. (launch() works under DOS and has +fewer security problems.) + +<P> +<LI> Fixed image filter code in which the field() region was being ignored +if it was combined with one or more exclude regions (and no other include +regions), resulting in no valid pixels. + +<P> +<LI> Changed use of getdtable() to FD_SETSIZE in calls to select(). + +<P> +<LI> Added code to guard against FITS binary tables without proper TFORMx +parameters. + +<P> +<LI> Added support to FunParamGets so that it returns the raw FITS card +if the specified input name is NULL and the input n value is positive. + +<P> +<LI> Fixed bug in underlying fitsy code that set the comment in a +header parameter. + +</UL> + + +<H2> Public Release 1.0.0 (31 July 2001)</H2> +<UL> +<P> +<LI> "a new day with no mistakes ... yet" +</UL> + +<HR> +<A HREF="./help.html">Index to the Funtools Help Pages</A> +<H5>Last updated: 22 April 2002</H5> +</BODY> +</HTML> diff --git a/funtools/doc/changelog_beta.html b/funtools/doc/changelog_beta.html new file mode 100644 index 0000000..4e3ccd3 --- /dev/null +++ b/funtools/doc/changelog_beta.html @@ -0,0 +1,953 @@ +<HTML> +<HEAD> +<TITLE>Funtools ChangeLog</TITLE> +</HEAD> +<BODY> +<H2>Funtools ChangeLog</H2> + +<P> +This ChangeLog covers both the Funtools library and the suite of +applications. It will be updated as we continue to develop and improve +Funtools. The up-to-date version can be found <A +HREF="http://hea-www.harvard.edu/RD/funtools/changelog.html">here</A>. + +<H2>Beta release 1.0.b28 (06/26/01)</H2> +<UL> + +<P> +<LI> In funcnts, removed extra new-lines from the primary table, +inadvertently added in cases where zero-area rows are skipped. + +<P> +<LI> Added code to support columns in binary tables that do not have names. + +<P> +<LI> Changed the ds9 radial plot so that the radius (x value) for each +plotted point is taken to be the middle of the annulus, (Rin+Rout)/2, +instead of just the inner annulus, Rin. + +</UL> + +<H2>Beta release 1.0.b27 (06/21/01)</H2> +<UL> + +<P> +<LI> Added missing new-lines in funcnts primary tables that did not +have -r switch. + +<P> +<LI> Filtering with dynamic shared objects (gcc only) now links the +region code into the shared object, instead of relying on finding the +region code in global space. + +<P> +<LI> A few minor changes to column headers in funcnts. +</UL> + +<H2>Beta release 1.0.b26 (05/21/01)</H2> +<UL> +<P> +<LI> Fixed funcnts to work properly when the background region +overlaps the source and therefore explicitly excludes the source. +It was simply ignoring the source regions in such a case. + +<P> +Added a second DS9 init file, funcnts2.ds9, which contains funcnts +and radial profile routines that work from the image stored in DS9's +memory, rather than the original FITS file. This is useful in cases +where no original FITS file exists (e.g., a temp file was created +by a program and sent to DS9). + +<P> +<LI> Fixed funcnts so that region specified with a variable number of +arguments must come last (as is the case with panda and the n= +accelerator). Thus, a region specification such as "pie 504 512 10 20 +30 & circle 504 512 10" is now properly an error. Using "circle 504 512 10 +& pie 504 512 10 20 30" instead ensures that the circle is applied to +each pie. + +<P> +<LI> Fixed funcnts output of radii/angles when boolean expressions +such as "pie & annulus" are specified. + +<P> +<LI> Enhanced funcnts so that -r outputs valid radii for circular regions. + +</UL> + +<H2>Beta release 1.0.b25 (4/19/01)</H2> +<UL> + +<P> +<LI> Added support to funtable to generate a FITS binary table +from an image. By default, a 3-column table is generated, where the +columns are "X", "Y", and "VALUE". For each pixel in the image, a +single row (event) is generated with the "X" and "Y" columns assigned +the dim1 and dim2 values of the image pixel, respectively and the +"VALUE" column assigned the value of the pixel. If the -i +("individual" rows) switch is specified, then only the "X" and "Y" +columns are generated. However the number of rows (events) are written +out for each image pixel is equal to the value of that image pixel(or +1, whichever is larger). + +<P> +<LI> Added -z switch to funcnts to display shapes in primary table +that have no area. This is useful when automatic processing assumes +that there are a set number of rows. + +<P> +<LI> Reworked how the "$region" column is handled. A "$region" column +can now be added to a table already having a "region" column. If a +"region" column already exists, the new column will be "region1". If +that exists, we try "region2", etc. + +<P> +<LI> Added the header parameters EXTNAME="IMAGE" and EXTVER=1 when an +image extension of created automatically. + +<P> +<LI> When funcnts is run with -r, the rad1, rad2, ang1, and ang2 columns +are filled in with "NA" if the shape is not annulus, pie, or panda (instead +of being left blank). + +<P> +<LI> Changed funcnts to display arc-sec/pixel instead of degrees/pixel in the +output header. + +<P> +<LI> Fixed bug in FunColumnSelect() that prevented a new table from +being created that consists only of new columns. + +<P> +<LI> Fixed processing of blank values in FunImageRowGet() (was a SEGV). + +<P> +<LI> Fixed fundisp processing of images (SEGV upon completion, trying to +free space only used by tables). + +<P> +<LI> Fixed bug in funcalc which left executable in /tmp space if -n +(no execute) was specified. + +<P> +<LI>Fixed various bugs processing raw event files, especially on +little-endian machines and reading these files via stdin. + +<P> +<LI> Changed rad1, rad2 columns to radius1, radius2 in funcnts. + +<P> +<LI> Purified dynamically loaded filter code (since purify now can work +with gcc). + +</UL> + +<H2>Beta release 1.0.b24 (03/26/01)</H2> +<UL> + +<P> +<LI> First release of <B>funcalc</B>, the Funtools table calculator. +funcalc is a calculator program that allows arbitrary +expressions to be constructed, compiled, and executed on columns in a +Funtools table (FITS binary table or raw event file). It works by +integrating user-supplied expression(s) into a template C program, +then compiling and executing the program. funcalc expressions +are valid C statements, although some important simplifications (such +as automatic declaration of variables) are supported. + +Within a funcalc expression, reference is made to a +column of the <B>current</B> row using the C syntax +<B>cur->[colname]</B>, e.g. cur->x, cur->pha, etc. Local scalar +variables can either be defined using C syntax at very the beginning +of the expression, or else they can be defined automatically by +funcalc (to be of type double). Thus, for example, a swap of +columns x and y in a table can be performed using either of the following +equivalent funcalc expressions: +<PRE> + double temp; + temp = cur->x; + cur->x = cur->y; + cur->y = temp; +</PRE> +or: +<PRE> + temp = cur->x; + cur->x = cur->y; + cur->y = temp; +</PRE> +When this expression is executed using a command such as: +<PRE> + funcalc -f swap.expr itest.ev otest.ev +</PRE> +the resulting file will have values of the x and y columns swapped. +Many other features are available in funcalc to make table +manipulation easy. See the Funtools program.html documentation. + +<P> +<LI> First release of the <B>funtools.ds9</B> set-up file for adding +Funtools analysis programs to the DS9 Analysis menu. This set-up file +is installed in the same bin directory where Funtools programs are +installed and can be loaded into DS9 from the <B>Load Analysis +Commands ...</B> option of the <B>Analysis</B> menu. Alternatively, +you can tell DS9 to load this file each time it starts by adding the +file to the <B>Edit</B>-><B>Preferences</B>-><B>Analysis +Menu</B>-><B>Analysis File</B> menu option. + +<P> +<LI> Added support for non-integral binning of binary tables. The bincols +specifier on the command line now can take the form: +<PRE> + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]) +</PRE> +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: +<PRE> + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) +</PRE> +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters (respectively) are present in the +FITS binary table header for the column in question. Note that if +only one parameter is specified, it is assumed to be tlmax, and tlmin +defaults to 1. If two parameters are specified, they are assumed to be +tlmin and tlmax. + +<P> +<LI> Added "mask=transparent" support to the plist argument in +FunTableRowGet(). If this string is passed in the call's plist +argument, then all events are passed back to the user. This is useful +when FunColumnSelect() specifies "$region" as a column in order to +return the regionid value for each event. In such a case, events +found in a region have regionid >0, events passing the filter but not +in a region have regionid == -1, events not passing the filter have +regionid ==0. + +<P> +<LI> Added FUN_WCS0 to the FUN_WCS option available to FunInfoGet(). +The original FUN_WCS option returns WCS Library handle (for use with +Doug Mink's wcssubs library) suitable for use with images, regardless +of whether the data are images or tables. For this structure, the WCS +reference point (CRPIX) has been converted to image coordinates if the +underlying file is a table (and therefore in physical coordinates). The +new FUN_WCS0 structure has not had its WCS reference point converted +to image coordinates. It therefore is useful when passing processing +physical coordinates from a table. + +<P> +<LI> Added -G switch to funcnts to print out floating point values +with maximum %.14g precision. + +<P> +<LI> Added -l switch to fundisp, which displays the pixels of an image as +a list with X, Y, VAL columns. + +<P> +<LI> Added support for images to funhist. The program will create +a histogram of the values found in each pixel, or it can perform +a projection over either axis. + +<P> +<LI> All Funtools programs now accept "-" to mean "stdin" or "stdout", +when the io mode is "r" or "w", respectively. + +<P> +<LI> Changed behavior of the prec (precision) argument in FunParamPutd() +(and the underlying fitsy routine ft_cardfmt()) so that if the double +value being put is less than 0.1 or greater than or equal to +10**(20-2-prec), then %20.[prec]e format is used, otherwise +%20.[prec]f format is used. + +<P> +<LI> Fixed behavior of "merge=replace" in FunColumnSelect() so that if +tlmin/tlmax values are not specified in the replacing column, but are +specified in the replaced column, then the original tlmin/tlmax values +are used in the replacing column. + +<P> +<LI> Improved funcnts error-handling when no valid region is specified. + +<P> +<LI> Fixed region parsing of '#' comment character so that comments +are terminated by new-lines, but not ';'. This is more intuitive behavior. + +<P> +<LI> Fixed region parser so that a region (without parens), followed by +a column expression (e.g., "circle 5 5 1,pha==4") is processed correctly. + +<P> +<LI> Fixed funcnts so that the timecorr parameter specified by -t +[timecorr] can be in lower case. + +<P> +<LI> Fixed panda region shapes when processing a blocked image. The +number of pies and number of annuli (args 5 and 8) were incorrectly +being divided by the block factor (i.e., they were being treated as +sizes). + +<P> +<LI> Fixed bug in funcnts that resulted in slightly smaller +integerized pixel boundaries being used when filtering events. This +does not affect ordinary event filtering(in fundisp, funtable, etc.). +In funcnts (which filters binary table events using image-style pixel +filtering), this bug could result in fewer photons being counted than +is the case when the equivalent image is used. + +<P> +<LI> Fixed funcnts to work with raw event files on little-endian machines. + +<P> +<LI> Fixed funhist so that it will read data from a pipe. + +<P> +<LI> Fixed region parser (and funcnts) so that an include file ending +with a comment stops the comment at the end of the include file. + +<P> +<LI> Clarified the meaning of the "," operator (should it be "or" or +"and") between a region and a non-region expression in a filter: if +the second operand in the expression contains a region, the operator +is "or", otherwise it is "and". + +<P> +<LI> Fixed bug in funmerge, which was not handling integerization of +negatively-valued physical pixels properly (not actually used in any +known application). + +</UL> + +<H2>Beta release 1.0.b23 (02/16/01)</H2> +<UL> +<P> +<LI> Fixed funcnts to report area correctly in arc-seconds. + +<P> +<LI> Fixed funcnts to report radii correctly when summing. + +<P> +<LI> Fixed bug in fundisp: a SEGV when trying to display an ASCII column +from a binary table. + +</UL> + +<H2>Beta release 1.0.b22 (02/15/01)</H2> +<UL> + +<P> +<LI> Funcnts now will automatically output appropriate columns from +the primary table in units of arc-seconds instead of pixels if WCS info +is present. Use -p to force the output to be in pixels. + +<P> +<LI> Added qualitative exposure correction to funcnts by means of the switch +"-e source_exp[;background_exp]". For each region, the average exposure is +calculated and net counts (and background) are divided by the average +exposure. See programs.html for more info. + +<P> +<LI> Added qualitative time correction to funcnts by means of the switch +"-t source_time[;background_time]". The net counts (and background) are +divided by this time. See programs.html for more info. + +<P> +<LI> Improved funcnts output. For example, column units are displayed +(since surf_bri units now can be cnts/pix**2, cnts/arcsec**2, etc.) + +<P> +<LI> Changed funcnts.gnuplot and funhist.gnuplot scripts to funcnts.plot +and funhist.plot, respectively. The new scripts take an argument +such as "gnuplot" or "ds9" and output data appropriate for each +target. Also enhanced funcnts.plot so that it senses the axis units +automatically. + +<P> +<LI> Fixed bug in funcnts when handling regions whose centers are out of the +image. Processing often resulted in BUS ERROR or zero counts, and it took +forever to reach those results + +<P> +<LI> Fixed bug in FunImagePut() when outputting float data on +little-endian machines (PCs/Dec Alpha) -- an erroneous error was +signaled trying to convert from native to IEEE before writing. + +<P> +<LI> Fixed bug in fundisp when displaying the mask of a double/float +image using the mask=all option. + +<P> +<LI> Fixed bug in fitsy/headimage/ft_imageloadhead(fits), in which the +default value for LTV[1,2] was incorrectly set to 1.0, not 0.0. This +means that region physical coordinates applied to FITS images and +arrays that did not have LTM/LTV keywords were 1 pixel off. + +<P> +<LI> Fixed obscure bug in region circle processing when a block factor +is specified in the section command but the circle has radius less +than the block. + +<P> +<LI> Fixed bug in FunImageGet which was returning the full image in +cases where a region was specified but no image pixels were in the +region. An empty image is now returned. +</UL> + +<H2>Beta release 1.0.b21 (02/01/01)</H2> +<UL> + +<P> +<LI> In funcnts, added ability to specify a separate background file. +When using a separate background file, the background area will be +normalized by the ration of the pixel sizes of the two files, if +requisite WCS info is available. + +<P> +<LI> In funcnts, added -r switch to output radii (and angle) +information. This is useful with annulus and panda shapes when +plotting radial profiles. An example plot script, funcnts.gnuplot, is +available for use with gnuplot (3.7 and higher): +<PRE> + funcnts ... | funcnts.gnuplot +</PRE> + +<P> +<LI> First version of the funhist program, which creates a 1D +histogram by binning the specified column in a binary table. The +tabular output can be plotted using funhist.gnuplot: +<PRE> + funhist snr.ev x | funhist.gnuplot +</PRE> + +<P> +<LI> Added additional error messages to funcnts when invalid binning +parameters are found for one or more binary table binning columns. + +<P> +<LI> Fixed bug in FunImagePut() which sometimes occurred when +dimensions were passed in the calling sequence. If, in addition, a +reference handle was passed in the FunOpen() call, then the output +dimensions are erroneously taken from the reference file, not the +passed dimensions. + +<P> +<LI> Fixed bug in filter lex code (filt.l) in which yyrestart was +being called incorrectly with the string to be parsed as the argument +(should be NULL). + +<P> +<LI> Fixed bug in filter code in which the Sun cc compiler was creating +a useless .o file in the working directory. + +<P> +<LI> Fixed bug in region parser which made it impossible to specify +angles with a 'd' or 'r' suffix unless WCS info was in the file. (The +use of 'd' or 'r' with angle is independent of WCS but the check was +there anyway.) + +</UL> + +<H2>Beta release 1.0.b20 (11/29/00)</H2> +<UL> + +<P> +<LI> Fixed a serious bug in which exclude regions were being ignored +when multiple annuli were specified. That is, in a region specification +such as: +<PRE> + annulus 512 512 0 100 n=4; -circle 510 510 20 +</PRE> +or +<PRE> + annulus 512 512 0 25 50 75 100; -circle 510 510 20 +</PRE> +the excluded region was not being sensed properly. Note that +single regions did work properly with exclude regions. + +<P> +<LI> Optimized funcnts so that the time for processing an event list +(binary table) is no longer proportional to the number of pixels in +the image. The unoptimized code was taking forever with Chandra ACIS +images (8192**2 pixels), even with relatively few events. + +<P> +<LI> Fixed bugs that gave incorrect answers when image regions were +combined with image sections. + +<P> +<LI> Fixed bug in parsing image section of the form: "foo.fits[*,6:9,2]", +i.e. the default ("*") x dimensions, followed by specified y dimensions. + +<P> +<LI> Added -g option to funcnts to change some output formats from +12.3f to 12.3g to accommodate display of very small numbers. +</UL> + +<H2>Beta release 1.0.b19 (11/21/00)</H2> +<UL> +<P> +<LI> Fixed bug in filter code that caused a SEGV on Solaris machines +when the first specified spatial region is an exclude region. Our +user-supplied qsort/compare algorithm was confusing the Solaris qsort() +routine, causing it to SEGV by trying to process a record prior to the +beginning of the passed array of records. +</UL> + +<H2>Beta release 1.0.b18 (11/13/00)</H2> +<UL> +<P> +<LI> Fixed bug in handling bitpix=-32 (single float) images. + +<P> +<LI> Fixed gio gskip routine to better handle skip of 0 bytes. +</UL> + +<H2>Beta release 1.0.b17 (11/10/00)</H2> +<UL> + +<P> +<LI> Fixed working of $REGION keyword in funtable. + +<P> +<LI> Removed FunFlush() from end of funtable (it was redundant). + +<P> +<LI> Fixed bug in gopen() handling of "pipe:". + +<P> +<LI> Fixed bug in gseek() where pipes, streams, and sockets were +not skipping bytes properly (that can do skips, but cannot do other +sorts of seek). + +</UL> + +<H2>Beta release 1.0.b16 (10/23/00)</H2> +<UL> + +<P> +<LI> Added -s switch to funcnts to support display of summed results +(as well as individual results for each region), i.e. each row in the +summed bkgd-subtracted table contains the sum of counts and areas from +previous rows. + +<P> +<LI> Added -f [format] switch to fundisp to allow control over the +display format for each data type. + +<P> +<LI> Fixed bug in which regions could be incorrectly re-ordered. This +was a problem with annular ellipses and rectangles created by ds9 and +then used in funcnts (the only program where the order of the regions is +important). +</UL> + +<H2>Beta release 1.0.b15 (10/11/00)</H2> +<UL> + +<P> +<LI> Changed the names of routines FunEventsGet() and FunEventsPut() +to FunTableRowGet() and FunTableRowPut(), respectively. The old names +are still valid (using #define in funtools.h), so no code change is +required. + +<P> +<LI> Changed funevents program name to funtable, in line with API changes. + +<P> +<LI> Renamed FunInfoGet/Put() parameter FUN_EVSIZE to FUN_ROWSIZE, in +order to reflect change from use of "events" to use of "row" in +funtools binary table support. The old name is still supported as +an alias. + +<P> +<LI> Completed first version of funmerge program to merge FITS binary tables. + +<P> +<LI> Fixed bug in line region shape that was causing a SEGV with event data. + +<P> +<LI> Fixed minor compiler warnings using "gcc-Wall". + +<P> +<LI> Added the ability for FunOpen() to open a list of event files and +read events from this list synchronously or asynchronously. This +facility is part of an experimental set of parallel processing +techniques that are being added to funtools. Documentation will be +forthcoming when we know which techniques have value! +</UL> + +<H2>Beta release 1.0.b14 (9/22/00)</H2> +<UL> + +<P> +<LI> Added first version of funmerge program to merge FITS binary tables. + +<P> +<LI> Changed the output format of funcnts so that the main results +(background-subtracted table) are displayed first. This means that the +result columns always start at line 4 in the file (after a 1-line +comment and a 2-line header) and end at the first blank line. The +fixed format makes it easier for programs such as sed to extract +results for further processing. For example: + +<PRE> + csh> cat fun.sed + 1,/---- .*/d + /^$/,$d + + csh> funcnts snr.ev[pha==1] "annulus 512 512 0 200 n=8" | sed -f fun.sed + 1 49.000 7.000 0.000 0.000 1941 0.025 0.004 + 2 91.000 9.539 0.000 0.000 5884 0.015 0.002 + 3 129.000 11.358 0.000 0.000 9820 0.013 0.001 + 4 159.000 12.610 0.000 0.000 13752 0.012 0.001 + 5 176.000 13.266 0.000 0.000 17652 0.010 0.001 + 6 183.000 13.528 0.000 0.000 21612 0.008 0.001 + 7 137.000 11.705 0.000 0.000 25528 0.005 0.000 + 8 198.000 14.071 0.000 0.000 29420 0.007 0.000 +</PRE> + +<P> +<LI> Fixed bug in row# processing in which all range was ignored if +lo range value was 1 (e.g., row#=1:7). + +<P> +<LI> Fixed bug in event header processing in which the multiple +instances of keywords HISTORY, COMMENT, and CONTINUE were not all +being copied from the old to the new header (e.g. in funevents). + +<P> +<LI> Fixed processing of ARRAY() and EVENTS() specifiers in FunOpen(). + +<P> +<LI> Fixed 'make clean' directive so that it also cleans funtools subdirs. +</UL> + +<H2>Beta release 1.0.b12 (9/5/00)</H2> +<UL> + +<P> +<LI> Improved the performance of panda regions so that the funcnts +"wall time" now is proportional to the size of the panda region, not +the size of the image. (The latter is the case with the pie shape; use +of panda is recommended over pie.) This means that it is possible to +run funcnts on an ACIS file at zoom 1 (8192x8192) in seconds rather +than (tens of) minutes. + +<P> +<LI> Added support for INET sockets to gio and hence, to funtools. +This means that you can read/write from/to sockets on other machines, +creating a distributed pipeline. For example: +<PRE> + on m1: funevents foo.ev m2:1428 + on m2: funevents :1428 m3:1428 + on m3: funevents :1428 ... +</PRE> +etc. Tests indicate that this is faster than pipes on a single +machine, once the CPU is saturated on that machine. (But note that it +is not faster until the CPU is saturated, due to the efficiency of +Unix pipes and the I/O wait time on non-saturated CPUs.) This new +facility implements the parallel processing technique called "process +decomposition" for pipelines, in which a pipeline process is run on +several machines at once, with different machines handling separate +parts of the pipeline process. +<P> +NB: socket support requires that the libraries: +<PRE> + -lsocket -lnsl +</PRE> +be added to the Solaris link line. + +<P> +<LI> Added support for the row#=lo:hi keyword to process specific rows +in a FITS binary table. For example: +<PRE> + funevents "test.ev[row#=3:8]" stdout ... +or + funevents "test.ev[row#=(3,8)]" stdout ... +</PRE> +will only process rows 3 to 8 (inclusive) of the test.ev file. Along +with image section specification, use of the row#= keyword implements +the parallel processing technique called "data decomposition", in +which several copies of a single program operate on different parts of +a single data file. + +<P> +<LI> Added guard code to image region processing to catch illegal event +values. + +<P> +<LI> Fixed bug when writing FITS image extensions in which dim1, dim2, +and bitpix were being output as 0. + +<P> +<LI> reversed the y row order of displayed images in fundisp, so that pixel +(1,1) is in the lower left corner, as is the case for ds9 image display. +</UL> + +<H2>Beta release 1.0.b11 (8/10/00)</H2> +<UL> + +<P> +<LI> Fixed annulus accelerators and panda regions -- again. Old +problems (from the original implementation) were uncovered related to +the use of these shapes in boolean expressions. Documented an old +restriction that panda and accelerators must be put last in a boolean +expression and added code to signal an error if they are not placed last. + +<P> +<LI> The behavior of the point shape was changed so that multiple x,y +pairs in a single shape specifier now are assigned different region +ids. This makes the behavior of points and annuli consistent with one +another. +</UL> + +<H2>Beta release 1.0.b10 (8/08/00)</H2> +<UL> + +<P> +<LI> Fixed annulus accelerators and panda regions. These were +broken when dynamic loading was implemented. + +<P> +<LI> Fixed a bug in the event filter body code (i.e., the basis for +the slave filter program). Reading the data sometimes incorrectly +calculated the number of events being passed -- which only showed up +occasionally on the Alpha! +</UL> + +<H2>Beta release 1.0.b9 (8/03/00)</H2> +<UL> +<P> +<LI> Removed compilation of some extraneous routines from wcs library. +Also renamed wcssubs directory to wcs. + +<P> +<LI> Added calls to hlength() before wcsinit(). This is necessary in +ds9 (and is a safeguard in other programs) because once hlength() is +called before any invocation of wcsinit(), it must always be used. + +<P> +<LI> Fixed bug in filter code in which CTYPE1 and CTYPE2 param values +were not being passed to wcsinit() as valid FITS strings: the single +quotes were missing. + +<P> +<LI> Fixed a bug in fitsy in which the card buffer was not being +null-terminated properly when a "card insert" call reallocated space +for more cards. + +<P> +<LI> Added protective code so that one cannot set FILTER_PTYPE to +"dynamic" if dynamic filter objects are not available. + +<P> +<LI> Ported to Debian Linux, which (believe it or not) required +removal of extraneous strdup() and strstr() declarations in the code +(apparently these are macros in that version of Debian gcc, so you +cannot declare them). +</UL> + +<H2>Beta release 1.0.b8 (8/01/00)</H2> +<UL> + +<P> +<LI> Added new filter ptype ("contained"), which builds a separate +process by compiling both the main routine and the region code. This +is different from the "process" ptype, which compiles the main +routine, but links in pre-compiled region code (in order to make the +program build more quickly). It is needed by ds9 so that the latter +does not have to keep track of the compiled region code module. +</UL> + +<H2>Beta release 1.0.b7 (7/25/00)</H2> +<UL> + +<P> +<LI> Changed filter code so that, in simple cases, we can access the +contents of a file. This is needed so that funcnts can work properly in one +oft-used case, i.e., if the file foo contains: +<PRE> + circle 5 5 1 + circle 4 4 1 +</PRE> +then: +<PRE> + funcnts foo.fits @foo +</PRE> +now will display the 2 regions in its output, instead of displaying the +near useless "@foo". This only works for simple cases where only a file +is input, not in odd combinations like: +<PRE> + funcnts foo.fits "@foo,circle 1 1 1" +</PRE> + +<P> +<LI> Added programming tutorial and enhanced the programming +reference documentation. + +<P> +<LI> Ported to Windows via the Cygwin environment from +RedHat. We tested on an NT box, which has decent multi-tasking +support. Whether it works on Windows95 is unknown. + +<P> +<LI> Upgraded WCS libraries to 2.8.3. + +<P> +<LI> Ported to new and strict SGI C compiler, which uncovered +lots of unused variables, etc. + +<P> +<LI> Fixed FunParamPut status return. +</UL> + +<H2>Beta release 1.0.b6 (7/15/00)</H2> +<UL> + +<P> +<LI> Ran Purify with each high level program and each funtest +program. +<P> +<LI> Changed behavior of merge=update option of +FunColumnSelect() so that the update of the user column value only +takes place if the user mode has "w" in it. Previously, merge=update +overrode the mode flag and always updated the user value. Note that +all calls to FunColumnSelect with merge=update must change "r" to "rw" +in order to have that user column merged. (The merge=replace already +was checking the mode flag -- the fact that they were doing different +things is a bug.) +<P> +<LI> Added ability to FunOpen extensions by HDU name. (I +thought I had already done this!) +<P> +<LI> Fixed bug that was adding a NULL table at the end of +most binary table files (i.e., those that were not copying the rest of +the input file). +<P> +<LI> Re-ported to Alpha. The problem found most often was the +casting of pointers to ints when doing pointer calculations, i.e.: +<PRE> + char *s, *t; + n = ((int)s - (int)t); +</PRE> +which is invalid on the 64-bit Alpha. Instead use: +<PRE> + char *s, *t; + n = (s - t); +</PRE> +Also broadened the check for use of dlopen in configure to match Alpha's +library configurations (On Alpha, dlopen is in libc). + +<P> +<LI> Changed FunColumnActivate() so that funtools will +process columns in the sorted order specified by that routine. Thus: +<PRE> + fundisp foo.ev "time y x" +</PRE> +will display columns in that order. + +<P> +Sorting does not take place if the activate list contains only exclude +columns (since there is nothing to sort). Also, you can turn off +sorting altogether (mimicking the old behavior) by calling +FunColumnActivate() with a "sort=false" in the plist argument: +<PRE> + FunColumnActivate(fun, "y x", "sort=false"); +</PRE> +or by adding "sort=false" to the activate string itself: +<PRE> + # by default, its sorted + fundisp $E "time y x" + TIME Y X + ---------------- ------- ------- + 6.8500 -7 -7 + 6.8600 -7 -7 +</PRE> +while: +<PRE> + # turn off sorting + ./fundisp $E "time y x sort=false" + X Y TIME + ------- ------- ---------------- + -7 -7 6.8500 + -7 -7 6.8600 + -7 -7 6.8700 +</PRE> +</UL> + +<H2>Beta release 1.0.b5 (7/8/00)</H2> +<UL> + +<P> +<LI> Changed all FunParamSet calls to FunParamPut, to make the +naming consistent with FunImagePut, FunEventsPut, etc. + +<P> +<LI> Fixed bugs preventing tlmin/tlmax from being changed by the +user in binary tables. Also tlmin/tlmax are now written out using a +data type that matches the data type of the respective column. + +<P> +<LI> Extended filter syntax to allow "," as separator between +filename and filters (as well as brackets), i.e.: +<PRE> + foo.fits,events # event extension + foo.fits,pha==1 # filter on default extension + foo.fits,1 # first extension +</PRE> +Note that all but simple expressions will need to be quoted because +of the shell: +<PRE> + foo.fits,pha==1&&pi==2 # & tells shell to run in bkgd + foo.fits,pha==1||pi==2 # similar problems with pipes + foo.fits,circle(1,2,3) # parens are grabbed by shell +</PRE> + +<P> +<LI> Fixed configure so that --with-lib is no longer necessary +to generate a single funtools.a library. Removed this argument from +saoconfig. Note that: +<PRE> + ./configure +</PRE> + +now works properly again, so saoconfig should not be used. + +<P> +<LI> Changed FunFlush() mode argument (single characters) to a +plist argument (keyword arguments). In particular, +<PRE> + FunFlush(fun, "C"); +</PRE> +is now: +<PRE> + FunFlush(fun, "copy=remaining"); +(or FunFlush(fun, "copy=remainder"); ) +</PRE> + +This syntax extension allows FunFlush to support the copy of the +extension associated with the reference handle, which allows one to +copy any extension from an input file to an output file: +<PRE> + /* open a new input extension */ + ifun = FunOpen(...); + /* make this new extension the output reference extension */ + FunInfoPut(ofun, FUN_IFUN, &ifun, 0); + /* copy the current reference extension to output */ + FunFlush(ofun, "copy=reference"); +</PRE> + +<P> +<LI> Fixed bugs in region parser that caused pure floating +point positions (i.e., numbers w/o format characters) always to be +interpreted as pixels. Also fixed galactic and ecliptic conversions. +</UL> + +<HR> +<P> +<A HREF="./help.html">Index to the Funtools Help Pages</A> + +<HR> +<A HREF="./help.html">Index to the Funtools Help Pages</A> + +<H5>Last updated: November 17, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/combo.html b/funtools/doc/combo.html new file mode 100644 index 0000000..ab0bc28 --- /dev/null +++ b/funtools/doc/combo.html @@ -0,0 +1,117 @@ +<!-- =defdoc funcombine funcombine n --> +<HTML> +<HEAD> +<TITLE>Combining Region and Table Filters</TITLE> +</HEAD> +<BODY> + +<!-- =section funcombine NAME --> +<H2><A NAME="funcombine">FunCombine: Combining Region and Table Filters</A></H2> + +<!-- =section funcombine SYNOPSIS --> +<H2>Summary</H2> +<P> +This document discusses the conventions for combining region and table +filters, especially with regards to the comma operator. + + +<!-- =section funcombine DESCRIPTION --> +<H2><A NAME="conventions">Comma Conventions</A></H2> +<P> +Filter specifications consist of a series of boolean expressions, +separated by commas. These expressions can be table filters, +spatial region filters, or combinations thereof. Unfortunately, +common usage requires that the comma operator must act differently +in different situations. Therefore, while its use is intuitive in +most cases, commas can be a source of confusion. + +<P> +According to long-standing usage in IRAF, when a comma separates two +table filters, it takes on the meaning of a boolean <B>and</B>. Thus: +<PRE> + foo.fits[pha==1,pi==2] +</PRE> +is equivalent to: +<PRE> + foo.fits[pha==1 && pi==2] +</PRE> + +When a comma separates two spatial region filters, however, it has +traditionally taken on the meaning of a boolean <B>or</B>. Thus: +<PRE> + foo.fits[circle(10,10,3),ellipse(20,20,8,5)] +</PRE> +is equivalent to: +<PRE> + foo.fits[circle(10,10,3) || ellipse(20,20,8,5)] +</PRE> +(except that in the former case, each region is given a unique id +in programs such as funcnts). + +<P> +Region and table filters can be combined: +<PRE> + foo.fits[circle(10,10,3),pi=1:5] +</PRE> +or even: +<PRE> + foo.fits[pha==1&&circle(10,10,3),pi==2&&ellipse(20,20,8,5)] +</PRE> +In these cases, it is not obvious whether the command should utilize an +<B>or</B> or <B>and</B> operator. We therefore arbitrarily chose to +implement the following rule: +<UL> +<LI> if both expressions contain a region, the operator used is <B>or</B>. +<LI> if one (or both) expression(s) does not contain a region, the operator +used is <B>and</B>. +</UL> +This rule handles the cases of pure regions and pure column filters properly. +It unambiguously assigns the boolean <B>and</B> to all mixed cases. Thus: +<PRE> + foo.fits[circle(10,10,3),pi=1:5] +</PRE> +and +<PRE> + foo.fits[pi=1:5,circle(10,10,3)] +</PRE> +both are equivalent to: +<PRE> + foo.fits[circle(10,10,3) && pi=1:5] +</PRE> + +<P> +[NB: This arbitrary rule <b>replaces the previous arbitrary rule</b> +(pre-funtools 1.2.3) which stated: +<UL> +<LI> if the 2nd expression contains a region, the operator used is <B>or</B>. +<LI> if the 2nd expression does not contain a region, the operator +used is <B>and</B>. +</UL> +In that scenario, the <B>or</B> operator was implied by: +<PRE> + pha==4,circle 5 5 1 +</PRE> +while the <B>and</B> operator was implied by +<PRE> + circle 5 5 1,pha==4 +</PRE> +Experience showed that this non-commutative treatment of the comma +operator was confusing and led to unexpected results.] + +<P> +The comma rule must be considered provisional: comments and complaints +are welcome to help clarify the matter. Better still, we recommend +that the comma operator be avoided in such cases in favor of an +explicit boolean operator. + +<!-- =section funcombine SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 16, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/doc40.fits b/funtools/doc/doc40.fits Binary files differnew file mode 100644 index 0000000..8ac4dcc --- /dev/null +++ b/funtools/doc/doc40.fits diff --git a/funtools/doc/ds9.html b/funtools/doc/ds9.html new file mode 100644 index 0000000..eefdfb6 --- /dev/null +++ b/funtools/doc/ds9.html @@ -0,0 +1,105 @@ +<!-- =defdoc funds9 funds9 n --> +<HTML> +<HEAD> +<TITLE>Funtools and DS9 Image Display</TITLE> +</HEAD> +<BODY> + +<!-- =section funds9 NAME --> +<H2><A NAME="funds9">FunDS9: Funtools and DS9 Image Display</A></H2> + +<!-- =section funds9 SYNOPSIS --> +<H2>Summary</H2> +Describes how funtools can be integrated into the ds9 Analysis menu. + +<!-- =section funds9 DESCRIPTION --> +<H2>Description</H2> +<P> +<A HREF="http://hea-www.harvard.edu/saord/ds9/index.html"> +SAOImage/DS9</A> is an astronomical imaging and data visualization +application used by astronomers around the world. DS9 can display +standard astronomical FITS images and binary tables, but also has +support for displaying raw array files, shared memory files, and data +files automatically retrieved via FTP and HTTP. Standard functional +capabilities include multiple frame buffers, colormap and region +manipulation, and many data scaling algorithms. DS9's advanced +features include TrueColor visuals, deep frame buffers, true +PostScript printing, and display of image mosaics. The program's +support of image tiling, "blinking", arbitrary zoom, rotation, and pan +is unparalleled in astronomy. It also has innovative support for +automatic retrieval and display of standard image data such as the +Digital Sky Survey (using servers at SAO, StScI, or ESO). + +<P> +DS9 can communicate with external programs such as Funtools using the +<A HREF="http://hea-www.harvard.edu/saord/xpa/index.html">XPA</A> +messaging system. In addition, programs can be integrated directly +into the DS9 GUI by means of a configurable Analysis menu. By +default, the DS9 Analysis menu contains algorithms deemed essential to +the core functions of DS9, e.g., display cross-cuts of data, +iso-intensity contours, and WCS grids. However, new programs can be +added to DS9 by creating a set-up file which can be loaded into DS9 +to reconfigure the Analysis menu. + +<PRE> +The basic format of the analysis set-up file is: +<P> + # + # Analysis command descriptions: + # menu label/description + # file templates for this command + # "menu" (add to menu) |"bind" (bind to key) + # analysis command line +</PRE> + +For example, the funcnts program can be specified in this way: +<PRE> + Funcnts (counts in source/bkgd regions; options: none) + * + menu + funcnts $filename $regions(source,,) $regions(background,,) | $text +</PRE> +As shown above, DS9 supports a macro facility to provide information +as well as task support to command lines. For example, the $regions +macro is expanded by DS9 to provide the current source and/or +background region to the analysis command. The $text macro is expanded +to generate a text window display. It also is possible to query for +parameters using a $param macro, plot data using a $plot macro, +etc. See the DS9 documentation for further details. + +<P> +A set-up file called <A HREF="./funtools.ds9">funtools.ds9</A> will +load some useful Funtools applications (counts in regions, radial +profile, X-ray light curve and energy spectrum, 1D histogram) into the DS9 +Analysis menu (version 2.1 and above). The file resides in the bin +directory where Funtools programs are installed. It can be manually +loaded into DS9 from the <B>Load Analysis Commands ...</B> option of +the <B>Analysis</B> menu. Alternatively, you can tell DS9 to load +this file automatically at start-up time by adding the pathname to the +<B>Edit</B>-><B>Preferences</B>-><B>Analysis Menu</B>-><B>Analysis +File</B> menu option. (NB: make sure you select +<B>Edit</B>-><B>Preferences</B>-><B>Save Preferences</B> after setting +the pathname.) + +<P> +The tasks in this setup file generally process the original disk-based +FITS file. Funcnts-based results (radial profile, counts in regions) +are presented in WCS units, if present in the FITS header. For +situations where a disk file is not available (e.g., image data +generated and sent to DS9's 'fits' XPA access point), versions of the +radial profile and counts in regions tasks also are also offered +utilizing DS9's internal image data. Results are presented in pixels. +Aside from the units, the results should be identical to the file-based +results. + +<!-- =section funds9 SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 16, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/env.html b/funtools/doc/env.html new file mode 100644 index 0000000..c060017 --- /dev/null +++ b/funtools/doc/env.html @@ -0,0 +1,222 @@ +<!-- =defdoc funenv funenv n --> +<HTML> +<HEAD> +<TITLE>Funtools Environment Variables</TITLE> +</HEAD> +<BODY> + +<!-- =section funenv NAME --> +<H2><A NAME="funenv">FunEnv: Funtools Environment Variables</A></H2> + +<!-- =section funenv SYNOPSIS --> +<H2>Summary</H2> +Describes the environment variables which can be used to tailor the overall +Funtools environment. + +<!-- =section funenv DESCRIPTION --> +<H2>Description</H2> +<P> +The following environment variables are supported by Funtools: +<DL> +<P> +<DT><B>FITS_EXTNAME</B> +<DD> The <B>FITS_EXTNAME</B> environment variable specifies the +default FITS extension name when <A HREF="./library.html#funopen">FunOpen()</A> is called on a file lacking +a primary image. Thus, +<PRE> + setenv FITS_EXTNAME "NEWEV" +</PRE> +will allow you to call <A HREF="./library.html#funopen">FunOpen()</A> on files without specifying NEWEV in +the +<A HREF="./files.html">Funtools bracket specification</A>. +If no FITS_EXTNAME variable is defined and the extension name also is +not passed in the bracket specification, then the default will be to +look for standard X-ray event table extension names "EVENTS" or +"STDEVT" (we are, after all, and X-ray astronomy group at heart!). + +<P> +<DT><B>FITS_EXTNUM</B> +<DD> The <B>FITS_EXTNUM</B> environment variable specifies the +default FITS extension number when <A HREF="./library.html#funopen">FunOpen()</A> is called on a file lacking +a primary image. Thus, +<PRE> + setenv FITS_EXTNUM 7 +</PRE> +will allow you to call <A HREF="./library.html#funopen">FunOpen()</A> on files to open the seventh +extension without specifying the number in the +<A HREF="./files.html">Funtools bracket specification</A>. + +<P> +<DT><B>FITS_BINCOLS</B> and <B>EVENTS_BINCOLS</B> +<DD> These environment variable specifies the default binning key for +FITS binary tables and raw event files, respectively. They can be +over-ridden using the <B>bincols=[naxis1,naxis2]</B> keyword in a +<A HREF="./files.html">Funtools bracket specification</A>. +The value of each environment variable +is a pair of comma-delimited columns, enclosed in parentheses, to use +for binning. For example, if you want to bin on detx and dety by +default, then use: +<PRE> + setenv FITS_BINCOLS "(detx,dety)" +</PRE> +in preference to adding a bincols specification to each filename: +<PRE> + foo.fits[bincols=(detx,dety)] +</PRE> + +<P> +<DT><B>FITS_BITPIX</B> and <B>EVENTS_BITPIX</B> +<DD> These environment variable specifies the default bitpix value for +binning FITS binary tables and raw event files, respectively. They can +be over-ridden using the <B>bitpix=[value]</B> keyword in a +<A HREF="./files.html">Funtools bracket specification</A>. The value +of each environment variable is one of the standard FITS bitpix values +(8,16,32,-32,-64). For example, if you want binning routines to +create a floating array, then use: +<PRE> + setenv FITS_BITPIX -32 +</PRE> +in preference to adding a bitpix specification to each filename: +<PRE> + foo.fits[bitpix=-32] +</PRE> + +<P> +<DT><B>ARRAY</B> +<DD> The <B>ARRAY</B> environment variable specifies the default +definition of an array file for Funtools. +It is used if there is no array specification passed in the +<B>ARRAY()</B> directive in a +<A HREF="./files.html#arrays">Non-FITS Array specification</A>. +The value of the environment variable is a valid array specification such as: +<PRE> + setenv ARRAY "s100.150" + foo.arr[ARRAY()] +</PRE> +This can be defined in preference to adding the specification to each filename: +<PRE> + foo.arr[ARRAY(s100.150)] +</PRE> + +<P> +<DT><B>EVENTS</B> +<DD> The <B>EVENTS</B> environment variable specifies the default +definition of an raw event file for Funtools. +It is used if there is no EVENTS specification passed in the +<B>EVENTS()</B> directive in a +<A HREF="./files.html#events">Non-FITS EVENTS specification</A>. +The value of the environment variable is a valid EVENTS specification such as: +<PRE> + setenv EVENTS "x:J:1024,y:J:1024,pi:I,pha:I,time:D,dx:E:1024,dx:E:1024" + foo.ev[EVENTS()] +</PRE> +This can be defined in preference to adding the specification to each filename: +<PRE> + foo.ev[EVENTS(x:J:1024,y:J:1024,pi:I,pha:I,time:D,dx:E:1024,dx:E:1024)] +</PRE> +</DL> + +The following filter-related environment variables are supported by Funtools: +<DL> + +<P> +<DT><B>FILTER_PTYPE</B> +<DD> The <B>FILTER_PTYPE</B> environment variable specifies how to +build a filter. There are three possible methods: +<DL> +<DT>process or p +<DD>The filter is compiled and linked against the funtools library (which +must therefore be accessible in the original install directory) to produce +a slave program. This program is fed events or image data and returns +filter results. + +<DT>dynamic or d (gcc only) +<DD>The filter is compiled and linked against the funtools library (which +must therefore be accessible in the original install directory) to produce +a dynamic shared object, which is loaded into the funtools program and +executed as a subroutine. (Extensive testing has shown that, contrary to +expectations, this method is no faster than using a slave process.) + +<DT>contained or c +<DD>The filter and all supporting region code is compiled and linked +without reference to the funtools library to produce a slave program +(which is fed events or image data and returns filter results). This method +is slower than the other two, because of the time it takes to compile the +region filtering code. It is used by stand-alone programs such as ds9, +which do not have access to the funtools library. +</DL> + +By default, <B>dynamic</B> is generally used for gcc compilers and +<B>process</B> for other compilers. However the filter building algorithm +will check for required external files and will use <B>contained</B> is +these are missing. + +<P> +<DT><B>FUN_MAXROW</B> +<DD> The <B>FUN_MAXROW</B> environment variable is used by core +row-processing Funtools programs (funtable, fundisp, funcnts, funhist, +funmerge, and funcalc) to set the maximum number of rows read at once +(i.e. it sets the third argument to the FunTableRowGet() call). The +default is 8192. Note that this variable is a convention only: it will +not be a part of a non-core Funtools program unless code is explicitly +added, since each call to FunTableRowGet() specifies its own maximum +number of rows to read. NB: if you make this value very large, you +probably will need to increase <B>FUN_MAXBUFSIZE</B> (see below) as well. + +<P> +<DT><B>FUN_MAXBUFSIZE</B> +<DD> The <B>FUN_MAXBUFSIZE</B> environment variable is used to limit the +max buffer size that will be allocated to hold table row data. This +buffer size is calculated to be the row size of the table multiplied +by the maximum number of rows read at once (see above). Since the +row size is unlimited (and we have examples of it being larger than 5 +Mb), it is possible that the total buffer size will exceed the machine +capabilities. We therefore set a default value of 5Mb for the max buffer +size, and adjust maxrow so that the total size calculated is less than +this max buffer size. (If the row size is greater than this max buffer +size, then maxrow is set to 1.) This environment variable will change +the max buffer size allowed. + +<P> +<DT><B>FILTER_CC</B> +<DD> The <B>FILTER_CC</B> environment variable specifies the compiler to +use for compiling a filter specification. You also can use the <B>CC</B> +environment variable. If neither has been set, then gcc will be used +if available. Otherwise cc is used if available. + +<P> +<DT><B>FILTER_EXTRA</B> +<DD> The <B>FILTER_EXTRA</B> environment variable specifies extra options +to add to a filter compile command line. In principle, you can add libraries, +include files, and compiler switches. This variable should be used with care. + +<P> +<DT><B>FILTER_TMPDIR</B> +<DD> The <B>FILTER_TMPDIR</B> environment variable specifies the temporary +directory for filter compilation intermediate files. You also can use +the <B>TMPDIR</B> and <B>TMP</B> variables. By default, /tmp is used +as the temporary directory. + +<P> +<DT><B>FILTER_KEEP</B> +<DD> The <B>FILTER_KEEP</B> environment variable specifies whether the +intermediate filter files (i.e. C source file and compile log file) +should be saved after a filter is built. The default is "false", so that +these intermediate files are deleted. This variable is useful for debugging, +but care should be taken to reset its value to false when debugging is +complete. + +</DL> + +<!-- =section funenv SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 16, 2005</H5> + +</BODY> +</HTML> + diff --git a/funtools/doc/evcol.c b/funtools/doc/evcol.c new file mode 100644 index 0000000..1e40df9 --- /dev/null +++ b/funtools/doc/evcol.c @@ -0,0 +1,130 @@ +/* + * + * evcol.c -- example program for processing all extensions and adding a + * column to the binary tables + * + */ +#include <funtools.h> +#include <word.h> + +#define MAXROW 8192 + +typedef struct evstruct{ + char shape[17]; + int mycol; +} *Ev, EvRec; + +int main (int argc, char **argv) +{ + int i, got; + int idx; + int tlmax; + int ext=1; + int total=1; + int haveshape=0; + char *s[3]; + char tbuf[SZ_LINE]; + Fun fun, fun2; + Ev ebuf, ev; + + /* exit on gio errors */ + setgerror(2); + + /* make sure we have minimal arguments */ + if( argc < 3 ) + gerror(stderr, "usage: %s iname oname\n", argv[0]); + + /* open a new output FITS file */ + if( !(fun2 = FunOpen(argv[2], "w", NULL)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + + /* process each input extension in turn */ + for(ext=0; ;ext++){ + /* get new extension name */ + sprintf(tbuf, "%s[%d]", argv[1], ext); + /* open it -- if we cannot open it, we are done */ + if( !(fun=FunOpen(tbuf, "r", NULL)) ) + break; + + /* make the new extension the reference handle for the output file */ + FunInfoPut(fun2, FUN_IFUN, &fun, 0); + + /* if its not a binary table, just write it out */ + for(i=0; i<=2; i++) s[i] = NULL; + if( !(s[0]=FunParamGets(fun, "XTENSION", 0, NULL, &got)) || + strcmp(s[0], "BINTABLE")){ + if( s[0] ) free(s[0]); + FunFlush(fun2, "copy=reference"); + FunClose(fun); + continue; + } + else{ + if( (s[1]=FunParamGets(fun, "EXTNAME", 0, NULL, &got)) ){ + fprintf(stdout, "processing %s", s[1]); + if( (s[2]=FunParamGets(fun, "HDUNAME", 0, NULL, &got)) ) + fprintf(stdout, " %s", s[2]); + fprintf(stdout, "\n"); + } + } + for(i=0; i<=2; i++) if( s[i] ) free(s[i]); + + /* select columns from this new extension for merging */ + /* we have some special code to test various bugs with bpix.fits */ + if( (idx = FunColumnLookup(fun, "SHAPE", 0, + NULL, NULL, NULL, NULL, NULL, NULL)) ){ + haveshape = 1; + FunColumnSelect(fun, sizeof(EvRec), "merge=update", + "SHAPE", "16A", "rw", FUN_OFFSET(Ev, shape), + "MYCOL", "J", "w", FUN_OFFSET(Ev, mycol), + NULL); + } + else{ + FunColumnSelect(fun, sizeof(EvRec), "merge=update", + "MYCOL", "J", "w", FUN_OFFSET(Ev, mycol), + NULL); + } + + /* we have some special code to test various bugs with bpix.fits */ + /* we need to increase tlmax value of the "component" column */ + if( (idx = FunColumnLookup(fun, "COMPONENT", 0, + NULL, NULL, NULL, NULL, NULL, NULL)) ){ + if( (tlmax=FunParamGeti(fun, "TLMAX", idx, -1, &got)) && got ) + FunParamPuti(fun2, "TLMAX", idx, tlmax+100, NULL, 0); + } + + /* get input rows (let routine allocate the row array) */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + /* make sure ascii vector is null-terminated */ + ev->shape[17] = '\0'; + /* used with bpix.ev */ + if( haveshape ){ + cluc(ev->shape); + fprintf(stdout, "\tshape: %s\n", ev->shape); + } + ev->mycol = total++; + } + /* write out this batch of rows, along with the new column */ + FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL); + /* write out one more record with special marker */ + ev = ebuf; + strcpy(ev->shape, "hexagon "); + ev->mycol = -1; + FunTableRowPut(fun2, (char *)ev, 1, 0, NULL); + /* free row data */ + if( ebuf ) free(ebuf); + } + + /* flush output extension (write padding, etc.) */ + FunFlush(fun2, NULL); + /* close the input extension */ + FunClose(fun); + } + + /* all done -- close output */ + FunClose(fun2); + return(0); +} diff --git a/funtools/doc/evmerge.c b/funtools/doc/evmerge.c new file mode 100644 index 0000000..6b7cebc --- /dev/null +++ b/funtools/doc/evmerge.c @@ -0,0 +1,77 @@ +/* + * + * evmerge.c -- example program for merging user values into a table file + * + */ +#include <funtools.h> +#include <word.h> + +#define MAXROW 8192 + +typedef struct evstruct{ + double time; + int time2; +} *Ev, EvRec; + +int main (int argc, char **argv) +{ + int i, got; + char tbuf[SZ_LINE]; + Fun fun, fun2; + Ev ebuf, ev; + + /* exit on gio errors */ + setgerror(2); + + /* make sure we have minimal arguments */ + if( argc < 4 ) + gerror(stderr, "usage: %s iname oname mergetype [columns]\n", argv[0]); + + /* open input file */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + + /* open the output FITS image, inheriting params from input */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + + /* "merge=[type]" merges original input columns with new ones, where type is: + * update -- add new columns, update value of existing ones (same data type) + * replace -- add new columns, replace data type and value of existing ones + * append -- only add new columns, do not "replace" or "update" existing ones + * If tbuf argument is NULL, no merging is performed -- only user-specified + * columns are output. + */ + if( isfalse(argv[3]) ) + *tbuf = '\0'; + else + sprintf(tbuf, "merge=%s", argv[3]); + FunColumnSelect(fun, sizeof(EvRec), tbuf, + "time", "D", "rw", FUN_OFFSET(Ev, time), + "time2", "J", "w", FUN_OFFSET(Ev, time2), + NULL); + + /* activate specified columns -- these will be written to the output file */ + if( argc >= 5 ) + FunColumnActivate(fun, argv[4], NULL); + + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ev->time2 = (int)(ev->time+.5); + ev->time = -(ev->time/10.0); + } + /* write out this batch of rows with the new column */ + FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL); + /* free row data */ + if( ebuf ) free(ebuf); + } + + /* clean up -- close output before input to perform flush automatically */ + FunClose(fun2); + FunClose(fun); + return(0); +} diff --git a/funtools/doc/evnext.c b/funtools/doc/evnext.c new file mode 100644 index 0000000..4a7fa09 --- /dev/null +++ b/funtools/doc/evnext.c @@ -0,0 +1,116 @@ +/* + * + * evnext.c -- example program for changing the time of an row to have + * the value from the next row + * + */ + +#include <funtools.h> + +#define MAXROW 100 + +typedef struct evstruct{ + double time; +} *Ev, EvRec; + +#ifdef ANSI_FUNC +int +main (int argc, char **argv) +#else +main(argc, argv) + int argc; + char **argv; +#endif +{ + int i; + int got, ogot; + char *oraw; + char *nraw; + char *null=NULL; + Ev ebuf, oebuf=NULL; + Ev ev, nev, oev; + Fun fun, fun2; + + /* exit on gio errors */ + setgerror(2); + + /* make sure we have minimal arguments */ + if( argc < 3 ) + gerror(stderr, "usage: %s iname oname [columns]\n", argv[0]); + + /* open file */ + if( !(fun = FunOpen(argv[1], "r", NULL)) ) + gerror(stderr, "could not FunOpen %s\n", argv[1]); + + /* open the output FITS image, inheriting params from input */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + + /* specify columns we want */ + got = FunColumnSelect(fun, sizeof(EvRec), "merge=update", + "time", "1D", "rw", FUN_OFFSET(Ev, time), + NULL); + /* activate specified columns -- these will be written to the output file */ + if( argc >= 4 ) + FunColumnActivate(fun, argv[3], NULL); + + /* get rows */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process the last row from previous batch, if we have it */ + if( oebuf ){ + /* this is the old array of rows -- we point to the last one */ + oev = oebuf+(ogot-1); + /* this is the new array of rows -- we point to the first one */ + ev = ebuf; + /* change time value of old row to new row's value */ + oev->time = ev->time; + /* now we have to write out this last row */ + /* first, save pointer to the new raw data */ + FunInfoGet(fun, FUN_RAWBUF, &nraw); + /* put back old pointer so we merge it with last row */ + FunInfoPut(fun, FUN_RAWBUF, &oraw); + /* write out last row merging with its raw data */ + /* (its arg 4 that tells funtools to merge with the last raw row) */ + FunTableRowPut(fun2, oebuf, 1, ogot-1, NULL); + /* free up old raw data */ + if( oraw ) free(oraw); + /* free up old user data */ + if( oebuf ) free(oebuf); + /* finally, put back new raw data pointer, so we can process new batch */ + FunInfoPut(fun, FUN_RAWBUF, &nraw); + } + /* process the got-1 rows by replacing the time with time from + the succeeding row */ + for(i=0; i<(got-1); i++){ + /* point to the i'th row */ + ev = ebuf+i; + nev = ebuf+(i+1); + ev->time = nev->time; + } + /* if we processed at least one row this time ... */ + if( got > 1 ){ + /* write out got-1 rows with the raw columns */ + FunTableRowPut(fun2, (char *)ebuf, (got-1), 0, NULL); + } + /* if we have a "last" row to process next time ... */ + if( got ){ + /* save the user row for when we have the next batch */ + oebuf = ebuf; + /* save pointer to raw data */ + FunInfoGet(fun, FUN_RAWBUF, &oraw); + /* null out rawbuf pointer so Funtools does not free it automatically */ + FunInfoPut(fun, FUN_RAWBUF, &null); + /* save old value of got */ + ogot = got; + } + } + + /* clean up last saved batch */ + if( oraw ) free(oraw); + if( oebuf ) free(oebuf); + + /* close and return */ + FunClose(fun2); + FunClose(fun); + return(0); +} diff --git a/funtools/doc/evread.c b/funtools/doc/evread.c new file mode 100644 index 0000000..aad5a13 --- /dev/null +++ b/funtools/doc/evread.c @@ -0,0 +1,304 @@ +/* + * + * evread.c -- example program for reading rows in different ways + * + */ + +#include <funtools.h> + +#define MAXROW 10000 + +extern char *optarg; +extern int optind; + +/* single event (used in array of structs) */ +typedef struct evstruct{ + short region; + double x, y; + int pi, pha; + double time; + double dx, dy; +} *Ev, EvRec; + +/* arrays of columns (used in struct of arrays) */ +typedef struct aevstruct{ + short region[MAXROW]; + double x[MAXROW], y[MAXROW]; + int pi[MAXROW], pha[MAXROW]; + double time[MAXROW]; + double dx[MAXROW], dy[MAXROW]; +} *AEv, AEvRec; + +/* pointers to arrays of columns (used in struct of arrays) */ +typedef struct pevstruct{ + short *region; + double *x, *y; + int *pi, *pha; + double *time; + double *dx, *dy; +} *PEv, PEvRec; + +#ifdef ANSI_FUNC +int +main (int argc, char **argv) +#else +main(argc, argv) + int argc; + char **argv; +#endif +{ + int i; + int c; + int got; + int put; + int args; + int doev=0; + int doualloc=0; + char *mode="r"; + char *iname; + char *oname=NULL; + char *params=NULL; + Fun fun=NULL; + Fun ofun=NULL; + void *buf=NULL; + Ev ev, ebuf=NULL; + AEv aev, abuf=NULL; + PEv pev, pbuf=NULL; + + /* process switch arguments */ + while ((c = getopt(argc, argv, "aepuw:")) != -1){ + switch(c){ + case 'a': + doev = 1; + params = "org=soa"; + break; + case 'e': + doev = 0; + params = "org=aos"; + break; + case 'f': + doev = 3; + params = "org=aos"; + break; + case 'p': + doev = 2; + params = "org=soa"; + break; + case 'u': + doualloc = 1; + break; + case 'w': + oname = optarg; + mode = "rw"; + break; + } + } + + /* check for required arguments */ + args = argc - optind; + if( args == 0 ) + iname = "test.ev[EVENTS]"; + else + iname = argv[optind]; + + /* open input file */ + if( !(fun = FunOpen(iname, "r", NULL)) ){ + gerror(stderr, "could not FunOpen %s\n", iname); + exit(1); + } + + /* open output file, if necessary */ + if( oname ){ + if( !(ofun = FunOpen(oname, "w", fun)) ){ + gerror(stderr, "could not FunOpen outout %s\n", oname); + exit(1); + } + } + + /* specify columns we want */ + switch(doev){ + /* array of structs */ + case 0: + got = FunColumnSelect(fun, sizeof(EvRec), params, + "$region", "I", mode, FUN_OFFSET(Ev, region), + "x", "D:10:10", mode, FUN_OFFSET(Ev, x), + "y", "D:10:10", mode, FUN_OFFSET(Ev, y), + "dx", "D:10:10", mode, FUN_OFFSET(Ev, dx), + "dy", "D:10:10", mode, FUN_OFFSET(Ev, dy), + "pi", "J", mode, FUN_OFFSET(Ev, pi), + "pha", "J", mode, FUN_OFFSET(Ev, pha), + "time", "1D", mode, FUN_OFFSET(Ev, time), + NULL); + if( doualloc ){ + ebuf = calloc(MAXROW, sizeof(EvRec)); + buf = ebuf; + } + break; + /* struct of arrays */ + case 1: + got = FunColumnSelect(fun, sizeof(AEvRec), params, + "$region", "I", mode, FUN_OFFSET(AEv, region), + "x", "D:10:10", mode, FUN_OFFSET(AEv, x), + "y", "D:10:10", mode, FUN_OFFSET(AEv, y), + "dx", "D:10:10", mode, FUN_OFFSET(AEv, dx), + "dy", "D:10:10", mode, FUN_OFFSET(AEv, dy), + "pi", "J", mode, FUN_OFFSET(AEv, pi), + "pha", "J", mode, FUN_OFFSET(AEv, pha), + "time", "1D", mode, FUN_OFFSET(AEv, time), + NULL); + if( doualloc ){ + abuf = calloc(1, sizeof(AEvRec)); + buf = abuf; + } + break; + /* struct of pointers */ + case 2: + got = FunColumnSelect(fun, sizeof(PEvRec), params, + "$region", "@I", mode, FUN_OFFSET(PEv, region), + "x", "@D:10:10", mode, FUN_OFFSET(PEv, x), + "y", "@D:10:10", mode, FUN_OFFSET(PEv, y), + "dx", "@D:10:10", mode, FUN_OFFSET(PEv, dx), + "dy", "@D:10:10", mode, FUN_OFFSET(PEv, dy), + "pi", "@J", mode, FUN_OFFSET(PEv, pi), + "pha", "@J", mode, FUN_OFFSET(PEv, pha), + "time", "@1D", mode, FUN_OFFSET(PEv, time), + NULL); + if( doualloc ){ + pbuf = calloc(1, sizeof(PEvRec)); + pbuf->region = calloc(MAXROW, sizeof(short)); + pbuf->x = calloc(MAXROW, sizeof(double)); + pbuf->y = calloc(MAXROW, sizeof(double)); + pbuf->pi = calloc(MAXROW, sizeof(int)); + pbuf->pha = calloc(MAXROW, sizeof(int)); + pbuf->time = calloc(MAXROW, sizeof(double)); + pbuf->dx = calloc(MAXROW, sizeof(double)); + pbuf->dy = calloc(MAXROW, sizeof(double)); + buf = pbuf; + } + break; + /* array of structs containing pointers */ + case 3: + got = FunColumnSelect(fun, sizeof(PEvRec), params, + "$region", "@I", mode, FUN_OFFSET(PEv, region), + "x", "@D:10:10", mode, FUN_OFFSET(PEv, x), + "y", "@D:10:10", mode, FUN_OFFSET(PEv, y), + "dx", "@D:10:10", mode, FUN_OFFSET(PEv, dx), + "dy", "@D:10:10", mode, FUN_OFFSET(PEv, dy), + "pi", "@J", mode, FUN_OFFSET(PEv, pi), + "pha", "@J", mode, FUN_OFFSET(PEv, pha), + "time", "@1D", mode, FUN_OFFSET(PEv, time), + NULL); + if( doualloc ){ + pbuf = calloc(1, sizeof(PEvRec)); + pbuf->region = calloc(1, sizeof(short)); + pbuf->x = calloc(1, sizeof(double)); + pbuf->y = calloc(1, sizeof(double)); + pbuf->pi = calloc(1, sizeof(int)); + pbuf->pha = calloc(1, sizeof(int)); + pbuf->time = calloc(1, sizeof(double)); + pbuf->dx = calloc(1, sizeof(double)); + pbuf->dy = calloc(1, sizeof(double)); + buf = pbuf; + } + break; + default: + gerror(stderr, "unknown params value: %d\n", doev); + break; + } + + /* get rows */ + while( (buf = (void *)FunTableRowGet(fun, buf, MAXROW, NULL, &got)) ){ + /* output if necessary */ + if( ofun ){ + if( (put=FunTableRowPut(ofun, buf, got, 0, NULL)) != got ){ + gerror(stderr, "expected to write %d rows; only wrote %d\n", + got, put); + } + } + /* and display */ + switch(doev){ + case 0: + for(i=0; i<got; i++){ + ev = (Ev)buf+i; + fprintf(stdout, "%d\t%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + (int)ev->region, + ev->x, ev->y, + ev->pi, ev->pha, + ev->dx, ev->dy, + ev->time); + fflush(stdout); + } + break; + case 1: + aev = (AEv)buf; + for(i=0; i<got; i++){ + fprintf(stdout, "%d\t%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + (int)aev->region[i], + aev->x[i], aev->y[i], + aev->pi[i], aev->pha[i], + aev->dx[i], aev->dy[i], + aev->time[i]); + fflush(stdout); + } + break; + case 2: + pev = (PEv)buf; + for(i=0; i<got; i++){ + fprintf(stdout, "%d\t%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + (int)pev->region[i], + pev->x[i], pev->y[i], + pev->pi[i], pev->pha[i], + pev->dx[i], pev->dy[i], + pev->time[i]); + fflush(stdout); + } + break; + case 3: + for(i=0; i<got; i++){ + pev = (PEv)buf+i; + fprintf(stdout, "%d\t%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + (int)pev->region[0], + pev->x[0], pev->y[0], + pev->pi[0], pev->pha[0], + pev->dx[0], pev->dy[0], + pev->time[0]); + fflush(stdout); + } + break; + default: + gerror(stderr, "unknown params value: %d\n", doev); + break; + } + /* if funtools did allocation, free it now */ + if( !doualloc ){ + if( buf ) xfree(buf); + buf = NULL; + } + } + + /* clean up */ + if( doualloc ){ + switch(doev){ + case 0: + if( ebuf ) xfree(ebuf); + break; + case 1: + if( abuf ) xfree(abuf); + break; + case 2: + if( pbuf->region ) xfree(pbuf->region); + if( pbuf->x ) xfree(pbuf->x); + if( pbuf->y ) xfree(pbuf->y); + if( pbuf->pi ) xfree(pbuf->pi); + if( pbuf->pha ) xfree(pbuf->pha); + if( pbuf->time ) xfree(pbuf->time); + if( pbuf->dx ) xfree(pbuf->dx); + if( pbuf->dy ) xfree(pbuf->dy); + if( pbuf ) xfree(pbuf); + break; + } + } + if( ofun ) FunClose(ofun); + FunClose(fun); + return(0); +} diff --git a/funtools/doc/files.html b/funtools/doc/files.html new file mode 100644 index 0000000..0d903a4 --- /dev/null +++ b/funtools/doc/files.html @@ -0,0 +1,627 @@ +<!-- =defdoc funfiles funfiles n --> +<HTML> +<HEAD> +<TITLE>Funtools Data Files</TITLE> +</HEAD> +<BODY> + +<!-- =section funfiles NAME --> +<H2><A NAME="files">FunFiles: Funtools Data Files</A></H2> + +<!-- =section funfiles SYNOPSIS --> +<H2>Summary</H2> +This document describes the data file formats (FITS, array, raw +events) as well as the file types (gzip, socket, etc.) supported +by Funtools. + +<!-- =section funfiles DESCRIPTION --> +<H2>Description</H2> +<P> +Funtools supports FITS images and binary tables, and binary files +containing array (homogeneous) data or event (heterogeneous) data. +IRAF-style brackets are appended to the filename to specify various +kinds of information needed to characterize these data: +<PRE> + file[ext|ind|ARRAY()|EVENTS(),section][filters] + or + file[ext|ind|ARRAY()|EVENTS(),section,filters] +</PRE> +where: +<UL> +<LI> <B>file</B> is the Funtools file name +<LI> <B>ext</B> is the FITS extension name +<LI> <B>ind</B> is the FITS extension number +<LI> <B>ARRAY()</B> is an array specification +<LI> <B>EVENTS()</B> is an event specification +<LI> <B>section</B> is the image section specification +<LI> <B>filters</B> are spatial region and table (row) filters +</UL> + +<H2><A NAME="formats">Supported Data Formats</A></H2> +<P> +Funtools programs (and the underlying libraries) support the +following data file formats: +<UL> +<LI> FITS images (and image extensions) +<LI> FITS binary tables +<LI> binary files containing an array of homogeneous data +<LI> binary files containing events, i.e. records of heterogeneous data +<LI> column-based text files, which are documented <A HREF="./text.html">here</A> +<LI> non-disk files and lists of files +</UL> +Information needed to identify and characterize +the event or image data can be specified on the command line +using IRAF-style bracket notation appended to the filename: +<PRE> + foo.fits # open FITS default extension + image.fits[3] # open FITS extension #3 + events.fits[EVENTS] # open EVENTS extension + array.file[ARRAY(s1024)] # open 1024x1024 short array + events.file[EVENTS(x:1024,y:1024...)] # open non-FITS event list +</PRE> +Note that in many Unix shells (e.g., csh and tcsh), filenames must +be enclosed in quotes to protect the brackets from shell processing. + +<H2><A NAME="fits">FITS Images and Binary Tables</A></H2> +<P> +When <A HREF="./library.html#funopen">FunOpen()</A> opens a FITS file +without a bracket specifier, the default behavior is to look for a +valid image in the primary HDU. In the absence of a primary image, +Funtools will try to open an extension named either <B>EVENTS</B> or +<B>STDEVT</B>, if one of these exists. This default behavior supports +both FITS image processing and standard X-ray event list processing +(which, after all, is what we at SAO/HEAD do). + +<P> +In order to open a FITS binary table or image extension explicitly, it +is necessary to specify either the extension name or the extension +number in brackets: +<PRE> + foo.fits[1] # open extension #1: the primary HDU + foo.fits[3] # open extension #3 of a FITS file + foo.fits[GTI] # open GTI extension of a FITS file +</PRE> +The ext argument specifies the name of the FITS extension (i.e. the +value of the EXTENSION header parameter in a FITS extension), while +the index specifies the value of the FITS EXTVER header parameter. +Following FITS conventions, extension numbers start at 1. + +<P> +When a FITS data file is opened for reading using +<A HREF="./library.html#funopen">FunOpen()</A>, the specified extension +is automatically located and is used to initialize the Funtools internal +data structures. + +<H2><A NAME="events">Non-FITS Raw Event Files</A></H2> + +In addition to FITS tables, Funtools programs and libraries can operate +on non-FITS files containing heterogeneous event records. To specify +such an event file, use: + +<UL> +<LI> file[EVENTS(event-spec)] +<LI> file[EVENTS()] +</UL> +where <B>event-spec</B> is a string that specified the names, data +types, and optional image dimensions for each element of the event +record: +<UL> +<LI> [name]:[n][type]:[(lodim:)hidim] +</UL> + +<P> +Data types follow standard conventions for FITS binary tables, but include +two extra unsigned types ('U' and 'V'): +<UL> +<LI> <B>B</B> -- unsigned 8-bit char +<LI> <B>I</B> -- signed 16-bit int +<LI> <B>J</B> -- signed 32-bit int +<LI> <B>K</B> -- signed 64-bit int +<LI> <B>E</B> -- 32-bit float +<LI> <B>D</B> -- 64-bit float +<LI> <B>U</B> -- unsigned 16-bit int +<LI> <B>V</B> -- unsigned 32-bit int +</UL> +An optional integer value <B>n</B> can be prefixed to the type to indicate +that the element is an array of n values. For example: +<PRE> + foo.fits[EVENTS(x:I,y:I,status:4J)] +</PRE> +defines x and y as 16-bit ints and status as an array of 4 32-bit ints. + +<P> +Furthermore, image dimensions can be attached to the event specification +in order to tell Funtools how to bin the events into an image. They +follow the conventions for the FITS TLMIN/TLMAX keywords. If the low +image dimension is not specified, it defaults to 1. Thus: + +<UL> +<LI> RAWX:J:1:100 +<LI> RAWX:J:100 +</UL> +both specify that the dimension of this column runs from 1 to 100. + +<P> +NB: it is required that all padding be specified in the record +definition. Thus, when writing out whole C structs instead of +individual record elements, great care must be taken to include +the compiler-added padding in the event definition. + +<P> +For example, suppose a FITS binary table has the following set of column +definitions: +<PRE> + TTYPE1 = 'X ' / Label for field + TFORM1 = '1I ' / Data type for field + TLMIN1 = 1 / Min. axis value + TLMAX1 = 10 / Max. axis value + TTYPE2 = 'Y ' / Label for field + TFORM2 = '1I ' / Data type for field + TLMIN2 = 2 / Min. axis value + TLMAX2 = 11 / Max. axis value + TTYPE3 = 'PHA ' / Label for field + TFORM3 = '1I ' / Data type for field + TTYPE4 = 'PI ' / Label for field + TFORM4 = '1J ' / Data type for field + TTYPE5 = 'TIME ' / Label for field + TFORM5 = '1D ' / Data type for field + TTYPE6 = 'DX ' / Label for field + TFORM6 = '1E ' / Data type for field + TLMIN6 = 1 / Min. axis value + TLMAX6 = 10 / Max. axis value + TTYPE7 = 'DY ' / Label for field + TFORM7 = '1E ' / Data type for field + TLMIN7 = 3 / Min. axis value + TLMAX7 = 12 / Max. axis value +</PRE> + +An raw event file containing these same data would have the event +specification: +<PRE> + EVENTS(X:I:10,Y:I:2:11,PHA:I,PI:J,TIME:D,DX:E:10,DY:E:3:12) +</PRE> + +<P> +If no event specification string is included within the EVENTS() operator, +then the event specification is taken from the <B>EVENTS</B> environment +variable: +<PRE> + setenv EVENTS "X:I:10,Y:I:10,PHA:I,PI:J,TIME:D,DX:E:10,DY:E:10" +</PRE> + +<P> +In addition to knowing the data structure, it is necessary to know the +<EM>endian</EM> ordering of the data, i.e., whether or not the data is +in <EM>bigendian</EM> format, so that we can convert to the native +format for this platform. This issue does not arise for FITS Binary +Tables because all FITS files use big-endian ordering, regardless of +platform. But for non-FITS data, big-endian data produced on a Sun +workstation but read on a Linux PC needs to be byte-swapped, since PCs +use little-endian ordering. To specify an ordering, use the +<EM>bigendian=</EM> or <EM>endian=</EM> keywords on the command-line +or the EVENTS_BIGENDIAN or EVENTS_ENDIAN environment variables. The +value of the <EM>bigendian</EM> variables should be "true" or "false", +while the value of the <EM>endian</EM> variables should be "little" or +"big". + +<P> +For example, a PC can access data produced by a Sun using: +<PRE> + hrc.nepr[EVENTS(),bigendian=true] +or + hrc.nepr[EVENTS(),endian=big] +or + setenv EVENTS_BIGENDIAN true +or + setenv EVENTS_ENDIAN big +</PRE> +If none of these are specified, the data are assumed to follow the +format for that platform and no byte-swapping is performed. + +<H2><A NAME="arrays">Non-FITS Array Files</A></H2> + +In addition to FITS images, Funtools programs and libraries can operate +on non-FITS files containing arrays of homogeneous data. To specify +an array file, use: +<UL> +<LI> file[ARRAY(array-spec)] +<LI> file[ARRAY()] +</UL> + +where array-spec is of the form: +<UL> +<LI> [type][dim1][.dim2][:skip][endian] +</UL> + +and where [type] is: +<UL> +<LI> b (8-bit unsigned char) +<LI> s (16-bit short int) +<LI> u (16-bit unsigned short int) +<LI> i (32-bit int) +<LI> r,f (32-bit float) +<LI> d (64-bit float) +</UL> + +<P> +The dim1 specification is required, but dim2 is optional and defaults +to dim1. The skip specification is optional and defaults to 0. The +optional endian specification can be 'l' or 'b' and defaults to the +endian type for the current machine. + +<P> +If no array specification is included within the ARRAY() operator, +then the array specification is taken from the <B>ARRAY</B> environment +variable. For example: + +<PRE> + foo.arr[ARRAY(r512)] # bitpix=-32 dim1=512 dim2=512 + foo.arr[ARRAY(r512.400)] # bitpix=-32 dim1=512 dim2=400 + foo.arr[ARRAY(r512.400]) # bitpix=-32 dim1=512 dim2=400 + foo.arr[ARRAY(r512.400:2880)] # bitpix=-32 dim1=512 dim2=400 skip=2880 + foo.arr[ARRAY(r512l)] # bitpix=-32 dim1=512 dim2=512 endian=little + setenv ARRAY "r512.400:2880" + foo.arr[ARRAY()] # bitpix=-32 dim1=512 dim2=400 skip=2880 +</PRE> + +<H2><A NAME="sections">Specifying Image Sections</A></H2> + +Once a data file (and possibly, a FITS extension) has been specified, +the next (optional) part of a bracket specification can be used to +select image <B>section</B> information, i.e., to specify the x,y +limits of an image section, as well as the blocking factor to apply to +that section. This information can be added to any file specification but +only is used by Funtools image processing routines. + +<P> +The format of the image section specification is one of the following: +<UL> +<LI> file[xy0:xy1,block] +<LI> file[x0:x1,y0:y1,block] +<LI> file[x0:x1,*,block] +<LI> file[*,y0:y1,block] +<LI> file[*,block] +</UL> +where the limit values can be ints or "*" for default. A single "*" +can be used instead of val:val, as shown. Note that blocking is +applied to the section after it is extracted. + +<P> +In addition to image sections specified by the lo and hi x,y limits, image +sections using center positions can be specified: +<UL> +<LI> file[dim1@xcen,dim2@ycen] +<LI> file[xdim2@xcen@ycen] +<LI> file[dim1@xcen,dim2@ycen,block] +<LI> file[dim@xcen@ycen,block] +</UL> +Note that the (float) values for dim, dim1, dim2, xcen, ycen must be +specified or else the expression does not make sense! + +<P> +In all cases, block is optional and defaults to 1. An 's' or 'a' can +be appended to signify "sum" or "average" blocking (default is "sum"). +Section specifications are given in image coordinates by default. If you +wish to specify physical coordinates, add a 'p' as the last character +of the section specification, before the closing bracket. +For example: +<PRE> +<UL> +<LI> file[-8:-7,-8:-7p] +<LI> file[-8:-7,-8:-7,2p] +</UL> +</PRE> +A section can be specified in any Funtools file name. If the operation +to be applied to that file is an imaging operation, then the +specification will be utilized. If the operation is purely a table +operation, then the section specification is ignored. + +<P> +Do not be confused by: +<PRE> + foo.fits[2] + foo.fits[*,2] +</PRE> +The former specifies opening the second extension of the FITS file. +The latter specifies application of block 2 to the image section. + +<P> +Note that the section specification must come after +any of FITS <B>ext</B> name or <B>ind</B> number, +but all sensible defaults are supported: +<UL> +<LI> file[ext] +<LI> file[ext,index] +<LI> file[index] +<LI> file[ext,section] +<LI> file[ext,index,section] +<LI> file[index,section] +<LI> file[section] +</UL> + +<H2><A NAME="binning">Binning FITS Binary Tables and Non-FITS Event Files</H2> + +If a FITS binary table or a non-FITS raw event file is to be binned +into a 2D image (e.g., using the +<A HREF="./programs.html#funimage">funimage</A> +program), it is necessary to specify the two columns to be used for the +binning, as well as the dimensions of the image. Funtools first looks +for a specifier of the form: +<PRE> + bincols=([xnam[:tlmin[:tlmax:[binsiz]]]],[ynam[:tlmin[:tlmax[:binsiz]]]]) +</PRE> +in bracket syntax, and uses the column names thus specified. The tlmin, tlmax, +and binsiz specifiers determine the image binning dimensions using: +<PRE> + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) +</PRE> +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters are present in the FITS binary +table header, respectively. If only one parameter is specified, it is +assumed to be tlmax, and tlmin defaults to 1. If two parameters are +specified, they are assumed to be tlmin and tlmax. + +For example, to bin an HRC event list columns "VPOS" and "UPOS", use: +<PRE> + hrc.nepr[bincols=(VPOS,UPOS)] +</PRE> +or +<PRE> + hrc.nepr[bincols=(VPOS:49152,UPOS:4096)] +</PRE> +Note that you can optionally specify the dimensions of these columns +to cover cases where neither TLMAX keywords are defined in +the header. If either dimension is specified, then both must be specified. + +<P> +You can set the FITS_BINCOLS or EVENTS_BINCOLS environment variable as +an alternative to adding the "bincols=" specifier to each file name +for FITS binary tables and raw event files, respectively. If no +binning keywords or environment variables are specified, or if the +specified columns are not in the binary table, the Chandra parameters +CPREF (or PREFX) are searched for in the FITS binary table header. +Failing this, columns named "X" and "Y" are sought. If these are not +found, the code looks for columns containing the characters "X" and +"Y". Thus, you can bin on "DETX" and "DETX" columns without +specifying them, if these are the only column names containing the "X" +and "Y" characters. + +<p> +Ordinarily, each event or row contributes one count to an image pixel +during the 2D binning process. Thus, if five events all have the same +(x,y) position, the image pixel value for that position will have a +value of five. It is possible to specify a variable contribution +for each event by using the vcol=[colname] filter spec: +<PRE> + vcol=[colname] +</PRE> +The vcol colname is a column containing a numeric value in each event row +that will be used as the contribution of the given event to its image +pixel. For example, consider an event file that has the following content: +<PRE> + x:e:4 y:e:4 v:e + ------ ------ ---- + 1 1 1.0 + 2 2 2.0 + 3 3 3.0 + 4 4 0.0 + 1 1 1.0 + 2 2 2.0 + 3 3 3.0 + 4 4 4.0 +</PRE> +There are two events with x,y value of (1,1) so ordinarily a 2D image will +have a value of 2 in the (1,1) pixel. If the v column is specified as the +value column: +<PRE> + foo.fits'[vcol=v]' +</PRE> +then each pixel will contain the additive sum of the associated (x,y) +column values from the v column. For example, image pixel (1,1) will +contain 1. + 1. = 2, image pixel (2,2) will contain (2 + 2) = 4, etc. + +<p> +An important variation on the use of a value column to specify the +contribution an event makes to an image pixel is when the value column +contains the reciprocal of the event contribution. For this case, the +column name should be prefixed with a / (divide sign) thus: +<PRE> + foo.fits'[vcol=/v]' +</PRE> +Each image pixel value will then be the sum of the reciprocals of the value +column. A zero in the value column results in NaN (not a number). +Thus, in the above example, image pixel (1.1) will contain 1/1 + 1/1 = 2, +image pixel (2,2) will contain (1/2 + 1/2) = 1, etc. Image pixel (4,4) +will contain (1/0 + 1/4) = NaN. + +<p> +You can set the FITS_VCOL or EVENTS_VCOL environment variable as +an alternative to adding the "vcol=" specifier to each file name +for FITS binary tables and raw event files, respectively. + +<P> +Finally, when binning events, the data type of the resulting 2D image +must be specified. This can be done with the "bitpix=[n]" keyword in +the bracket specification. For example: +<PRE> + events.fits[bincols=(VPOS,UPOS),bitpix=-32] +</PRE> +will create a floating point image binned on columns VPOS and UPOS. +If no bitpix keyword is specified, bitpix=32 is assumed. As with +bincols values, you also can use the FITS_BITPIX and EVENTS_BITPIX +environment variables to set this value for FITS binary tables and +raw event files, respectively. + +<P> +The <b>funimage</b> program also allows you to create a 1D image projection +along any column of a table by using the <b>bincols=[column]</b> +filter specification and specifying a single column. +For example, the following command projects a 1D image along +the chipx column of a table: +<PRE> + funimage ev.fits'[bincols=chipx]' im.fits +</PRE> +See <A HREF="./programs.html#funimage">funimage</A> for more +information about creating 1D and 2D images. + +<P> +Finally, please note that Funtools supports most FITS standards. +We will add missing support as required by the community. In general, +however, we do not support non-standard extensions. For example, we +sense the presence of the binary table 'variable length array' +proposed extension and we pass it along when copying and filtering +files, but we do not process it. We will add support for new standards +as they become official. + +<H2><A NAME="filters">Table and Spatial Region Filters</H2> +<P> +Note that, in addition extensions and image sections, Funtools bracket +notation can be used to specify table and spatial region filters. These +filters are always placed after the image section information. They +can be specified in the same bracket or in a separate bracket +immediately following: +<UL> +<LI> file[ext|ind|ARRAY()|EVENTS(),section][filters] +<LI> file[ext|ind|ARRAY()|EVENTS(),section,filters] +</UL> +where: +<UL> +<LI> <B>file</B> is the Funtools file name +<LI> <B>ARRAY()</B> is an array specification +<LI> <B>EVENTS()</B> is an event list specification +<LI> <B>ext</B> is the FITS extension name +<LI> <B>ind</B> is the FITS extension number +<LI> <B>section</B> is the image section to extract +<LI> <B>filters</B> are spatial region and table (row) filters to apply +</UL> + +The topics of table and region filtering are covered in detail in: +<UL> +<LI><A HREF="./filters.html">Table Filtering</A> +<LI><A HREF="./regions.html">Spatial Region Filtering</A> +</UL> + +<H2><A NAME="types">Disk Files and Other Supported File Types</A></H2> +<P> +The specified <B>file</B> usually is an ordinary disk file. In +addition, gzip'ed files are supported in Funtools: gzip'ed input files +are automatically uncompressed as they are read, and gzip'ed output +files are compressed as they are written. NB: if a FITS binary table +is written in gzip format, the number of rows in the table will be set +to -1. Such a file will work with Funtools programs but will not work +with other FITS programs such as ds9. + +<P> +The special keywords "stdin" and "stdout" designate Unix standard +input and standard output, respectively. The string "-" (hyphen) will +be taken to mean "stdin" if the file is opened for reading and +"stdout" if the file is opened for writing. + +<P> +A file also can be an INET socket on the same or another machine using +the syntax: +<PRE> + machine:port +</PRE> +Thus, for example: +<PRE> + karapet:1428 +</PRE> +specifies that I/O should be performed to/from port 1428 on the +machine karapet. If no machine name is specified, the default is to +use the current machine: +<PRE> + :1428 +</PRE> +This means to open port 1428 on the current machine. Socket support +allows you to generate a distributed pipe: +<PRE> + on karapet: funtask1 in.fits bynars:1428 + on bynars: funtask2 :1428 out.fits +</PRE> +The socket mechanism thus supports simple parallel processing using +<B>process decomposition</B>. Note that parallel processing using +<B>data decomposition</B> is supported via the <B>section</B> specifier (see +below), and the <B>row#</B> specifier, which is part of +<A HREF="./filters.html">Table Filtering</A>. + +<P> +A file also can be a pointer to shared memory using the syntax: +<PRE> + shm:[id|@key][:size] +</PRE> +A shared memory segment is specified with a <B>shm:</B> prefix, +followed by either the shared memory id or the shared memory key +(where the latter is prefixed by the '@' character). The size (in +bytes) of the shared memory segment can then be appended (preceded by +the ':' character). If the size specification is absent, the code will +attempt to determine the length automatically. + +If the open mode contains the string "w+", then the memory segment will be +created if it does not exist. (It also will be released and deleted when the +file is closed.) In the case where a memory segment is being created, the +length of the segment is required. + +<P> +A file also can be Unix piped command (i.e. a program to run) using the syntax: +<PRE> + "pipe: command arg1 ... argn" +</PRE> +The output from the command must be a valid FITS file. It is important +to use quotes to protect spaces so that command arguments are passed +correctly. A silly example is: +<PRE> + fundisp "pipe: funtable 'foo.fits[cir 512 512 .1]' stdout" +</PRE> +This seemed like a good idea at the time ... + +<H2><A NAME="types">Lists of Files</A></H2> + +<P> +Funtools also will process a list of files as a single file using the +syntax: +<PRE> + "list: file1 file2 ... filen" +</PRE> +The files in the list are separated by whitespace. Any of the +above file types can be used. For example, if two files, foo1.fits and +foo2.fits, are part of the same observation, they can be processed as +a single file (using their own filters): +<PRE> + fundisp "list: foo1.fits[cir(512,512,10)] foo2.fits[cir(511,511,10)]" + X Y PHA PI TIME DX DY + -------- -------- -------- -------- --------------------- -------- -------- + 512 512 6 7 79493997.45854475 578 574 + 512 512 8 9 79494575.58943175 579 573 + 512 512 5 6 79493631.03866175 578 575 + 512 512 5 5 79493290.86521725 578 575 + 512 512 8 9 79493432.00990875 579 573 + 511 511 5 5 79488631.09462625 580 575 + 511 511 10 11 79488780.60006675 580 573 + 511 511 4 4 79494562.35474326 580 575 + 511 511 6 6 79488203.01561825 580 575 + 511 511 6 6 79488017.99730176 580 575 + 511 511 4 4 79494332.45355175 580 575 + 511 511 9 10 79492685.94014275 581 574 + 511 511 5 5 79487708.71298325 580 575 + 511 511 8 9 79493719.00160225 581 573 +</PRE> +Again, note that it is important to avoid spaces in the filters +because the list separator also is whitespace. To protect whitespace +in a filter, enclose the file specification in quotes: +<PRE> + fundisp "list: 'foo1.fits[cir 512 512 .1]' foo2.fits[cir(511,511,.1)]" +</PRE> + +<!-- =section funfiles SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: February 15, 2006</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/filters.html b/funtools/doc/filters.html new file mode 100644 index 0000000..bd78fea --- /dev/null +++ b/funtools/doc/filters.html @@ -0,0 +1,325 @@ +<!-- =defdoc funfilters funfilters n --> +<HTML> +<HEAD> +<TITLE>Table Filtering</TITLE> +</HEAD> +<BODY> + +<!-- =section funfilters NAME --> +<H2><A NAME="funfilters">Funfilters: Filtering Rows in a Table</A></H2> + +<!-- =section funfilters SYNOPSIS --> +<H2>Summary</H2> +<P> +This document contains a summary of the user interface for +filtering rows in binary tables. + +<!-- =section funfilters DESCRIPTION --> +<H2>Description</H2> +<P> +Table filtering allows a program to select rows from an table (e.g., +X-ray event list) by checking each row against one or more expressions +involving the columns in the table. When a table is filtered, only +valid rows satisfying these expressions are passed through for processing. + +<P> +A filter expression is specified using bracket notation appended to +the filename of the data being processed: +<PRE> + foo.fits[pha==1&&pi==2] +</PRE> +It is also possible to put region specification inside a file and +then pass the filename in bracket notation: +<PRE> + foo.fits[@my.reg] +</PRE> +Filters must be placed after the extension and image section +information, when such information is present. The correct order is: +<UL> +<LI> file[fileinfo,sectioninfo][filters] +<LI> file[fileinfo,sectioninfo,filters] +</UL> +where: +<UL> +<LI> <B>file</B> is the Funtools file name +<LI> <B>fileinfo</B> is an ARRAY, EVENT, FITS extension, or FITS index +<LI> <B>sectioninfo</B> is the image section to extract +<LI> <B>filters</B> are spatial region and table (row) filters to apply +</UL> +See <A HREF="./files.html">Funtools Files</A> for more information +on file and image section specifications. + +<H2>Filter Expressions</H2> + +<P> +Table filtering can be performed on columns of data in a FITS +binary table or a raw event file. Table filtering is accomplished by +means of <B>table filter specifications</B>. An table filter +specification consists of one or more <B>filter expressions</B> Filter +specifications also can contain comments and local/global processing +directives. + +<P> +More specifically, a filter specification consist of one or more lines +containing: +<PRE> + # comment until end of line + # include the following file in the table descriptor + @file + # each row expression can contain filters separated by operators + [filter_expression] BOOLOP [filter_expression2], ... + # each row expression can contain filters separated by the comma operator + [filter_expression1], [filter_expression2], ... + # the special row# keyword allows a range of rows to be processed + row#=m:n + # or a single row + row#=m + # regions are supported -- but are described elsewhere + [spatial_region_expression] +</PRE> + +<P> +A single filter expression consists of an arithmetic, logical, or +other operations involving one or more column values from a +table. Columns can be compared to other columns, to header values, +or to numeric constants. Standard math functions can be applied to +columns. Separate filter expressions can be combined using boolean operators. +Standard C semantics can be used when constructing expressions, with +the usual precedence and associativity rules holding sway: +<PRE> + Operator Associativity + -------- ------------- + () left to right + !! (logical not) right to left + ! (bitwise not) - (unary minus) right to left + * / left to right + + - left to right + < <= > >= left to right + == != left to right + & (bitwise and) left to right + ^ (bitwise exclusive or) left to right + | (bitwise inclusive or) left to right + && (logical and) left to right + || (logical or) left to right + = right to left +</PRE> +For example, if energy and pha are columns in a table, +then the following are valid expressions: +<PRE> + pha>1 + energy == pha + (pha>1) && (energy<=2) + max(pha,energy)>=2.5 +</PRE> + +<P> +Comparison values can be integers or floats. Integer comparison values can be +specified in decimal, octal (using '0' as prefix), hex (using '0x' as prefix) +or binary (using '0b' as prefix). Thus, the following all specify the same +comparison test of a status mask: +<PRE> + (status & 15) == 8 # decimal + (status & 017) == 010 # octal + (status & 0xf) == 0x8 # hex + (status & 0b1111) == 0b1000 # binary +</PRE> +<P> +The special keyword row# allows you to process a range of rows. +When row# is specified, the filter code skips to the designated +row and only processes the specified number of rows. The +"*" character can be utilized as the high limit value to denote +processing of the remaining rows. Thus: +<PRE> + row#=100:109 +</PRE> +processes 10 rows, starting with row 100 (counting from 1), +while: +<PRE> + row#=100:* +</PRE> +specifies that all but the first 99 rows are to be processed. + +<P> +Spatial region filtering allows a program to select regions of an +image or rows of a table (e.g., X-ray events) using simple geometric +shapes and boolean combinations of shapes. For a complete description +of regions, see <A HREF="./regions.html">Spatial Region Filtering</A>. + +<H2><A NAME="separators">Separators Also Are Operators</A></H2> +<P> +As mentioned previously, multiple filter expressions can be specified +in a filter descriptor, separated by commas or new-lines. +When such a comma or new-line separator is used, the boolean AND operator +is automatically generated in its place. Thus and expression such as: +<PRE> + pha==1,pi=2:4 +</PRE> +is equivalent to: +<PRE> + (pha==1) && (pi>=2&&pi<=4) +</PRE> +<P> +[Note that the behavior of separators is different for filter expressions +and spatial region expressions. The former uses AND as the operator, while +the latter user OR. See +<A HREF="./combo.html">Combining Region and Table Filters</A> +for more information about these conventions and how they are treated +when combined.] + +<H2><A NAME="range">Range Lists</A></H2><P> +<P> +Aside from the standard C syntax, filter expressions can make use of +IRAF-style <B>range lists</B> which specify a range of values. The +syntax requires that the column name be followed by an '=' sign, which +is followed by one or more comma-delimited range expressions of the form: +<PRE> + col = vv # col == vv in range + col = :vv # col <= vv in range + col = vv: # col >= vv in range + col = vv1:vv2 # vv1 <= col <= vv2 in range +</PRE> +The vv's above must be numeric constants; the right hand side of a +range list cannot contain a column name or header value. +<P> +Note that, unlike an ordinary comma separator, the comma separator used +between two or more range expressions denotes OR. Thus, when two or +more range expressions are combined with a comma separator, the resulting +expression is a shortcut for more complicated boolean logic. For example: +<PRE> + col = :3,6:8,10: +</PRE> +is equivalent to: +<PRE> + (col<=3) || (col>=6 && col <=8) || (col >=10) +</PRE> +Note also that the single-valued rangelist: +<PRE> + col = val +</PRE> +is equivalent to the C-based filter expression: +<PRE> + col == val +</PRE> +assuming, of course, that val is a numeric constant. + +<H2><A NAME="math">Math Operations and Functions</A></H2><P> +<P> +It is permissible to specify C math functions as part of the filter syntax. +When the filter parser recognizes a function call, it automatically +includes the math.h and links in the C math library. Thus, it is +possible to filter rows by expressions such as these: +<UL> +<LI>(pi+pha)>(2+log(pi)-pha) +<LI> min(pi,pha)*14>x +<LI> max(pi,pha)==(pi+1) +<LI> feq(pi,pha) +<LI> div(pi,pha)>0 +</UL> +The function feq(a,b) returns true (1) if the difference between a and b +(taken as double precision values) is less than approximately 10E-15. +The function div(a,b) divides a by b, but returns NaN (not a number) +if b is 0. It is a safe way to avoid floating point errors when +dividing one column by another. + +<H2><A NAME="include">Include Files</A></H2><P> +<P> +The special <B>@filename</B> directive specifies an include file +containing filter expressions. This file is processed as part of +the overall filter descriptor: +<PRE> + foo.fits[pha==1,@foo] +</PRE> + +<H2><A NAME="header">Header Parameters</A></H2><P> +<P> +The filter syntax supports comparison between a column value and a +header parameter value of a FITS binary tables (raw event files have no +such header). The header parameters can be taken from the binary +table header or the primary header. For example, assuming there is a +header value MEAN_PHA in one of these headers, you can select photons +having exactly this value using: + +<UL> +<LI> pha==MEAN_PHA +</UL> + +<H2>Examples</H2> +<P> +Table filtering is more easily described by means of examples. +Consider data containing the following table structure: +<UL> +<LI> double TIME +<LI> int X +<LI> int Y +<LI> short PI +<LI> short PHA +<LI> int DX +<LI> int DY +</UL> + +<P> +Tables can be filtered on these columns using IRAF/QPOE range syntax or +any valid C syntax. The following examples illustrate the possibilities: +<DL> + +<P> +<DT> pha=10 +<DT> pha==10 +<DD> select rows whose pha value is exactly 10 + +<P> +<DT> pha=10:50 +<DD> select rows whose pha value is in the range of 10 to 50 + +<P> +<DT> pha=10:50,100 +<DD> select rows whose pha value is in the range of 10 to 50 or is +equal to 100 + +<P> +<DT> pha>=10 && pha<=50 +<DD> select rows whose pha value is in the range of 10 to 50 + +<P> +<DT> pi=1,2&&pha>3 +<DD> select rows whose pha value is 1 or 2 and whose pi value is 3 + +<P> +<DT> pi=1,2 || pha>3 +<DD> select rows whose pha value is 1 or 2 or whose pi value is 3 + +<P> +<DT> pha==pi+1 +<DD> select rows whose pha value is 1 less than the pi value + +<P> +<DT> (pha==pi+1) && (time>50000.0) +<DD> select rows whose pha value is 1 less than the pi value +and whose time value is greater than 50000 + +<P> +<DT>(pi+pha)>20 +<DD> select rows in which the sum of the pi and pha values is greater +than 20 + +<P> +<DT> pi%2==1 +<DD> select rows in which the pi value is odd +</DL> + +<P> +Currently, integer range list limits cannot be specified in binary +notation (use decimal, hex, or octal instead). Please contact us if +this is a problem. + +<!-- =section funfilters SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 17, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/funcalc.sed b/funtools/doc/funcalc.sed new file mode 100644 index 0000000..d865a5d --- /dev/null +++ b/funtools/doc/funcalc.sed @@ -0,0 +1,9 @@ +# cc <foo.c> 2>&1 | sed -n -f funauto.sed +# Linux gcc (and others) +s/.*`\([^)]*\)' undeclared (first use in this function).*/\1/p +# Solaris cc +s/.*undefined symbol: \([^)]*\)/\1/p +# Dec Alpha/OSF cc +s/.*In this statement, "\([^)]*\)" is not declared.*/\1/p +# SGI cc +s/.*The identifier "\([^)]*\)" is undefined.*/\1/p diff --git a/funtools/doc/funtools.ds9 b/funtools/doc/funtools.ds9 new file mode 100644 index 0000000..f445a36 --- /dev/null +++ b/funtools/doc/funtools.ds9 @@ -0,0 +1,83 @@ +# +# Analysis command descriptions: +# menu label/description +# file templates for this command (* for all files) +# "menu" (add to menu) |"bind" (bind to key) +# analysis command line +# + +hmenu Funtools + +# 1D histogram params for a table +param thist +column entry "Enter column name to histogram" pi "(histo)" +bins entry "Enter [min:max:]bins" 0 "('0' for default number of bins)" +norm checkbox "Normalize by binsize?" 0 +end + +# 1D histogram params for an image +param ihist +bins entry "Enter [min:max:]bins" 0 "('0' for default number of bins)" +end + +# Light Curve params +param ltc +bins entry "Enter [min:max:]bins" 0 "('0' for default number of bins)" +norm checkbox "Normalize by time?" 0 +end + +Funtools Overview +* +menu +echo "Funtools Tasks in DS9\r\rThe tasks in this section of the Analysis menu utilize the Funtools programs\r'funcnts' and 'funhist'. With the exceptions noted below, they all work off the \roriginal disk-based FITS file. The funcnts-based tasks (radial profile, counts\rin regions) present their results using WCS units, if the appropriate info is\rpresent in the FITS header. \r\rFor situations where a disk file is not available (e.g., image data generated\rby a program and sent to DS9's 'fits' XPA access point), alternate versions of\rthe radial profile and counts in regions tasks are offered that work from DS9's\rinternal image data. In these two cases, results are presented in pixels. Aside\rfrom the units, the results should be identical to the file-based results.\r\rFor more sophisticated analysis, please consider using the underlying Funtools\rtasks directly. These are documented at:\r\r http://hea-www.harvard.edu/RD/funtools/programs.html\r\r" | $text + +Radial Profile Plot (annulus regions; options: none) +* +menu +funds9 funcnts_plot "$xpa" "$filename" "$regions(source,,)" "$regions(background,,)" | $plot(stdin) + +Histogram Plot (any column; options: column name, number of bins) +* +menu +$param(thist); funds9 funhist_plot "$xpa" "$filename[$regions]" "$norm" "$column" "$bins" | $plot(stdin) + +Light Curve Plot ("time" column; options: number of bins) +* +menu +$param(ltc); funds9 funhist_plot "$xpa" "$filename[$regions]" "$norm" time "$bins" | $plot(stdin) + +Energy Spectrum Plot ("pi" column; options: none) +* +menu +funds9 funhist_plot "$xpa" "$filename[$regions]" "0" "pi" "$bins" | $plot(stdin) + +Counts in Regions (any regions; options: none) +* +menu +funds9 funcnts "$xpa" "$filename" "$regions(source,,)" "$regions(background,,)" | $text + +Column Histogram (any column; options: colname, <min:max:>bins) +* +menu +$param(thist); funds9 funhist "$xpa" "$filename[$regions]" "$norm" "$column" "$bins" | $text + +Image Histogram (options: <min:max:>bins) +* +menu +$param(ihist); $data | funhist "stdin[$regions]" xy "$bins" | $text + +--- + +Radial Profile Plot in Pixels using DS9's Internal Image Data (annulus regions) +* +menu +$data | funcnts -rpG stdin $regions(source,,) $regions(background,,) | funcnts.plot -file "$filename" ds9 | $plot(stdin) + +Counts in Regions in Pixels using DS9's Internal Image Data (any regions) +* +menu +$data | funcnts -rp stdin $regions(source,,) $regions(background,,) | $text + +--- + +end diff --git a/funtools/doc/funtools.pdf b/funtools/doc/funtools.pdf Binary files differnew file mode 100644 index 0000000..56575ac --- /dev/null +++ b/funtools/doc/funtools.pdf diff --git a/funtools/doc/funtools.ps b/funtools/doc/funtools.ps new file mode 100644 index 0000000..210149e --- /dev/null +++ b/funtools/doc/funtools.ps @@ -0,0 +1,11446 @@ +%!PS +%%Title: The Funtools Help Facility +%%Creator: html2ps version 1.0 beta5 +%%EndComments +save +2000 dict begin +/d {bind def} bind def +/D {def} d +/t true D +/f false D +/FL [/Times-Roman +/Times-Italic +/Times-Bold +/Times-BoldItalic +/Courier +/Courier-Oblique +/Courier-Bold +/Courier-BoldOblique +/Helvetica +/Helvetica-Oblique +/Helvetica-Bold +/Helvetica-BoldOblique] D +/WF t D +/WI 0 D +/F 1 D +/IW 471 F div D +/IL 621 F div D +/PS 791 D +/EF [0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 2 2] D +/EZ [11 9 19 17 15 13 12 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 8 8] D +/Ey [0 0 2 2 2 2 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] D +/EG [-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1] D +/Tm [1 1 0.8 0.8 0.8 0.8 0.8 0.8 0 0 0 0 0 0 0.5 1 1 1 1 0 0 1.3 0 0] D +/Bm [1 1 0.5 0.5 0.5 0.5 0.5 0.5 0 0 0 0 0 0 0.5 1 1 1 1 0 0 1 0 0] D +/Lm [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 0 0 2 0 0 0] D +/Rm [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0] D +/EU [-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0] D +/NO t D +/YY [[{()}1][{()}0][{()}2]] D +/ZZ [[{()}1][{()}0][{()}2]] D +/Ts EZ 0 get D +/TU f D +/Xp t D +/AU t D +/SN 0 D +/Cf f D +/Tp f D +/Fe f D +/TI 1 Ts mul D +/Fm 14 D +/xL 71 D +/xR 71 D +/yL 706 D +/yR 706 D +/Wl 471 F div D +/Wr 471 F div D +/hL 621 F div D +/hR 621 F div D +/FE {newpath Fm neg Fm M CP BB IW Fm add Fm L IW Fm add IL Fm add neg L CP BB + Fm neg IL Fm add neg L closepath} D +/LA {PM 0 eq{/IW Wl D /IL hL D}{/IW Wr D /IL hR D}ie /W IW D /LL W D /LS W D + TU PM 0 eq and{IW 56 F div add SA{Sf div}if 0 translate} + {PM 0 eq{xL yL}{xR yR}ie translate F SA{Sf mul}if dup scale + CS CF FS Cf{CA CL get VC}if /Bb f D}ie 0 0 M + TF not Tc or {Cf{gsave SA{1 Sf div dup scale}if Cb VC FE fill grestore}if}if}D +/Pi 0 Ts mul D +/SG [0.8 1 1] D +/Ab 15 D +/J 0 D +/Tc t D +/NH 6 D +/Nf f D +/Pa f D +/LH 1.2 D +/XR f D +/Xr {/pN E D ( [p ) WB pN WB (] )WB} D +/Db [16#FF 16#FF 16#FF] D +/Dt [16#00 16#00 16#00] D +/eA f D +/Fi f D +/bT f D +/Lc t D +/Dl [16#00 16#00 16#00] D +/LX f D +/Br 0.25 D +/IA ([IMAGE]) D +/DS {/PF f D()WB NL NP()pop RC ZF} D +/Gb f D +/Mb t D +/Hc [16#00 16#00 16#00] D +/Bl 3 D +/MI -15.2 D +/DX (DRAFT) D +/Di 0 D +/Tt 113.385826771654 D +/Th {()2 Al()BR ( + ) 0 1 -1 H()4 FZ Ti ES()EH ( + ) 0 2 -1 H() ME 0 get join EH()Ea()BR()} D +/tH {()0 1 -1 H (Table of Contents) EH()} D +/FD 2 D +/Dy 2 D +/cD [16#F0 16#F0 16#F0] D +/FW 0.6 D +/FU [16#00 16#00 16#00] D +/ET {/RM f D /A0 0 D /PN SN D /OU t D /Ou t D /W IW D /LL W D D1 + Ms not TP and{Ip}if /TF f D} D +[{true statusdict/setduplexmode get exec} stopped cleartomark +%-- End of variable part -- +/MySymbol 10 dict dup begin + /FontType 3 D /FontMatrix [.001 0 0 .001 0 0 ] D /FontBBox [25 -10 600 600] D + /Encoding 256 array D 0 1 255{Encoding exch /.notdef put}for + Encoding (e) 0 get /euro put + /Metrics 2 dict D Metrics begin + /.notdef 0 D + /euro 651 D + end + /BBox 2 dict D BBox begin + /.notdef [0 0 0 0] D + /euro [25 -10 600 600] D + end + /CharacterDefs 2 dict D CharacterDefs begin + /.notdef {} D + /euro{newpath 114 600 moveto 631 600 lineto 464 200 lineto 573 200 lineto + 573 0 lineto -94 0 lineto 31 300 lineto -10 300 lineto closepath clip + 50 setlinewidth newpath 656 300 moveto 381 300 275 0 360 arc stroke + -19 350 moveto 600 0 rlineto -19 250 moveto 600 0 rlineto stroke}d + end + /BuildChar{0 begin + /char E D /fontdict E D /charname fontdict /Encoding get char get D + fontdict begin + Metrics charname get 0 BBox charname get aload pop setcachedevice + CharacterDefs charname get exec + end + end}D + /BuildChar load 0 3 dict put /UniqueID 1 D +end +definefont pop + +/Cd {aload length 2 idiv dup dict begin {D} repeat currentdict end} D +/EX {EC cvx exec} D +/DU {} d +/BB {pop pop}d +/ie {ifelse} d +/E {exch} d +/M {moveto} d +/R {rmoveto} d +/L {lineto} d +/RL {rlineto} d +/CP {currentpoint} d +/SW {stringwidth} d +/GI {getinterval} d +/PI {putinterval} d +/Sg {setgray} d +/LW {setlinewidth} d +/S {dup () ne OU and{0 Co R AT 3 eq LB and HF not and A1 0 ne A2 0 ne or and + {A2 0 32 A1 0 6 -1 roll awidthshow}{show}ie 0 Co neg R}{pop}ie + OU PH 3 eq or{/Ms t D}if} D +/U {OU{gsave CP currentfont /FontInfo get /UnderlinePosition get + 0 E currentfont /FontMatrix get dtransform E pop add newpath M dup SW pop + CJ 0 RL stroke grestore}if} D +/B {OU Br 0 gt and{CP Ts neg Ts .33 mul R gsave 0 Sg + CP newpath Ts Br mul 0 360 arc closepath UI 2 mod 0 eq{stroke}{fill}ie + grestore M CP E Ts Br 1 add mul sub E BB /Ms t D}if}D +/NP {Ms TP not or PA and OU and{TP{OR}if f1{mF k2 /mF E D /YC 0 D}if + TP TU not PM 0 eq or and{showpage}if DU Ip TE not{LA}if 0.6 LW + /CI 0 D /TP t D /Hs f D /hl 6 D /Hv 6 D /HI hi D /Ms f D}if Bs XO BO M} D +/Np {LE sub CP E pop gt PL 0 eq and{NP}if}D +/Ip {/PN PN 1 add D /Pn RM{1}{4}ie PN Ns D /PM PN SN sub 2 mod D} D +/GP {E dup 3 -1 roll get PN 1 add 2 mod get dup type /integertype eq + {get 0 get}{E pop}ie}d +/Fc {dup 2 GP exec SW pop /S1 E D dup 1 GP exec SW pop /S2 E D 0 GP exec SW + pop /S3 E D S1 0 gt{S2 2 mul S1 add S3 2 mul S1 add 2 copy lt{E}if pop}{0}ie + S2 S3 add 2 copy lt{E}if pop IW .9 mul div dup 1 gt{1 E div}{pop 1}ie}D +/OR {Df{Sd}if tp not{gsave SA{1 Sf div dup scale}if Fe{Cf{FU VC}if FW LW + 1 setlinejoin FE stroke}if /YO {60 F div dup 40 gt{pop 40}if}D /cs CS D + /cf CF D /CF 0 D /pf PF D /PF f D /Fn FN D /At AT D /AT 0 D /FN EF Hf 1 add + get D Fz Fs FS ZZ Fc Fz mul Fs FS EU Hf 1 add get dup type /arraytype eq + Cf and{VC}{pop 0 Sg}ie IW IL neg YO sub M ZZ 1 GP exec dup SW pop neg 0 R Sh + 0 IL neg YO sub M ZZ 0 GP exec Sh ZZ 2 GP exec dup SW pop IW E sub 2 div + IL neg YO sub M Sh Fz Fs FS NO{/AW IW Pn SW pop sub D AW 2 div IL neg YO sub + S1 0 gt S2 AW .45 mul gt or S3 AW .45 mul gt or{Fz 2 mul sub}if M Pn Sh}if + EU Hf get dup type /arraytype eq Cf and{VC}{pop 0 Sg}ie YY Fc /FN EF Hf get D + Hz mul HS FS IW YO M YY 1 GP exec dup SW pop neg 0 R Sh 0 YO M YY 0 GP exec Sh + YY 2 GP exec dup SW pop IW E sub 2 div YO M Sh /FN Fn D /AT At D t Pb XO SZ + SL get neg R /PF pf D grestore /CF 0 D cs cf FS}if}D +/Sh {dup () ne{CP Hz 4 div sub BB show CP CS add BB}{pop}ie}D +/Pb {/OU E D /Ou OU D /PB t D 0 0 M Ba{/Sa save D /BP t D /Fl t D RC /PL 0 D + /PH 0 D /W IW D /LE IL .7 mul D /EO 0 D SI ZF /YA 0 D /BO 0 D /C1 () D + BA 0 Ts neg R Bb{Xl Yl Xh Yh}if Bb CP Sa restore M + {/Yh E D /Xh E D /Yl E D /Xl E D}if /Fl t D}if + BL /OU t D /HM f D /Ou t D /PB f D} D +/Bs {/BP Ba not D}D +/reencodeISO { + dup dup findfont dup length dict begin{1 index /FID ne{D}{pop pop}ie}forall + /Encoding ISOLatin1Encoding D currentdict end definefont} D +/ISOLatin1Encoding [ +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/space/exclam/quotedbl/numbersign/dollar/percent/ampersand/quoteright +/parenleft/parenright/asterisk/plus/comma/hyphen/period/slash +/zero/one/two/three/four/five/six/seven/eight/nine/colon/semicolon +/less/equal/greater/question/at/A/B/C/D/E/F/G/H/I/J/K/L/M/N +/O/P/Q/R/S/T/U/V/W/X/Y/Z/bracketleft/backslash/bracketright +/asciicircum/underscore/quoteleft/a/b/c/d/e/f/g/h/i/j/k/l/m +/n/o/p/q/r/s/t/u/v/w/x/y/z/braceleft/bar/braceright/asciitilde +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef/.notdef +/.notdef/space/exclamdown/cent/sterling/currency/yen/brokenbar +/section/dieresis/copyright/ordfeminine/guillemotleft/logicalnot +/hyphen/registered/macron/degree/plusminus/twosuperior/threesuperior +/acute/mu/paragraph/periodcentered/cedilla/onesuperior/ordmasculine +/guillemotright/onequarter/onehalf/threequarters/questiondown +/Agrave/Aacute/Acircumflex/Atilde/Adieresis/Aring/AE/Ccedilla +/Egrave/Eacute/Ecircumflex/Edieresis/Igrave/Iacute/Icircumflex +/Idieresis/Eth/Ntilde/Ograve/Oacute/Ocircumflex/Otilde/Odieresis +/multiply/Oslash/Ugrave/Uacute/Ucircumflex/Udieresis/Yacute +/Thorn/germandbls/agrave/aacute/acircumflex/atilde/adieresis +/aring/ae/ccedilla/egrave/eacute/ecircumflex/edieresis/igrave +/iacute/icircumflex/idieresis/eth/ntilde/ograve/oacute/ocircumflex +/otilde/odieresis/divide/oslash/ugrave/uacute/ucircumflex/udieresis +/yacute/thorn/ydieresis +] D +[128/backslash 129/parenleft 130/parenright 141/circumflex 142/tilde +143/perthousand 144/dagger 145/daggerdbl 146/Ydieresis 147/scaron 148/Scaron +149/oe 150/OE 151/guilsinglleft 152/guilsinglright 153/quotesinglbase +154/quotedblbase 155/quotedblleft 156/quotedblright 157/endash 158/emdash +159/trademark] +aload length 2 idiv 1 1 3 -1 roll{pop ISOLatin1Encoding 3 1 roll put}for +/colorimage where{pop}{ + /colorimage { + pop pop /Pr E D {/Cv Pr D /Gr Cv length 3 idiv string D 0 1 Gr length 1 sub + {Gr E dup /i E 3 mul D Cv i get 0.299 mul Cv i 1 add get 0.587 mul add + Cv i 2 add get 0.114 mul add cvi put}for Gr} image} D +}ie +/pdfmark where{pop}{userdict /pdfmark /cleartomark load put}ie +WF{FL{reencodeISO D}forall}{4 1 FL length 1 sub{FL E get reencodeISO D}for}ie +/Symbol dup dup findfont dup length dict begin + {1 index /FID ne{D}{pop pop}ie}forall /Encoding [Encoding aload pop] + dup 128 /therefore put D currentdict end definefont D + +/SF {/CS E D SZ SL CS put FO SL FN put /YI CS LH neg mul D dup ST cvs ( ) join + CS ST cvs join C1 E join ( NF ) join /C1 E D CS NF /Wf WF FN 0 gt or D + /BW Wf{( ) SW pop}{0}ie D}D +/NF {/cS E D /cF E D cF 0 ge{FL cF get}{cF -1 eq{/Symbol}{/MySymbol}ie}ie + findfont cS scalefont setfont} D +/FS {CF or /CF E D FR SL CF put CF CF 0 ge{FN 4 mul add}if E SF} D +/PC {SH /BP f D fin not GL not and{NL}if /HM t D /LL LS D} D +/BS {/TX E D Wf{/fin f D /CW 0 D /LK 0 D /SC 0 D + /RT TX D {RT ( ) search{/NW E D pop /RT E D /WH NW SW pop D CW WH add LL gt + {TX SC LK SC sub 1 sub NN GI GL{SH cF cS OC + 2 copy cS ne E cF ne or{NF}{pop pop}ie}{PC /CW WH BW add D}ie + /SC LK D} + {GL{JC}if + /CW CW WH add BW add D /HM t D}ie /GL f D /Ph f D + /LK LK NW length 1 add add D}{pop exit}ie}loop + /fin t D TX SC LK SC sub GI SH RT () ne{GL not{CC}if}if + /LC TX length D /WH RT SW pop D CW WH add Hy{HC SW pop add}if LL gt + {RT GL{SH cF cS OC 2 copy cS ne E cF ne or{NF}{pop pop}ie + Hy{/Ph t D}if /LL LS D}{NL /LL LS D SH}ie} + {RT PC Hy{CC}if /Ph Ph Hy or D}ie RT () ne{/GL t D /HM t D}if} + {TX SW pop LL le{TX SH}{/NW () D 0 2 TX length 1 sub + {/CW E D TX 0 CW GI dup SW pop LL gt{pop NW SH /HM t D NL/LL W XO sub MR sub D + /CW CW 2 sub NN D /TX TX CW TX length CW sub GI D TX BS exit} + {/NW E D}ie}for}ie}ie /HM t D}D +/CC {C0 length 0 gt{JC}if /C0 [C1 L1 YA YB Mf NS NB TB AF Bw] D + /C1 () D /L0 L1 D /YA 0 D /YB 0 D /Mf 0 D /NS 0 D /NB 0 D}D +/JC {C0 aload length 0 gt{pop pop pop NB add /NB E D NS add /NS E D + dup Mf gt{/Mf E D}{pop}ie dup YB gt{/YB E D}{pop}ie + dup YA gt{/YA E D}{pop}ie pop C1 join /C1 E D /C0 [] D}if}D +/OC {C0 length 0 gt{C1 L1 L0 sub YA YB Mf NS NB TB AF Bw GL C0 aload pop + /Bw E D /AF E D /TB E D /NB E D /NS E D /Mf E D /YB E D /YA E D /C0 [] D + /L1 E D /C1 E D Ph{HC SH}if NL /GL E D /Bw E D /AF E D /TB E D /NB E D /NS E D + /Mf E D /YB E D /YA E D /L1 E D /LL W L1 sub XO sub MR sub WH sub D /CW 0 D + C1 E join /C1 E D}if}D +/BT {/LB t D dup length string copy RS dup dup () ne E ( ) ne and + {/CI 0 D /LS LL D /LL W L1 sub XO sub MR sub D BS} + {dup ( ) eq{/GL f D}if dup () eq L1 0 eq or{pop}{SH /BP f D /Ph f D}ie}ie + /LB f D} D +/BL {CP E pop XO E M} D +/NL {JC /GL f D /SK W XO sub MR sub L1 sub TB{Bw add}if D + /YA LF{Mf HM Fl not and PF or{LH mul}if}{0 /LF t D}ie YA 2 copy lt{E}if pop D + C1 () ne{/FB YB Mf SA{Sf mul}if 4 div 2 copy lt{E}if pop D}if Fl{/Ya YA D}if + CP E pop YA sub YB sub LE neg lt Fl not and PB not and{NP}if NT TL BL + OU PF not and PB or{/RE L1 TB{Bw sub}if + W XO sub MR sub div YA YB add LE BO add div 2 copy lt{E}if pop D + RE 1 gt{BL 1 RE div dup scale}if}if + AT 2 le{SK AT mul 2 div YA neg R}if + AT 3 eq{0 YA neg R TB{/NB NB 1 sub D /NS NS 1 sub D}if /NB NB 1 sub NN D + /A3 NS 6 mul NB add D NS NB add 0 eq + {/A1 0 D /A2 0 D} + {NS 0 eq{/A1 SK NB div dup J gt{pop 0}if D /A2 0 D}{J A3 mul SK lt + {/A1 J D /A2 SK J NB mul sub NS div dup Ab gt{/A1 0 D pop 0}if D} + {/A1 SK A3 div D /A2 A1 6 mul D}ie}ie}ie /A1 A1 NN D /A2 A2 NN D}if + AT 4 eq{0 YA neg R PH 2 le{PD 0 lt{/PD L1 D}if PD M1 gt{/M1 PD D}if + L1 PD sub M2 gt{/M2 L1 PD sub D}if}{DV ID 1 sub get 0 ge{Lo 0 R}if}ie}if + F0 cF ne Cs cS ne or{F0 Cs NF}if + /ms Ms D /Ms f D CP FB sub + C1 cvx exec XO EO sub L1 add TB{BW sub}if dup LM gt{/LM E D}{pop}ie + PH 0 eq PH 4 eq or Ms and{HF not{/PO t D /AH t D}if + BB CP YA add E AT 3 eq LB and{A1 sub}if TB{BW sub}if E BB} + {pop pop}ie Ms HM PH 3 eq and or{/BP f D /Fl f D}if + /Lo 0 D /L1 0 D /F0 cF D /Cs cS D BP not{0 YB NN neg R}if + OU f1 and mF not and{k2 /f1 f D}if + OU PF not and PB or{RE 1 gt{RE dup scale}if}if /Ms ms Ms or D + /C1 AF{(Cp )}{()}ie D /YA 0 D /YB 0 D BL + AT 4 eq LB not and PH 3 ge and + {ID DV length lt{DV ID get dup 0 ge{DO E sub /Lo E D /L1 Lo D}{pop}ie + /ID ID 1 add D}if}if /T t D CD{/LN LN 1 add D PD}if + /PD -1 D /NS 0 D /NB 0 D /TB f D /Ph f D /Mf 0 D /HM f D} D +/RS {/TM E D /CN 0 D TM{10 eq{TM CN ( ) PI}if /CN CN 1 add D}forall + /CN 0 D /BK HM EN and{0}{1}ie D TM + {dup 32 ne{TM CN 3 2 roll put /CN CN 1 add D /BK 0 D} + {pop BK 0 eq{TM CN 32 put /CN CN 1 add D}if /BK 1 D}ie}forall + TM 0 CN GI dup dup () ne E ( ) ne and + {dup CN 1 sub get 32 eq{/EN f D}{/EN t D}ie}if} D +/join {2 copy length E length add string dup 4 2 roll 2 index 0 3 index + PI E length E PI}d +/WR {(\n) search{dup () ne BP not or + {Li 4 le CP E pop YI Li mul add LE add 0 lt and PL 0 eq and{NP}if + SH NL pop /Li Li 1 sub D WR}{pop pop WR}ie}{SH}ie /CI 0 D /BP f D} D +/SH {dup dup () ne E ( ) ne and PF or CS Mf gt and{/Mf CS D}if + T not Wf and{( ) E join /T t D}if dup BP{/MF CS D}if + AT 3 eq{2 copy length dup 0 gt{/NB E NB add D + {( ) search{/NS NS 1 add D pop pop}{pop exit}ie}loop}{pop pop}ie}if + CD PD 0 lt and{dup DC search{SW pop /PD E L1 add D pop pop}{pop}ie}if + 0 Np dup SW pop L1 add /L1 E D dup () ne + {C1 (\() join E join (\)) join AU AF and UF or Wf and{( U ) join}if + sF{( s ) join}if ( S ) join + /C1 E D dup length 1 sub get 32 eq /TB E D /Bw BW D}{pop pop}ie} D +/BG {AI LG BC add add 0 eq} D +/ON {OU{Ty AR AI NN get dup 1 add Ln Ns Ty 2 mod 0 eq{(. )}{(\) )}ie join + dup SW pop neg 0 R CP E 0 lt{0 E M}{pop}ie CP BB show /Ms t D}if} D +/Ln {AR AI 3 -1 roll put}D +/SP {dup CI lt BP not and{dup CI sub 0 E R /CI E D}{pop}ie} D +/BN {PF{WR /HM f D}{BT NL}ie} D +/NN {dup 0 lt{pop 0}if} D +/h {(h) HI ST cvs join cvx exec dup 1 get E Nf{0 get E join}{pop}ie} D +/H {/fn FN D /Hi E 1 add D 1 sub /HL E D /H2 HL 2 add D /GS EZ H2 get D + E Tm H2 get GS mul BE dup 0 gt{1 sub}{pop EG H2 get dup 0 lt{pop AT}if}ie NA + WW Np /SL SL 1 add D /FN EF H2 get D GS Ey H2 get FS + EU H2 get Sc Hs not HL Hl lt and Hs HL hl lt and or Hi 0 eq or + {/HI Hi D /Hs t D /hl HL D /Hv HL D}if HL Hl lt{/hi Hi D}if + Nf HI 0 gt and{(h) Hi ST cvs join cvx exec 0 get WB}if + /HF t D /AH f D /PO f D} D +/EH {Bm H2 get GS mul BE OA /SL SL 1 sub NN D /CF 0 D /FN fn D + SZ SL get FR SL get FS /HF f D /GS Ts D ()Ec} D +/P {E PF{WR}{PO{EP}{BN}ie Ts 4 mul Np AE not{Tm 0 get Ts mul neg SP}if + dup 0 ge AH and{Pi Pd}if}ie 1 sub dup 0 lt{pop AV AL get}if /AT E D /PO t D} D +/EP {PF{WR}{BN Ts 4 mul Np}ie AE not{Bm 0 get Ts mul neg SP}if + /AT AV AL get D /PO f D} D +/BE {E PO{EP}{BN}ie Ts 4 mul Np neg SP} D +/HR {/Aw W EO sub D /RW E dup 0 gt{Aw mul}{neg}ie dup Aw gt{pop Aw}if D /RZ E D + E BN Ts neg SP 1 sub 2 div Aw RW sub mul EO add CP E pop M PF{0 Ps neg R}if + 0 Np OU{gsave RZ LW Cf{Hc VC}{0 Sg}ie CP BB RW 0 RL CP BB stroke grestore}if + /CI 0 D /BP f D PF not{Ts neg SP}if /Ms t D} D +/AD {I NL EG 14 get dup 0 lt{pop AT}if NA /AE t D Tm 14 get Ts mul neg SP + Cf{EU 14 get dup -1 eq{pop CA CL get}if Sc}if} D +/DA {BN ()ES OA /AE f D ()Ec Bm 14 get Ts mul neg SP} D +/PR {/MW E D /Li E D Tm 1 get Ps mul BE 0 NA /FN Fp D /PF t D SI /SL SL 1 add D + /CF 0 D Ps CS mul Ts div MW WC mul CS mul Ts div dup LL gt PL 0 eq and + {LL div div}{pop}ie Ey 1 get FS CP E pop LE add YI neg div cvi dup Li lt + AH and{4 lt YI Li mul 5 mul LE add 0 gt or PL 0 eq and{NP}if}{pop}ie + EU 1 get Sc /GS Ps D}D +/RP {WR NL () /PF f D SI /FN 0 D ES Bm 1 get Ps mul neg SP OA /GS Ts D} D +/SI {/XO Lm 15 get BC NN mul Lm 16 get AI UI sub NN mul add + Lm 17 get UI NN mul add Lm 20 get LG NN mul add Ts mul + PF{Lm 1 get Ps mul add}if EO add D + /MR Rm 15 get BC NN mul Rm 16 get AI UI sub NN mul add + Rm 17 get UI NN mul add Rm 20 get LG NN mul add Ts mul + PF{Rm 1 get Ps mul add}if D /LL W XO sub MR sub D} D +/DT {/cC E D BN /LG LG 1 sub D SI /LG LG 1 add D WW 2 div Np BL} D +/DD {WB Cc 0 eq cC 0 eq and L1 0 eq or Lm 20 get Ts mul L1 sub TB{BW add}if + Ts 2 div lt or NL /LF E D SI BL /cC 0 D} D +/DL {Dc LG Cc put /Cc E D BG{Tm 18 get Ts mul BE}{BN}ie /LG LG 1 add D BL} D +/LD {BN LG 0 gt{/LG LG 1 sub D}if /Cc Dc LG get D SI + BG{()Bm 18 get Ts mul BE}if BL} D +/UL {BG{Tm 17 get Ts mul BE}{BN}ie NR AI NN 0 put /UI UI 1 add D + /AI AI 1 add D SI BL} D +/LU {BN /UI UI 1 sub D /AI AI 1 sub D SI BG{()Bm 17 get Ts mul BE}if BL} D +/OL {E BG{Tm 16 get Ts mul BE}{BN}ie TR AI NN Ty put /Ty E D NR AI NN 1 put + /AI AI 1 add D SI BL 1 Ln} D +/LO {BN /AI AI 1 sub D /Ty TR AI get D SI BG{()Bm 16 get Ts mul BE}if BL} D +/LI {E BN -1 SP /BP f D /CI 0 D 0 Np NR AI 1 sub NN get 1 eq + {dup dup 0 gt E 4 le and{/Ty E D}{pop}ie + /L1 L1 Ty AR AI NN get Ns SW pop XO sub dup 0 lt{pop 0}if add D ( ON )} + {pop ( B )}ie C1 E join /C1 E D CS Mf gt{/Mf CS D}if BL} D +/BQ {Tm 15 get Ts mul BE /BC BC 1 add D SI BL} D +/QB {Bm 15 get Ts mul BE /BC BC 1 sub D SI BL} D +/Al {E EP 1 sub dup 0 lt{pop AV AL get}if NA} D +/Ea {EP OA} D +/WB {PF{WR}{BT}ie} D +/F1 {WB /FN 0 D CS 0 FS} D +/F2 {WB /FN WI D CS 0 FS} D +/HY {/Hy t D WB /Hy f D} D +/YH {WB} D +/A {/LT E D LT 1 eq{/RN E D}if /Lh E D WB /C1 C1 ( Cp ) join D + Lc AF not and{Cl Sc}if /AF t D} D +/EA {Lc AF and{Ec}{WB}ie TL Pa AF and Lh 0 ne and + {( \() Lh join (\)) join /AF f D WB}if /AF f D} D +/TL {C1 ( Tl ) apa /C1 E D} d +/apa {AF OU and Lh 0 ne LT 1 eq or and{LT 1 eq{RN ( /) E ST cvs join} + {(\() Lh join (\)) join}ie E join join}{pop}ie} d +/Cp {/Xc CP /Yc E D D} D +/SS {Cf{dup 0 ge{EU E get dup -1 eq{pop CA CL get}if}{pop CA CL get}ie Sc} + {pop}ie SZ SL get /SL SL 1 add D} D +/I {WB 8 SS 1 FS} D +/EM {WB 8 SS /CF CF 1 xor D 0 FS} D +/BD {WB 9 SS 2 FS} D +/TT {WB 10 SS /FN Fp D 0 FS} D +/KB {WB 11 SS /FN Fp D 2 FS} D +/CT {WB 12 SS 1 FS} D +/SM {WB 13 SS /FN Fp D 0 FS} D +/Q {/QL QL 1 add D QO QL 2 mod get La get join WB} D +/EQ {QC QL 2 mod get La get join WB /QL QL 1 sub D} D +/RO {WB -1 SS /CF 0 D 0 FS} D +/SY {WB -1 SS -1 FS} D +/MY {WB -1 SS -2 FS} D +/ES {WB /SL SL 1 sub NN D /CF 0 D /FN FO SL get D SZ SL get FR SL get FS ()Ec}D +/FZ {3 sub 1.2 E exp GS mul E WB TL /C1 C1 ( Cp ) join D /SL SL 1 add D 0 FS} D +/Ef {WB TL ()ES /C1 C1 ( Cp ) join D} D +/BZ {dup /Bf E D FZ}D +/Sc {dup -1 ne Cf and{/CL CL 1 add D dup 0 eq{pop [0 0 0]}if + dup CA E CL E put VS ( VC ) join C1 E join /C1 E D}{pop}ie} D +/Ec {WB Cf{/CL CL 1 sub NN D CA CL get VS ( VC ) join C1 E join /C1 E D}if} D +/VS {dup type /arraytype eq{([) E {ST cvs join ( ) join}forall (]) join}if} D +/VC {{255 div}forall setrgbcolor} D +/Sl {dup type /integertype ne{Ds}if /La E D WB}d +/UN {WB /UF t D} D +/NU {WB /UF f D} D +/SE {WB /sF t D} D +/XE {WB /sF f D} D +/sM {/C1 C1 ( k1 ) join D}d +/eM {/C1 C1 ( k2 ) join D}d +/k1 {/YC CP E pop Ts add D /mF t D /f1 t D}d +/k2 {gsave 3 LW -9 CP E pop Ts 0.2 mul sub M -9 YC L stroke grestore /mF f D}d +/Ac {/AC E D WB}d +/Ca {eA{( \()join AC join(\) )join}if WB}d +/s {OU{gsave 0 CS .25 mul R dup SW pop CJ 0 RL stroke grestore}if}D +/CJ {AT 3 eq LB and{E dup dup length 1 sub A1 mul E + {( ) search{pop pop E A2 add E}{pop exit}ie}loop 3 -1 roll add + W CP pop sub 2 copy gt{E}if pop}if}D +/So {/Co E D} D +/SO {C1 Yo ST cvs join ( So ) join /C1 E D (j) SW pop 2 div Pd} D +/Se {E WB CS E div Pd}D +/Pd {dup type /stringtype eq{SW pop}if dup /L1 E L1 add D + ST cvs ( 0 R ) join C1 E join /C1 E D} D +/Sp {0.35 CO} D +/Sb {-0.2 CO} D +/CO {OV Io Yo put /Yo E CS mul Yo add D /Io Io 1 add D -1.5 Io mul 3 add FZ SO + CS Yo add dup YA gt{/YA E D}{pop}ie + Yo neg dup YB gt{/YB E D}{pop}ie} D +/Es {ES /Io Io 1 sub NN D /Yo OV Io get D SO} D +/SB {/N2 0 D 0 1 NI{/N E D{IX N2 get 0 lt{/N2 N2 1 add D}{exit}ie}loop + /K WS N get FC N get mul D /NY AY N2 get D /BV NY array D + 0 1 NY 1 sub{/TM K string D currentfile TM readhexstring pop pop BV E TM put} + for BM N BV put /N2 N2 1 add D}for} D +/IC [{/MA E D /MB 0 D}{2 div /MA E D /MB MA D}{/MB E CS sub D /MA CS D} + {pop /MA YS AB mul D /MB 1 AB sub YS mul D}{pop /MA 0 D /MB 0 D}] D +/IP {BV N get /N N 1 add D} D +/II {/K E D IX K get 0 lt{/EC E D}if /TY E D + TY 4 eq{/Y E D /X E D}if TY 3 eq{/AB E D}if + /XW AX K get D /YW AY K get D /IS SG IT K get get D /XS XW IS mul D + /YS YW IS mul D YS IC TY get exec /MA MA Fl not{3 add}if D} D +/IM {II /ty TY D /xs XS D /ys YS D /ya YA D /yb YB D /ma MA D /mb MB D /k K D + /ec EC D /BP f D /CI 0 D WB TL L1 xs add dup XO add MR add W gt + {pop /ma ma Fl{3 add}if D NL /YA ma D /YB mb D /YS ys D /L1 xs D} + {/L1 E D ma YA gt{/YA ma D}if mb YB gt{/YB mb D}if}ie /TB f D + OU{CP E pop YS sub LE neg lt Fl not and PB not and{NP /YA ma D /YB mb D}if + /BP f D ty ST cvs ( ) join IX k get 0 lt{(\() join ec join (\) ) join}if + k ST cvs join ty 3 eq{AB ST cvs ( ) join E join}if + ty 4 eq{X ST cvs ( ) join Y ST cvs join ( ) join E join}if C1 E join + ( DI ) join FP 2 eq FP 1 eq AF and or{( FM ) join}if + ( Il Cp ) apa /C1 E D /EN f D}if /HM t D /T f D} D +/DI {II /Xc CP /Yc E D D /YN YW neg D /HM t D /CI 0 D /K2 IX K get D gsave + TY 4 eq{OX X IS mul add OY FY add YS sub Y IS mul sub} + {/FY YS D CP MB sub 2 copy /OY E D /OX E D}ie + translate K2 0 ge{/DP AZ K2 get D /BV BM K2 get D XS YS scale /N 0 D XW YW DP + [XW 0 0 YN 0 YW] {IP} FC K2 get 1 eq{image}{f 3 colorimage}ie} + {EX}ie grestore XS 0 R /Ms t D} D +/FM {gsave 0 Sg CP MB sub translate XS neg 0 M 0 YS RL XS 0 RL 0 YS neg RL + XS neg 0 RL stroke grestore} D +/NA {/AT E D /AL AL 1 add D AV AL AT put} D +/OA {AL 0 gt{/AL AL 1 sub D /AT AV AL get D}if} D +/D1 {/BR {CP E pop E BN Mb{CP E pop eq{0 YI R}if}{pop}ie} D + /Sn {OU{C1 E ST cvs join ( Ld ) join /C1 E D}{pop}ie} D} D +/D1 {/BR {BN} D /Sn {OU {C1 E ST cvs join ( Ld ) join /C1 E D} {pop} ie} D} D +/TC {/TF t D /ML 0 D HN{SW pop dup ML gt{/ML E D}{pop}ie}forall NP /RM RM not D + RC /OU Tc D Ep /PN 0 D Ms not TP and{Ip}if /W IW ML sub Ts sub D + /A0 0 D TH{/BR {( ) join BT} D /Sn {pop} D /Au () D}if} D +/TN {0 eq{E EA PF HF or not XR and{HN E get Xr}{pop}ie} + {OU{Tn 0 ge{() BN}if /Tn E D}{pop}ie WB}ie} D +/NT {OU LB not and Tn 0 ge and{PL 0 eq{Ms not{CS CF FS}if CP dup + /y E YA sub D W 9 sub CS -1.8 mul XO L1 add 2 add{y M (.) show}for + HN Tn get dup SW pop IW E sub y M show CP BB M}if /Tn -1 D}if} D +/Ld {/DN E D HN DN Pn put [/View [/XYZ -4 Fl{PS}{CP YA add US E pop}ie null] + /Dest DN ST cvs cvn /DEST pdfmark} D +/C {ND 1 eq{1 sub}if TI mul /XO E D NL Nf not{pop()}if 0 3 -1 roll 1 A} D +/OP {BP not{NP}if PN 2 mod 0 eq{/Ms t D NP}if}D +/Ep {Xp PN 2 mod 0 eq and OU and{/Pn (-) D showpage /PM 1 D LA}if}D +/Dg [73 86 88 76 67 68 77] D +/Rd [0 [1 1 0][2 1 0][3 1 0][2 1 1][1 1 1][2 2 1][3 3 1][4 4 1][2 1 2]] D +/Ns {/m E D /c E 32 mul D /j m 1000 idiv D /p j 12 add string D + c 96 le m 0 gt and{c 32 le {/i 0 D /d 77 D /l 100 D /m m j 1000 mul sub D + j -1 1 {pop p i d c add put /i i 1 add D}for + 4 -2 0 {/j E D /n m l idiv D /m m n l mul sub D /d Dg j get D + n 0 gt {/x Rd n get D x 0 get -1 1 {pop p i d c add put /i i 1 add D}for + p i x 1 get sub Dg x 2 get j add get c add put}if /l l 10 idiv D + }for p 0 i GI} + {/i ST length 1 sub D m {1 sub dup 0 ge{dup 26 mod c add 1 add + ST i 3 -1 roll put 26 idiv dup 0 eq{pop exit}if}if /i i 1 sub D}loop + ST i ST length i sub GI}ie} + {m p cvs}ie} D +/US {matrix currentmatrix matrix defaultmatrix matrix invertmatrix + matrix concatmatrix transform} D +/GB {Gb{US}if}D +/Tl {/Rn E D Xc CP pop ne{ + [/Rect [Xc 1 sub Yc cS 0.25 mul sub GB CP E 1 add E cS 0.85 mul add GB] + /Subtype /Link /Border [0 0 Cf Lc and LX and AU or{0}{1}ie] Rn type + /nametype eq {/Dest Rn}{/Action [/Subtype /URI /URI Rn] Cd}ie + /ANN pdfmark}if} D +/Il {/Rn E D [/Rect [Xc Yc GB Xc XS add Yc YS add GB] /Subtype /Link + /Border [0 0 0] Rn type /nametype eq{/Dest Rn} + {/Action [/Subtype /URI /URI Rn] Cd}ie /ANN pdfmark} D +/XP {[{/Z Bz 2 div D Z 0 R Z Z RL Z neg Z RL Z neg Z neg RL Z Z neg RL + Fi cH 1 eq and{fill}if} {Bz 0 RL 0 Bz RL Bz neg 0 RL 0 Bz neg RL + Fi cH 1 eq and{fill}if} {0 -5 R Bz 0 RL 0 21 RL Bz neg 0 RL 0 -21 RL}]} D +/MS {/Sm E D WB}D +/O {BN()Sm BX} D +/O {BN()0 Sm BX} D +/BX {/Bt E D Bt 2 lt{/Ch E D CS 0.8 mul}{11 mul}ie W XO sub MR sub + 2 copy gt{E}if pop /HZ E D Bt 2 eq{Fi not{pop()}if ( )E join /Ft E D TT + /PF t D /MW 1 D /Li 1 D /Fw Ft SW pop D Fw HZ gt{/HZ Fw 8 add D}if + HZ ST cvs( )join}{WB Ch ST cvs( )join}ie L1 HZ add XO add MR add W gt{NL}if + Bt 2 eq{Ft ES Fw neg HM{CS sub}if Pd}if Bt ST cvs join( Bx )join + Bt 2 eq HM and{CS Pd}if C1 E join /C1 E D /L1 L1 HZ add D /T f D + ( ) Pd /PF f D Bt 2 lt{YA CS .8 mul lt{/YA CS .8 mul D}if} + {YB 5 lt{/YB 5 D}if YA 21 lt{/YA 21 D}if}ie /CI 0 D} D +/Bx {dup 2 eq{E /Bz E D}{E /cH E D /Bz CS .8 mul D}ie + OU {gsave 0 Sg XP E get exec stroke grestore}{pop}ie Bz 0 R /Ms t D}D +/SD {FD 4 mul Dy add DZ NF newpath 0 0 M DX t charpath pathbbox + 3 -1 roll sub /DY E D E dup /X1 E D sub WM mul WX DY mul add WM DG mul E div + /DF E D /DR WX DF mul DY mul WM div 2 div D} d +/Sd {gsave 0 IL Di mul neg translate IL IW atan Di 0 eq{neg}if rotate + FD 4 mul Dy add DZ NF DR X1 sub DY 2 div neg M cD VC DX show grestore} d +/Pt {/tp t D Tp{NP /Pn (TP) D 0 Tt neg R Th BN NP Ep ET RC ZF}if /tp f D} D +/RC {/AI 0 D /LG 0 D /BC 0 D /UI 0 D /PF f D /Cc 0 D /cC 0 D /Dc 10 array D + /NR [0 1 9{pop 0}for] D /La Ds D /AR 10 array D /TR 10 array D /AV 30 array D + SI /AL -1 D /AT A0 D AT NA /OV 9 array D /Yo 0 D /Co 0 D /Io 0 D /Hy f D + /Ph f D /CL -1 D Ct Sc}D +/ZF {/FR [0 1 30{pop 0}for] D /SZ [0 1 30{pop 0}for] D /FO [0 1 30{pop 0}for] D + /SL 0 D /CF 0 D /FN 0 D 0 Ts SF}D +/QO [[(\234)(\233)(\253\240)(\232)(\273)(\253)][(')(`)(\253\240)(\231)(\273)(\253)]] D +/QC [[(\234)(\234)(\240\273)(\233)(\253)(\273)][(')(')(\240\273)(`)(\253)(\273)]] D +/Hf EF length 2 sub D +/Hz EZ Hf get D +/HS Ey Hf get D +/Fz EZ Hf 1 add get D +/Fs Ey Hf 1 add get D +/LE IL D +/Ps EZ 1 get D +/Fp EF 1 get D +/XO 0 D +/YI 0 D +/CI 0 D +/FP 0 D +/WW Ts 7 mul D +/Mf 0 D +/YA 0 D +/YB 0 D +/Cs Ts D +/GS Ts D +/F0 0 D +/NS 0 D +/NB 0 D +/N 0 D +/C0 [] D +/C1 () D +/Lo 0 D +/L1 0 D +/LM 0 D +/PH 0 D +/EC 0 D +/Lh 0 D +/LT 0 D +/CH 1 string D +/ST 16 string D +/CA 9 array D +/HC (\255) D +/HM f D +/PF f D +/EN f D +/TB f D +/UF f D +/sF f D +/AE f D +/AF f D +/BP t D +/CD f D +/PA t D +/GL f D +/T t D +/HF f D +/AH f D +/SA f D +/PB f D +/f1 f D +/mF f D +/OX 0 D +/OY 0 D +/FY 0 D +/EO 0 D +/FB 0 D +/PL 0 D +/Bw 0 D +/PD -1 D +/TP f D +/tp f D +/TH f D +/Ty 4 D +/Tn -1 D +/Fl t D +/LB t D +/PM 1 D +/Ms f D +/Ba f D +/Bb f D +/Hl 3 D +/hl 6 D +/Hv 6 D +/Hs f D +/HI 0 D +/hi 0 D +/PO t D +/TE f D +/LF t D +/BO 0 D +/Sm 1 D +/Bf 3 D +/A1 0 D +/A2 0 D +/Ds 1 D +/QL -1 D +/Cb Db D +/Ct Dt D +/Cl Dl D +[/Creator (html2ps version 1.0 beta5) /Author () /Keywords () /Subject () + /Title (The Funtools Help Facility) /DOCINFO pdfmark +/ND 18 D +/HN [(1) (1) (1) (1) (8) (9) (19) (21) (28) (30) (33) (37) (38) (41) (42) (45) +(48) (51) (53) (??) (54) (56) (56) (87) (57) (58) (60) (62) (64) (65) (76) +(77) (66) (73) (75) (82) (85) (78) (80) (86) (87) (57) (91) (91) (92) (92) +(94) (102) (111) (95) (96) (100) (117) (123) (127) (136) (152) (157) (161) +(164) (165) (167) (171) (1) (1) (1) (3) (1) (1) (1) (8) (9) (19) (21) (28) +(30) (33) (37) (38) (41) (42) (45) (48) (50) (51) (51) (51) (51) (52) (53) +(53) (53) (53) (53) (54) (56) (56) (57) (57) (58) (60) (62) (64) (65) (66) +(73) (75) (76) (77) (80) (82) (85) (86) (87) (87) (87) (88) (90) (91) (98) +(91) (91) (91) (91) (92) (92) (94) (95) (96) (98) (99) (100) (101) (102) +(102) (102) (102) (102) (103) (104) (105) (105) (108) (109) (110) (111) +(111) (111) (111) (111) (111) (112) (113) (114) (114) (115) (116) (116) +(116) (117) (119) (119) (120) (120) (120) (117) (117) (117) (117) (119) +(119) (120) (120) (120) (121) (122) (123) (123) (123) (123) (126) (127) +(127) (127) (127) (127) (129) (129) (130) (131) (131) (132) (132) (133) +(135) (136) (136) (136) (136) (146) (151) (152) (152) (152) (152) (156) +(157) (157) (157) (157) (157) (158) (159) (160) (161) (161) (161) (161) +(161) (161) (162) (162) (163) (164) (164) (164) (164) (164) (165) (165) +(165) (165) (165) (166) (167) (167) (167) (167) (170) (171) (171) (171) +(171) (171) (172) (172) (172) (175) (178) (178) (179) (180) (181) (181) +(182) (182) (183) (183) (184) (184) (184) (184) (185) (185) (185) (186) +(187) (187)] D +/h0 [()(Table of Contents)] D +/h1 [(0.1\240\240)(Funtools: FITS Users Need Tools)] D +/h2 [(0.2\240\240)(Summary)] D +/h3 [(0.3\240\240)(Description)] D +/h4 [(0.3.0.0.1\240\240)(Last updated: January 6, 2006)] D +/h5 [(0.4\240\240)(Funtools Programs)] D +/h6 [(0.5\240\240)(Summary)] D +/h7 [(0.6\240\240)(funcalc - Funtools calculator \(for binary tables\))] D +/h8 [(0.7\240\240)(funcen - find centroid \(for binary tables\))] D +/h9 [(0.8\240\240)(funcnts - count photons in specified regions, with bkgd subtraction)] D +/h10 [(0.9\240\240)(funcone - cone search of a binary table containing RA, Dec columns)] D +/h11 [(0.10\240\240)(fundisp - display data in a Funtools data file)] D +/h12 [(0.11\240\240)(funhead - display a header in a Funtools file)] D +/h13 [(0.12\240\240)(funhist - create a 1D histogram of a column \(from a FITS binary table or raw event file\) or an image)] D +/h14 [(0.13\240\240)(funimage - create a FITS image from a Funtools data file)] D +/h15 [(0.14\240\240)(funindex - create an index for a column of a FITS binary table)] D +/h16 [(0.15\240\240)(funjoin - join two or more FITS binary tables on specified columns)] D +/h17 [(0.16\240\240)(funmerge - merge one or more Funtools table files)] D +/h18 [(0.17\240\240)(funsky - convert between image and sky coordinates)] D +/h19 [(0.18\240\240)(funtable - copy selected rows from a Funtools file to a FITS binary table)] D +/h20 [(0.19\240\240)(funtbl - extract a table from Funtools ASCII output)] D +/h21 [(0.19.0.0.1\240\240)(Last updated: April 1, 2007)] D +/h22 [(0.20\240\240)(FunDS9: Funtools and DS9 Image Display)] D +/h23 [(0.21\240\240)(Summary)] D +/h24 [(0.22\240\240)(Description)] D +/h25 [(0.22.0.0.1\240\240)(Last updated: November 16, 2005)] D +/h26 [(0.23\240\240)(FunLib: the Funtools Programming Interface)] D +/h27 [(0.24\240\240)(Summary)] D +/h28 [(0.25\240\240)(Introduction to the Funtools Programming Interface)] D +/h29 [(0.26\240\240)(Funtools Programming Tutorial)] D +/h30 [(0.27\240\240)(Compiling and Linking)] D +/h31 [(0.28\240\240)(A Short Digression on Subroutine Order)] D +/h32 [(0.29\240\240)(Funtools Programming Examples)] D +/h33 [(0.30\240\240)(The Funtools Programming Reference Manual)] D +/h34 [(0.31\240\240)(FunOpen - open a Funtools data file)] D +/h35 [(0.32\240\240)(FunImageGet - get an image or image section)] D +/h36 [(0.33\240\240)(FunImagePut - put an image to a Funtools file)] D +/h37 [(0.34\240\240)(FunImageRowGet - get row\(s\) of an image)] D +/h38 [(0.35\240\240)(FunImageRowPut - put row\(s\) of an image)] D +/h39 [(0.36\240\240)(FunColumnSelect - select Funtools columns)] D +/h40 [(0.37\240\240)(FunColumnActivate - activate Funtools columns)] D +/h41 [(0.38\240\240)(FunColumnLookup - lookup a Funtools column)] D +/h42 [(0.39\240\240)(FunTableRowGet - get Funtools rows)] D +/h43 [(0.40\240\240)(FunTableRowPut - put Funtools rows)] D +/h44 [(0.41\240\240)(FunParamPut - put a Funtools param value)] D +/h45 [(0.42\240\240)(FunInfoGet - get information from Funtools struct)] D +/h46 [(0.43\240\240)(FunInfoPut - put information into a Funtools struct)] D +/h47 [(0.44\240\240)(FunFlush - flush data to output file)] D +/h48 [(0.45\240\240)(FunClose - close a Funtools data file)] D +/h49 [(0.46\240\240)(FunRef: the Funtools Reference Handle)] D +/h50 [(0.47\240\240)(Summary)] D +/h51 [(0.48\240\240)(Description)] D +/h52 [(0.48.0.0.1\240\240)(Last updated: December 1, 2005)] D +/h53 [(0.49\240\240)(FunFiles: Funtools Data Files)] D +/h54 [(0.50\240\240)(Summary)] D +/h55 [(0.51\240\240)(Description)] D +/h56 [(0.52\240\240)(Supported Data Formats)] D +/h57 [(0.53\240\240)(FITS Images and Binary Tables)] D +/h58 [(0.54\240\240)(Non-FITS Raw Event Files)] D +/h59 [(0.55\240\240)(Non-FITS Array Files)] D +/h60 [(0.56\240\240)(Specifying Image Sections)] D +/h61 [(0.57\240\240)(Binning FITS Binary Tables and Non-FITS Event Files)] D +/h62 [(0.58\240\240)(Table and Spatial Region Filters)] D +/h63 [(0.59\240\240)(Disk Files and Other Supported File Types)] D +/h64 [(0.60\240\240)(Lists of Files)] D +/h65 [(0.60.0.0.1\240\240)(Last updated: February 15, 2006)] D +/h66 [(0.61\240\240)(Funtext: Support for Column-based Text Files)] D +/h67 [(0.62\240\240)(Summary)] D +/h68 [(0.63\240\240)(Description)] D +/h69 [(0.64\240\240)(Standard Text Files)] D +/h70 [(0.65\240\240)(Comments Convert to Header Params)] D +/h71 [(0.66\240\240)(Multiple Tables in a Single File)] D +/h72 [(0.67\240\240)(TEXT\(\) Specifier)] D +/h73 [(0.68\240\240)(Text\(\) Keyword Options)] D +/h74 [(0.69\240\240)(Environment Variables)] D +/h75 [(0.70\240\240)(Restrictions and Problems)] D +/h76 [(0.70.0.0.1\240\240)(Last updated: August 3, 2007)] D +/h77 [(0.71\240\240)(Funview: Database View Support for Tables)] D +/h78 [(0.72\240\240)(Summary)] D +/h79 [(0.73\240\240)(Description)] D +/h80 [(0.74\240\240)(Database Views)] D +/h81 [(0.75\240\240)(Funtools View Attributes)] D +/h82 [(0.76\240\240)(Invoking a Funtools View \(in Place of an Input File\))] D +/h83 [(0.77\240\240)(Basic View Matching Rules)] D +/h84 [(0.78\240\240)(More on View Matching Rules: Single vs. Multiple Matches)] D +/h85 [(0.79\240\240)(View Lists: Applying a View to Any File)] D +/h86 [(0.80\240\240)(Overriding Values Associated with a View)] D +/h87 [(0.81\240\240)(Environment Variables)] D +/h88 [(0.82\240\240)(Restrictions and Problems)] D +/h89 [(0.82.0.0.1\240\240)(Last updated: August 3, 2007)] D +/h90 [(0.83\240\240)(Funfilters: Filtering Rows in a Table)] D +/h91 [(0.84\240\240)(Summary)] D +/h92 [(0.85\240\240)(Description)] D +/h93 [(0.86\240\240)(Filter Expressions)] D +/h94 [(0.87\240\240)(Separators Also Are Operators)] D +/h95 [(0.88\240\240)(Range Lists)] D +/h96 [(0.89\240\240)(Math Operations and Functions)] D +/h97 [(0.90\240\240)(Include Files)] D +/h98 [(0.91\240\240)(Header Parameters)] D +/h99 [(0.92\240\240)(Examples)] D +/h100 [(0.92.0.0.1\240\240)(Last updated: November 17, 2005)] D +/h101 [(0.93\240\240)(Funidx: Using Indexes to Filter Rows in a Table)] D +/h102 [(0.94\240\240)(Summary)] D +/h103 [(0.95\240\240)(Description)] D +/h104 [(0.95.0.0.1\240\240)(Last updated: August 3, 2007)] D +/h105 [(0.96\240\240)(Regions: Spatial Region Filtering)] D +/h106 [(0.97\240\240)(Summary)] D +/h107 [(0.98\240\240)(Description)] D +/h108 [(0.99\240\240)(Region Expressions)] D +/h109 [(0.100\240\240)(Columns Used in Region Filtering)] D +/h110 [(0.101\240\240)(Region Algebra)] D +/h111 [(0.102\240\240)(Region Separators Also Are Operators)] D +/h112 [(0.103\240\240)(Region Exclusion)] D +/h113 [(0.104\240\240)(Include Files)] D +/h114 [(0.105\240\240)(Global and Local Properties of Regions)] D +/h115 [(0.106\240\240)(Coordinate Systems)] D +/h116 [(0.107\240\240)(Specifying Positions, Sizes, and Angles)] D +/h117 [(0.107.0.0.1\240\240)(Last updated: November 17, 2005)] D +/h118 [(0.108\240\240)(RegGeometry: Geometric Shapes in Spatial Region Filtering)] D +/h119 [(0.109\240\240)(Summary)] D +/h120 [(0.110\240\240)(Geometric shapes)] D +/h121 [(0.111\240\240)(Region accelerators)] D +/h122 [(0.111.0.0.1\240\240)(Last updated: March 12, 2007)] D +/h123 [(0.112\240\240)(RegAlgebra: Boolean Algebra on Spatial Regions)] D +/h124 [(0.113\240\240)(Summary)] D +/h125 [(0.114\240\240)(Description)] D +/h126 [(0.114.0.0.1\240\240)(Last updated: November 17, 2005)] D +/h127 [(0.115\240\240)(RegCoords: Spatial Region Coordinates)] D +/h128 [(0.116\240\240)(Summary)] D +/h129 [(0.117\240\240)(Pixel coordinate systems)] D +/h130 [(0.118\240\240)(World Coordinate Systems)] D +/h131 [(0.119\240\240)(WCS Positions and Sizes)] D +/h132 [(0.120\240\240)(NB: The Meaning of Pure Numbers Are Context Sensitive)] D +/h133 [(0.120.0.0.1\240\240)(Last updated: November 17, 2005)] D +/h134 [(0.121\240\240)(RegBounds: Region Boundaries)] D +/h135 [(0.122\240\240)(Summary)] D +/h136 [(0.123\240\240)(Description)] D +/h137 [(0.124\240\240)(Image boundaries : radially-symmetric shapes \(circle, annuli, ellipse\))] D +/h138 [(0.125\240\240)(Image Boundaries: non-radially symmetric shapes \(polygons, boxes\))] D +/h139 [(0.126\240\240)(Row Boundaries are Analytic)] D +/h140 [(0.127\240\240)(Image Boundaries vs. Row Boundaries: Practical Considerations)] D +/h141 [(0.127.0.0.1\240\240)(Last updated: November 16, 2005)] D +/h142 [(0.128\240\240)(RegDiff:Differences Between Funtools and IRAF Regions)] D +/h143 [(0.129\240\240)(Summary)] D +/h144 [(0.130\240\240)(Description)] D +/h145 [(0.130.0.0.1\240\240)(Last updated: November 16, 2005)] D +/h146 [(0.131\240\240)(FunCombine: Combining Region and Table Filters)] D +/h147 [(0.132\240\240)(Summary)] D +/h148 [(0.133\240\240)(Comma Conventions)] D +/h149 [(0.133.0.0.1\240\240)(Last updated: November 16, 2005)] D +/h150 [(0.134\240\240)(FunEnv: Funtools Environment Variables)] D +/h151 [(0.135\240\240)(Summary)] D +/h152 [(0.136\240\240)(Description)] D +/h153 [(0.136.0.0.1\240\240)(Last updated: November 16, 2005)] D +/h154 [(0.137\240\240)(Funtools ChangeLog)] D +/h155 [(0.138\240\240)(Patch Release 1.4.5 \(internal ds9 release\))] D +/h156 [(0.139\240\240)(Patch Release 1.4.4 \(internal ds9 release\))] D +/h157 [(0.140\240\240)(Patch Release 1.4.3 \(internal ds9 release\))] D +/h158 [(0.141\240\240)(Patch Release 1.4.2 \(internal ds9 release\))] D +/h159 [(0.142\240\240)(Patch Release 1.4.1 \(internal ds9 release\))] D +/h160 [(0.143\240\240)(Public Release 1.4.0 \(15 August 2007\))] D +/h161 [(0.144\240\240)(Release 1.3.0b[n] \(mainly internal SAO beta releases\))] D +/h162 [(0.145\240\240)(Patch Release 1.2.4 \(internal SAO and beta release only\))] D +/h163 [(0.146\240\240)(Patch Release 1.2.3 \(12 January 2004\))] D +/h164 [(0.147\240\240)(Patch Release 1.2.3b1 \(19 August 2003\))] D +/h165 [(0.148\240\240)(Patch Release 1.2.2 \(18 May 2003\))] D +/h166 [(0.149\240\240)(Patch Release 1.2.1 \(24 April 2003\))] D +/h167 [(0.150\240\240)(Public Release 1.2.0 \(24 March 2003\))] D +/h168 [(0.151\240\240)(Beta Release 1.2.b3 \(4 February 2003\))] D +/h169 [(0.152\240\240)(Beta Release 1.2.b2 \(7 October 2002\))] D +/h170 [(0.153\240\240)(Beta Release 1.2.b1 \(24 September 2002\))] D +/h171 [(0.154\240\240)(Public Release 1.1.0 \(22 April 2002\))] D +/h172 [(0.155\240\240)(Pre-Release 1.1.0e \(10 April 2002\))] D +/h173 [(0.156\240\240)(Pre-Release 1.1.0e \(19 March 2002\))] D +/h174 [(0.157\240\240)(Pre-Release 1.1.0e \(27 February 2002\))] D +/h175 [(0.158\240\240)(Pre-Release 1.1.0e \(11 February 2002\))] D +/h176 [(0.159\240\240)(Beta Release 1.0.1b5 \(31 January 2002\))] D +/h177 [(0.160\240\240)(Beta Release 1.0.1b4 \(26 January 2002\))] D +/h178 [(0.161\240\240)(Beta Release 1.0.1b3 \(4 January 2002\))] D +/h179 [(0.162\240\240)(Beta Release 1.0.1b2 \(14 November 2001\))] D +/h180 [(0.163\240\240)(Beta Release 1.0.1b1 \(6 November 2001\))] D +/h181 [(0.164\240\240)(Public Release 1.0.0 \(31 July 2001\))] D +/h182 [(0.164.0.0.1\240\240)(Last updated: 22 April 2002)] D +/Hr [-63 63 64 65 -66 -66 66 -67 67 68 69 70 71 72 73 74 75 76 77 78 79 +80 81 82 -83 -83 83 -85 85 86 87 -88 -88 88 -91 91 92 93 94 95 96 97 98 +99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 -117 +-117 117 -120 120 121 122 123 124 125 126 127 128 129 130 131 -132 -132 +132 -134 134 135 136 137 138 139 140 141 142 143 -144 -144 144 -146 146 +147 148 149 150 151 152 153 154 155 156 157 -158 -158 158 -165 165 166 167 +168 169 170 171 172 173 174 -175 -175 175 -177 177 178 179 -180 -180 180 +-182 182 183 184 185 186 187 188 189 190 191 192 193 -194 -194 194 -196 +196 197 198 199 -200 -200 200 -202 202 203 204 -205 -205 205 -207 207 208 +209 210 211 212 -213 -213 213 -215 215 216 217 218 219 220 221 -222 -222 +222 -224 224 225 226 -227 -227 227 -230 230 231 232 -233 -233 233 -235 235 +236 237 -238 -238 238 -239 239 240 241 242 243 244 245 246 247 248 249 250 +251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 -267 -267 +267]D +/HV [1 2 2 2 3 4 5 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 4 5 1 2 2 2 3 4 +5 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 4 5 1 2 2 2 2 +2 2 2 2 2 2 2 2 3 4 5 1 2 2 2 2 2 2 2 2 2 2 3 4 5 1 2 2 2 2 2 2 2 2 2 2 +2 2 3 4 5 1 2 2 2 2 2 2 2 2 2 2 3 4 5 1 2 2 2 3 4 5 1 2 2 2 2 2 2 2 2 2 +2 2 2 3 4 5 1 2 2 2 2 3 4 5 1 2 2 2 3 4 5 1 2 2 2 2 2 2 3 4 5 1 2 2 2 2 +2 2 2 3 4 5 1 2 2 2 3 4 5 1 2 2 2 3 4 5 1 2 2 2 3 4 5 1 2 2 2 2 2 2 2 2 +2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 4 5]D +/Cn [3 0 0 1 1 1 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 3 0 0 1 1 1 +0 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 12 0 0 0 +0 0 0 0 0 0 0 0 1 1 1 0 10 0 0 0 0 0 0 0 0 0 1 1 1 0 12 0 0 0 0 0 0 0 0 +0 0 0 1 1 1 0 10 0 0 0 0 0 0 0 0 0 1 1 1 0 3 0 0 1 1 1 0 12 0 0 0 0 0 0 +0 0 0 0 0 1 1 1 0 4 0 0 0 1 1 1 0 3 0 0 1 1 1 0 6 0 0 0 0 0 1 1 1 0 7 0 +0 0 0 0 0 1 1 1 0 3 0 0 1 1 1 0 3 0 0 1 1 1 0 3 0 0 1 1 1 0 28 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0]D +Hr length 0 gt{[/PageMode /UseOutlines /DOCVIEW pdfmark}if +/Hn 1 D +0 1 Hr length 1 sub{ + /Bn E D [Cn Bn get dup 0 gt{/Count E HV Bn get Bl ge{neg}if}{pop}ie + /Dest Hr Bn get dup abs ST cvs cvn E 0 ge{(h)Hn ST cvs join cvx exec + dup 1 get E Nf{0 get E join}{pop}ie /Hn Hn 1 add D}{()}ie + /Title E dup length 255 gt{0 255 getinterval}if /OUT pdfmark}for +ZF /FN Fp D Ps 0 FS /WC Wf{( )}{<A1A1>}ie SW pop D +ET RC ZF +/Df f D +/R1 (http://hea-www.harvard.edu/saord/ds9/index.html) D +/R2 (http://hea-www.harvard.edu/saord/xpa/index.html) D +/R3 (ftp://cfa-ftp.harvard.edu/pub/gsc/WCSTools/home.html) D +/R4 (http://tdc-www.harvard.edu/software/wcstools/) D +/R5 (http://hea-www.harvard.edu/RD/funtools/changelog.html) D +/R6 (http://hea-www.harvard.edu/RD/funtools/changelog_beta.html) D +/Ba f D /BO 0 D Bs +/UR (help.html) D +/Ti (The Funtools Help Facility) D +/Au () D +/Df f D +/ME [()] D + TC + +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc + +/Ba f D /BO 0 D Bs +/UR (help.html) D +/Ti (The Funtools Help Facility) D +/Au () D +/Df f D +/ME [()] D + TC + +NP RC ZF +()1 Sl()WB 0 Sn( + + +)0 2 0 H(Funtools:)WB 63 Sn()WB 1 Sn( FITS Users Need Tools)EA()EH( + + +)0 2 1 H(Summary)WB 64 Sn()EH( +This document is the Table of Contents for Funtools. + + +)0 2 2 H(Description)WB 65 Sn()EH( +Funtools, is a "minimal buy-in" FITS library and utility package developed +at the the High Energy Astrophysics Division of SAO. The Funtools +library provides simplified access to a wide array of file types: +standard astronomical FITS images and binary tables, raw arrays and +binary event lists, and even tables of ASCII column data. A +sophisticated region filtering library \201compatible with ds9\202 filters +images and tables using boolean operations between geometric shapes, +support world coordinates, etc. Funtools also supports advanced +capabilities such as optimized data searching using index files. + +The main goal of the Funtools project has been to develop a minimal buy-in +FITS library for researchers who are occasional \201but serious\202 coders. In +this case, "minimal buy-in" means "easy to learn, easy to use, and easy to +re-learn next month". We have tried to achieve this goal by emphasizing two +essential capabilities. The first is the ability to develop FITS programs +without knowing much about FITS, i.e., without having to deal with the +arcane rules for generating a properly formatted FITS file. The second is +to support the use of already-familiar C/Unix facilities, especially C +structs and Unix stdio. Taken together, these two capabilities should allow +researchers to leverage their existing programming expertise while +minimizing the need to learn new and complex coding rules. + + +)0 P(Choose from the following topics: + +)0 P()UL()-1 LI()0 2 1 A(Funtools User Programs)2 1 TN TL()Ec /AF f D( +)UL()-1 LI()0 3 1 A(funcalc: Funtools calculator \201for binary tables\202)3 1 TN TL()Ec /AF f D( + +)-1 LI()0 4 1 A(funcen: find centroid \201for binary tables\202)4 1 TN TL()Ec /AF f D( + +)-1 LI()0 5 1 A(funcnts: count photons in specified regions)5 1 TN TL()Ec /AF f D( + +)-1 LI()0 6 1 A(funcone: cone search on RA, Dec columns)6 1 TN TL()Ec /AF f D( + +)-1 LI()0 7 1 A(fundisp: display data in a Funtools data file)7 1 TN TL()Ec /AF f D( + +)-1 LI()0 8 1 A(funhead: display a header in a Funtools file)8 1 TN TL()Ec /AF f D( + +)-1 LI()0 9 1 A(funhist: create a 1D histogram of a column)9 1 TN TL()Ec /AF f D( + +)-1 LI()0 10 1 A(funimage: create a FITS image from a Funtools data file)10 1 TN TL()Ec /AF f D( + +)-1 LI()0 11 1 A(funindex: create an index on a column in a binary table)11 1 TN TL()Ec /AF f D( + +)-1 LI()0 12 1 A(funjoin: join two or more FITS binary tables on specified columns)12 1 TN TL()Ec /AF f D( + +)-1 LI()0 13 1 A(funmerge: merge one or more Funtools table files)13 1 TN TL()Ec /AF f D( + +)-1 LI()0 14 1 A(funsky: convert between image and sky coordinates, using WCS info from a FITS header)14 1 TN TL()Ec /AF f D( + +)-1 LI()0 15 1 A(funtable: copy selected rows from a Funtools file to a FITS binary table)15 1 TN TL()Ec /AF f D( + +)-1 LI()0 16 1 A(funtbl: extract a table from +Funtools ASCII output)16 1 TN TL()Ec /AF f D( + +)-1 LI()0 17 1 A(funtools and ds9 image display)17 1 TN TL()Ec /AF f D( +)LU( + +)-1 LI()0 18 1 A(Funtools Programming)18 1 TN TL()Ec /AF f D( +)UL()-1 LI()0 19 1 A(Funtools Programming Summary)19 1 TN TL()Ec /AF f D( + +)-1 LI()0 20 1 A(Funtools Programming Tutorial)20 1 TN TL()Ec /AF f D( + +)-1 LI()0 21 1 A(A Short Digression on Subroutine Order)21 1 TN TL()Ec /AF f D( + +)-1 LI()0 22 1 A(Compiling and Linking)22 1 TN TL()Ec /AF f D( + +)-1 LI()0 23 1 A(The Funtools Reference Handle)23 1 TN TL()Ec /AF f D( + +)-1 LI()0 24 1 A(The Funtools Programming Reference Manual)24 1 TN TL()Ec /AF f D( +)UL()-1 LI( )0 25 1 A(FunOpen: open a Funtools file)25 1 TN TL()Ec /AF f D( + +)-1 LI()0 26 1 A(FunImageGet: retrieve image data)26 1 TN TL()Ec /AF f D( + +)-1 LI()0 27 1 A(FunImagePut: output image data)27 1 TN TL()Ec /AF f D( + +)-1 LI()0 28 1 A(FunImageRowGet: retrieve image data by row)28 1 TN TL()Ec /AF f D( + +)-1 LI()0 29 1 A(FunImageRowPut: output image data by row)29 1 TN TL()Ec /AF f D( + +)-1 LI()0 30 1 A(FunTableRowGet: retrieve rows from a table)30 1 TN TL()Ec /AF f D( + +)-1 LI()0 31 1 A(FunTableRowPut: output rows to a table)31 1 TN TL()Ec /AF f D( + +)-1 LI()0 32 1 A(FunColumnSelect: select columns in a table for access)32 1 TN TL()Ec /AF f D( + +)-1 LI()0 33 1 A(FunColumnActivate: activate columns in a table for read/write)33 1 TN TL()Ec /AF f D( + +)-1 LI()0 34 1 A(FunColumnLookup: lookup info about the columns in a table)34 1 TN TL()Ec /AF f D( + +)-1 LI()0 35 1 A(FunInfoGet: get info about an image or table)35 1 TN TL()Ec /AF f D( + +)-1 LI()0 36 1 A(FunInfoPut: put info about an image or table)36 1 TN TL()Ec /AF f D( + +)-1 LI()0 37 1 A(FunParamGet: get header param)37 1 TN TL()Ec /AF f D( + +)-1 LI()0 38 1 A(FunParamPut: put header param)38 1 TN TL()Ec /AF f D( + +)-1 LI()0 39 1 A(FunFlush: flush I/O in a Funtools file)39 1 TN TL()Ec /AF f D( + +)-1 LI()0 40 1 A(FunClose: close a Funtools file)40 1 TN TL()Ec /AF f D( +)LU( + +)-1 LI()0 41 1 A(Funtools Programming Examples)41 1 TN TL()Ec /AF f D( + +)UL( +)-1 LI()0 2 A(evmerge: merge new columns with existing columns)EA( +)-1 LI()0 2 A(evcols: add column and rows to binary tables)EA( +)-1 LI()0 2 A(imblank: blank out image values below a threshold)EA()LU()LU( + +)-1 LI()0 42 1 A(Funtools Data Files)42 1 TN TL()Ec /AF f D( + +)UL( +)-1 LI()0 43 1 A(Supported Data Formats)43 1 TN TL()Ec /AF f D( +)UL()-1 LI()0 44 1 A(FITS File and Extensions)44 1 TN TL()Ec /AF f D( +)-1 LI()0 45 1 A(Non-FITS Raw Event Files)45 1 TN TL()Ec /AF f D( +)-1 LI()0 46 1 A(Non-FITS Array Files)46 1 TN TL()Ec /AF f D( +)-1 LI()0 47 1 A(Column-based Text \201ASCII\202 Files)47 1 TN TL()Ec /AF f D( +)-1 LI()0 48 1 A(Database Views of Tables)48 1 TN TL()Ec /AF f D()LU( +)-1 LI()0 49 1 A(Image Sections and Blocking)49 1 TN TL()Ec /AF f D( +)-1 LI()0 50 1 A(Binning FITS Binary Tables and Non-FITS Event Files)EH( +)-1 LI()0 51 1 A(Disk Files and Other Supported File Types)50 1 TN TL()Ec /AF f D()LU( + +)-1 LI(Funtools Data Filtering +)UL()-1 LI()0 52 1 A(Table Filtering)51 1 TN TL()Ec /AF f D( + +)-1 LI()0 53 1 A(Fast Table Filtering using Indexes)52 1 TN TL()Ec /AF f D( + +)-1 LI()0 54 1 A(Spatial Region Filtering)53 1 TN TL()Ec /AF f D( + +)UL()-1 LI()0 55 1 A(Region Geometry)54 1 TN TL()Ec /AF f D( + +)-1 LI()0 56 1 A(Region Algebra)55 1 TN TL()Ec /AF f D( + +)-1 LI()0 57 1 A(Region Coordinates)56 1 TN TL()Ec /AF f D( + +)-1 LI()0 58 1 A(Region Boundaries)57 1 TN TL()Ec /AF f D( + +)-1 LI()0 59 1 A(Differences Between Funtools and IRAF Regions)58 1 TN TL()Ec /AF f D( +)LU( +)-1 LI()0 60 1 A(Combining Table and Region Filters)59 1 TN TL()Ec /AF f D( +)LU( + +)-1 LI( Miscellaneous +)UL()-1 LI()0 61 1 A(Funtools Environment Variables)60 1 TN TL()Ec /AF f D( + +)-1 LI()0 62 1 A(Funtools ChangeLog)61 1 TN TL()Ec /AF f D()LU( +)LU( + + + +)0 5 3 H(Last)WB 66 Sn( updated: January 6, 2006)EH( + + +)WB NL NP Ep ET /Tc f D +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (programs.html) D +/Ti (Funtools Programs) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 2 Sn( +)0 2 4 H(Funtools)WB 67 Sn( Programs)EH( + +)0 2 5 H(Summary)WB 68 Sn()EH( + +)0 P() 27 129 PR()0 3 1 A(funcalc)3 0 TN TL()Ec /AF f D( [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] [-u] <iname> [oname [columns]] + +)0 4 1 A(funcen)4 0 TN TL()Ec /AF f D( [-i] [-n iter] [-t tol] [-v lev] <iname> <region> + +)0 5 1 A(funcnts)5 0 TN TL()Ec /AF f D( [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_cnts] + +)0 6 1 A(funcone)6 0 TN TL()Ec /AF f D( [-n] [-x|-X|-j|-J] [[-l|-L] list] [-r ra_col] [-d dec_col] <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns] + +)0 7 1 A(fundisp)7 0 TN TL()Ec /AF f D( [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n] + +)0 8 1 A(funhead)8 0 TN TL()Ec /AF f D( [-a] [-l] [-s] [-t] [-L] <iname> [oname ename] + +)0 9 1 A(funhist)9 0 TN TL()Ec /AF f D( [-n|-w|-T] <iname> [column] [[lo_edge:hi_edge:]bins] + +)0 10 1 A(funimage)10 0 TN TL()Ec /AF f D( [-a] [-l] [-p x|y] <iname> <oname> [bitpix=n] + +)0 11 1 A(funindex)11 0 TN TL()Ec /AF f D( <iname> <key> [oname] + +)0 12 1 A(funjoin)12 0 TN TL()Ec /AF f D( [switches] <ifile1> <ifile2> ... <ifilen> <ofile> + +)0 13 1 A(funmerge)13 0 TN TL()Ec /AF f D( <iname1> <iname2> ... <oname> + +)0 14 1 A(funsky)14 0 TN TL()Ec /AF f D( [switches] <iname1> [<lname2> <col1> <col2>] + +)0 15 1 A(funtable)15 0 TN TL()Ec /AF f D( [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns] + +)0 16 1 A(funtbl)16 0 TN TL()Ec /AF f D( [-c cols] [-h] [-n table] [-p prog] [-s sep] [-T] <iname>)RP( + + + + +)0 2 6 H(funcalc)WB 69 Sn()WB 3 Sn( - Funtools calculator \201for binary tables\202)EA()EH( + +)BD() 1 90 PR(funcalc [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] <iname> [oname [columns]])RP()ES( + + +)0 P() 7 73 PR( -a argstr # user arguments to pass to the compiled program + -e expr # funcalc expression + -f file # file containing funcalc expression + -l libs # libs to add to link command + -n # output generated code instead of compiling and executing + -p prog # generate named program, no execution + -u # die if any variable is undeclared \201don't auto-declare\202)RP( + + +)0 P()BD(funcalc)ES( is a calculator program that allows arbitrary +expressions to be constructed, compiled, and executed on columns in a +Funtools table \201FITS binary table or raw event file\202. It works by +integrating user-supplied expression\201s\202 into a template C program, +then compiling and executing the program. )BD(funcalc)ES( expressions +are C statements, although some important simplifications \201such +as automatic declaration of variables\202 are supported. + +)0 P()BD(funcalc)ES( expressions can be specified in three ways: on the +command line using the )BD(-e [expression])ES( switch, in a file using +the )BD(-f [file])ES( switch, or from stdin \201if neither )BD(-e)ES( nor +)BD(-f)ES( is specified\202. Of course a file containing )BD(funcalc)ES( +expressions can be read from stdin. + +)0 P(Each invocation of )BD(funcalc)ES( requires an input Funtools table +file to be specified as the first command line argument. The output +Funtools table file is the second optional argument. It is needed only +if an output FITS file is being created \201i.e., in cases where the +)BD(funcalc)ES( expression only prints values, no output file is +needed\202. If input and output file are both specified, a third optional +argument can specify the list of columns to activate \201using +)0 33 1 A(FunColumnActivate\201\202)33 0 TN TL()Ec /AF f D(\202. Note +that )BD(funcalc)ES( determines whether or not to generate code for +writing an output file based on the presence or absence of an +output file argument. + +)0 P(A )BD(funcalc)ES( expression executes on each row of a table and +consists of one or more C statements that operate on the columns of +that row \201possibly using temporary variables\202. Within an expression, +reference is made to a column of the )BD(current)ES( row using the C +struct syntax )BD(cur->[colname])ES(, e.g. cur->x, cur->pha, etc. +Local scalar variables can be defined using C declarations at very the +beginning of the expression, or else they can be defined automatically +by )BD(funcalc)ES( \201to be of type double\202. Thus, for example, a swap of +columns x and y in a table can be performed using either of the +following equivalent )BD(funcalc)ES( expressions: + +) 4 18 PR( double temp; + temp = cur->x; + cur->x = cur->y; + cur->y = temp;)RP( + +or: + +) 3 18 PR( temp = cur->x; + cur->x = cur->y; + cur->y = temp;)RP( + +When this expression is executed using a command such as: +) 1 40 PR( funcalc -f swap.expr itest.ev otest.ev)RP( +the resulting file will have values of the x and y columns swapped. + +)0 P(By default, the data type of the variable for a column is the same as +the data type of the column as stored in the file. This can be changed +by appending ":[dtype]" to the first reference to that column. In the +example above, to force x and y to be output as doubles, specify the +type 'D' explicitly: +) 3 20 PR( temp = cur->x:D; + cur->x = cur->y:D; + cur->y = temp;)RP( + +Data type specifiers follow standard FITS table syntax for defining +columns using TFORM: +)UL()-1 LI(A: ASCII characters +)-1 LI(B: unsigned 8-bit char +)-1 LI(I: signed 16-bit int +)-1 LI(U: unsigned 16-bit int \201not standard FITS\202 +)-1 LI(J: signed 32-bit int +)-1 LI(V: unsigned 32-bit int \201not standard FITS\202 +)-1 LI(E: 32-bit float +)-1 LI(D: 64-bit float +)-1 LI(X: bits \201treated as an array of chars\202)LU( +Note that only the first reference to a column should contain the +explicit data type specifier. + +)0 P(Of course, it is important to handle the data type of the columns +correctly. One of the most frequent cause of error in )BD(funcalc)ES( +programming is the implicit use of the wrong data type for a column in +expression. For example, the calculation: +) 1 43 PR( dx = \201cur->x - cur->y\202/\201cur->x + cur->y\202;)RP( +usually needs to be performed using floating point arithmetic. In +cases where the x and y columns are integers, this can be done by +reading the columns as doubles using an explicit type specification: +) 1 47 PR( dx = \201cur->x:D - cur->y:D\202/\201cur->x + cur->y\202;)RP( + +Alternatively, it can be done using C type-casting in the expression: +) 1 75 PR( dx = \201\201double\202cur->x - \201double\202cur->y\202/\201\201double\202cur->x + \201double\202cur->y\202;)RP( + +)0 P(In addition to accessing columns in the current row, reference also +can be made to the )BD(previous)ES( row using )BD(prev->[colname])ES(, +and to the )BD(next)ES( row using )BD(next->[colname])ES(. Note that if +)BD(prev->[colname])ES( is specified in the )BD(funcalc)ES( +expression, the very first row is not processed. If +)BD(next->[colname])ES( is specified in the )BD(funcalc)ES( +expression, the very last row is not processed. In this way, +)BD(prev)ES( and )BD(next)ES( are guaranteed always to point to valid +rows. For example, to print out the values of the current x column +and the previous y column, use the C fprintf function in a +)BD(funcalc)ES( expression: +) 1 46 PR( fprintf\201stdout, "%d %d\200n", cur->x, prev->y\202;)RP( + +)0 P(New columns can be specified using the same )BD(cur->[colname])ES( +syntax by appending the column type \201and optional tlmin/tlmax/binsiz +specifiers\202, separated by colons. For example, cur->avg:D will define +a new column of type double. Type specifiers are the same those +used above to specify new data types for existing columns. + +)0 P(For example, to create and output a new column that is the average value of the +x and y columns, a new "avg" column can be defined: +) 1 36 PR( cur->avg:D = \201cur->x + cur->y\202/2.0)RP( +Note that the final ';' is not required for single-line expressions. + +)0 P(As with FITS TFORM data type specification, the column data type +specifier can be preceded by a numeric count to define an array, e.g., +"10I" means a vector of 10 short ints, "2E" means two single precision +floats, etc. A new column only needs to be defined once in a +)BD(funcalc)ES( expression, after which it can be used without +re-specifying the type. This includes reference to elements of a +column array: + +) 2 41 PR( cur->avg[0]:2D = \201cur->x + cur->y\202/2.0; + cur->avg[1] = \201cur->x - cur->y\202/2.0;)RP( + +)0 P(The 'X' \201bits\202 data type is treated as a char array of dimension +\201numeric_count/8\202, i.e., 16X is processed as a 2-byte char array. Each +8-bit array element is accessed separately: +) 2 24 PR( cur->stat[0]:16X = 1; + cur->stat[1] = 2;)RP( +Here, a 16-bit column is created with the MSB is set to 1 and the LSB set to 2. + +)0 P(By default, all processed rows are written to the specified output +file. If you want to skip writing certain rows, simply execute the C +"continue" statement at the end of the )BD(funcalc)ES( expression, +since the writing of the row is performed immediately after the +expression is executed. For example, to skip writing rows whose +average is the same as the current x value: + +) 4 41 PR( cur->avg[0]:2D = \201cur->x + cur->y\202/2.0; + cur->avg[1] = \201cur->x - cur->y\202/2.0; + if\201 cur->avg[0] == cur->x \202 + continue;)RP( + +)0 P(If no output file argument is specified on the )BD(funcalc)ES( command +line, no output file is opened and no rows are written. This is useful +in expressions that simply print output results instead of generating +a new file: +) 5 73 PR( fpv = \201cur->av3:D-cur->av1:D\202/\201cur->av1+cur->av2:D+cur->av3\202; + fbv = cur->av2/\201cur->av1+cur->av2+cur->av3\202; + fpu = \201\201double\202cur->au3-cur->au1\202/\201\201double\202cur->au1+cur->au2+cur->au3\202; + fbu = cur->au2/\201double\202\201cur->au1+cur->au2+cur->au3\202; + fprintf\201stdout, "%f\200t%f\200t%f\200t%f\200n", fpv, fbv, fpu, fbu\202;)RP( +In the above example, we use both explicit type specification +\201for "av" columns\202 and type casting \201for "au" columns\202 to ensure that +all operations are performed in double precision. + +)0 P(When an output file is specified, the selected input table is +processed and output rows are copied to the output file. Note that +the output file can be specified as "stdout" in order to write the +output rows to the standard output. If the output file argument is +passed, an optional third argument also can be passed to specify which +columns to process. + +)0 P(In a FITS binary table, it sometimes is desirable to copy all of the +other FITS extensions to the output file as well. This can be done by +appending a '+' sign to the name of the extension in the input file +name. See )BD(funtable)ES( for a related example. + +)0 P()BD(funcalc)ES( works by integrating the user-specified expression +into a template C program called )0 2 A(tabcalc.c)EA(. +The completed program then is compiled and executed. Variable +declarations that begin the )BD(funcalc)ES( expression are placed in +the local declaration section of the template main program. All other +lines are placed in the template main program's inner processing +loop. Other details of program generation are handled +automatically. For example, column specifiers are analyzed to build a +C struct for processing rows, which is passed to +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( and used +in )0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D(. If +an unknown variable is used in the expression, resulting in a +compilation error, the program build is retried after defining the +unknown variable to be of type double. + +)0 P(Normally, )BD(funcalc)ES( expression code is added to +)BD(funcalc)ES( row processing loop. It is possible to add code +to other parts of the program by placing this code inside +special directives of the form: +) 3 26 PR( [directive name] + ... code goes here ... + end)RP( + +The directives are: +)UL()-1 LI()BD(global)ES( add code and declarations in global space, before the main routine. + +)-1 LI()BD(local)ES( add declarations \201and code\202 just after the local declarations in +main + +)-1 LI()BD(before)ES( add code just before entering the main row processing loop + +)-1 LI()BD(after)ES( add code just after exiting the main row processing loop)LU( + +Thus, the following )BD(funcalc)ES( expression will declare global +variables and make subroutine calls just before and just after the +main processing loop: +) 16 62 PR( global + double v1, v2; + double init\201void\202; + double finish\201double v\202; + end + before + v1 = init\201\202; + end + ... process rows, with calculations using v1 ... + after + v2 = finish\201v1\202; + if\201 v2 < 0.0 \202{ + fprintf\201stderr, "processing failed %g -> %g\200n", v1, v2\202; + exit\2011\202; + } + end)RP( +Routines such as init\201\202 and finish\201\202 above are passed to the generated +program for linking using the )BD(-l [link directives ...])ES( +switch. The string specified by this switch will be added to the link +line used to build the program \201before the funtools library\202. For +example, assuming that init\201\202 and finish\201\202 are in the library +libmysubs.a in the /opt/special/lib directory, use: +) 1 47 PR( funcalc -l "-L/opt/special/lib -lmysubs" ...)RP( + +)0 P(User arguments can be passed to a compiled funcalc program using a string +argument to the "-a" switch. The string should contain all of the +user arguments. For example, to pass the integers 1 and 2, use: +) 1 22 PR( funcalc -a "1 2" ...)RP( +The arguments are stored in an internal array and are accessed as +strings via the ARGV\201n\202 macro. For example, consider the following +expression: +) 11 60 PR( local + int pmin, pmax; + end + + before + pmin=atoi\201ARGV\2010\202\202; + pmax=atoi\201ARGV\2011\202\202; + end + + if\201 \201cur->pha >= pmin\202 && \201cur->pha <= pmax\202 \202 + fprintf\201stderr, "%d %d %d\200n", cur->x, cur->y, cur->pha\202;)RP( +This expression will print out x, y, and pha values for all rows in which +the pha value is between the two user-input values: +) 11 51 PR( funcalc -a '1 12' -f foo snr.ev'[cir 512 512 .1]' + 512 512 6 + 512 512 8 + 512 512 5 + 512 512 5 + 512 512 8 + + funcalc -a '5 6' -f foo snr.ev'[cir 512 512 .1]' + 512 512 6 + 512 512 5 + 512 512 5)RP( + +)0 P(Note that it is the user's responsibility to ensure that the correct +number of arguments are passed. The ARGV\201n\202 macro returns a NULL if a +requested argument is outside the limits of the actual number of args, +usually resulting in a SEGV if processed blindly. To check the +argument count, use the ARGC macro: +) 12 42 PR( local + long int seed=1; + double limit=0.8; + end + + before + if\201 ARGC >= 1 \202 seed = atol\201ARGV\2010\202\202; + if\201 ARGC >= 2 \202 limit = atof\201ARGV\2011\202\202; + srand48\201seed\202; + end + + if \201 drand48\201\202 > limit \202 continue;)RP( + +)0 P(The macro WRITE_ROW expands to the FunTableRowPut\201\202 call that writes +the current row. It can be used to write the row more than once. In +addition, the macro NROW expands to the row number currently being +processed. Use of these two macros is shown in the following example: +) 7 41 PR( if\201 cur->pha:I == cur->pi:I \202 continue; + a = cur->pha; + cur->pha = cur->pi; + cur->pi = a; + cur->AVG:E = \201cur->pha+cur->pi\202/2.0; + cur->NR:I = NROW; + if\201 NROW < 10 \202 WRITE_ROW;)RP( + +)0 P(If the )BD(-p [prog])ES( switch is specified, the expression is not +executed. Rather, the generated executable is saved with the specified +program name for later use. + +)0 P(If the )BD(-n)ES( switch is specified, the expression is not +executed. Rather, the generated code is written to stdout. This is +especially useful if you want to generate a skeleton file and add your +own code, or if you need to check compilation errors. Note that the +comment at the start of the output gives the compiler command needed +to build the program on that platform. \201The command can change from +platform to platform because of the use of different libraries, +compiler switches, etc.\202 + +)0 P(As mentioned previously, )BD(funcalc)ES( will declare a scalar +variable automatically \201as a double\202 if that variable has been used +but not declared. This facility is implemented using a sed script +named )0 2 A(funcalc.sed)EA(, which processes the +compiler output to sense an undeclared variable error. This script +has been seeded with the appropriate error information for gcc, and for +cc on Solaris, DecAlpha, and SGI platforms. If you find that automatic +declaration of scalars is not working on your platform, check this sed +script; it might be necessary to add to or edit some of the error +messages it senses. + +)0 P(In order to keep the lexical analysis of )BD(funcalc)ES( expressions +\201reasonably\202 simple, we chose to accept some limitations on how +accurately C comments, spaces, and new-lines are placed in the +generated program. In particular, comments associated with local +variables declared at the beginning of an expression \201i.e., not in a +)BD(local...end)ES( block\202 will usually end up in the inner loop, not +with the local declarations: +) 8 69 PR( /* this comment will end up in the wrong place \201i.e, inner loop\202 */ + double a; /* also in wrong place */ + /* this will be in the the right place \201inner loop\202 */ + if\201 cur->x:D == cur->y:D \202 continue; /* also in right place */ + a = cur->x; + cur->x = cur->y; + cur->y = a; + cur->avg:E = \201cur->x+cur->y\202/2.0;)RP( +Similarly, spaces and new-lines sometimes are omitted or added in a +seemingly arbitrary manner. Of course, none of these stylistic +blemishes affect the correctness of the generated code. + +)0 P(Because )BD(funcalc)ES( must analyze the user expression using the data +file\201s\202 passed on the command line, the input file\201s\202 must be opened +and read twice: once during program generation and once during +execution. As a result, it is not possible to use stdin for the +input file: )BD(funcalc)ES( cannot be used as a filter. We will +consider removing this restriction at a later time. + +)0 P(Along with C comments, )BD(funcalc)ES( expressions can have one-line +internal comments that are not passed on to the generated C +program. These internal comment start with the )BD(#)ES( character and +continue up to the new-line: +) 7 56 PR( double a; # this is not passed to the generated C file + # nor is this + a = cur->x; + cur->x = cur->y; + cur->y = a; + /* this comment is passed to the C file */ + cur->avg:E = \201cur->x+cur->y\202/2.0;)RP( + +)0 P(As previously mentioned, input columns normally are identified by +their being used within the inner event loop. There are rare cases +where you might want to read a column and process it outside the main +loop. For example, qsort might use a column in its sort comparison +routine that is not processed inside the inner loop \201and therefore not +implicitly specified as a column to be read\202. To ensure that such a +column is read by the event loop, use the )BD(explicit)ES( keyword. +The arguments to this keyword specify columns that should be read into +the input record structure even though they are not mentioned in the +inner loop. For example: + ) 1 17 PR( explicit pi pha)RP( +will ensure that the pi and pha columns are read for each row, +even if they are not processed in the inner event loop. The )BD(explicit)ES( +statement can be placed anywhere. + +)0 P(Finally, note that )BD(funcalc)ES( currently works on expressions +involving FITS binary tables and raw event files. We will consider +adding support for image expressions at a later point, if there is +demand for such support from the community. + + + + +)0 2 7 H(funcen)WB 70 Sn()WB 4 Sn( - find centroid \201for binary tables\202)EA()EH( + +)BD() 1 56 PR(funcen [-i] [-n iter] [-t tol] [-v lev] <iname> <region>)RP()ES( + + +)0 P() 4 64 PR( -i # use image filtering \201default: event filtering\202 + -n iter # max number of iterations \201default: 0\202 + -t tol # pixel tolerance distance \201default: 1.0\202 + -v [0,1,2,3] # output verbosity level \201default: 0\202)RP( + + +)0 P()BD(funcen)ES( iteratively calculates the centroid position within one +or more regions of a Funtools table \201FITS binary table or raw event +file\202. Starting with an input table, an initial region specification, +and an iteration count, the program calculates the average x and y +position within the region and then uses this new position as the +region center for the next iteration. Iteration terminates when the +maximum number of iterations is reached or when the input tolerance +distance is met for that region. A count of events in the final region +is then output, along with the pixel position value \201and, where +available, WCS position\202. + +)0 P(The first argument to the program specifies the Funtools table file to +process. Since the file must be read repeatedly, a value of "stdin" +is not permitted when the number of iterations is non-zero. Use +)0 42 1 A(Funtools Bracket Notation)42 0 TN TL()Ec /AF f D( to specify FITS +extensions and filters. + +)0 P(The second required argument is the initial region descriptor. Multiple +regions are permitted. However, compound regions \201accelerators, +variable argument regions and regions connected via boolean algebra\202 +are not permitted. Points and polygons also are illegal. These +restrictions might be lifted in a future version, if warranted. + +)0 P(The )BD(-n)ES( \201iteration number\202 switch specifies the maximum number of +iterations to perform. The default is 0, which means that the program will +simply count and display the number of events in the initial region\201s\202. +Note that when iterations is 0, the data can be input via stdin. + +)0 P(The )BD(-t)ES( \201tolerance\202 switch specifies a floating point tolerance +value. If the distance between the current centroid position value and +the last position values is less than this value, iteration terminates. +The default value is 1 pixel. + +)0 P(The )BD(-v)ES( \201verbosity\202 switch specifies the verbosity level of the +output. The default is 0, which results in a single line of output for +each input region consisting of the following values: +) 1 30 PR( counts x y [ra dec coordsys])RP( +The last 3 WCS values are output if WCS information is available in the +data file header. Thus, for example: +) 5 47 PR( [sh] funcen -n 0 snr.ev "cir 505 508 5" + 915 505.00 508.00 345.284038 58.870920 j2000 + + [sh] funcen -n 3 snr.ev "cir 505 508 5" + 1120 504.43 509.65 345.286480 58.874587 j2000)RP( +The first example simply counts the number of events in the initial region. +The second example iterates the centroid calculation three times to determine +a final "best" position. + +)0 P( +Higher levels of verbosity obviously imply more verbose output. At +level 1, the output essentially contains the same information as level +0, but with keyword formatting: + + [sh] funcen -v 1 -n 3 snr.ev "cir 505 508 5" + event_file: snr.ev + initial_region: cir 505 508 5 + tolerance: 1.0000 + iterations: 1 + + events: 1120 + x,y\201physical\202: 504.43 509.65 + ra,dec\201j2000\202: 345.286480 58.874587 + final_region1: cir 504.43 509.65 5)RP( +Level 2 outputs results from intermediate calculations as well. + +)0 P(Ordinarily, region filtering is performed using analytic \201event\202 +filtering, i.e. that same style of filtering as is performed by +)BD(fundisp)ES( and )BD(funtable)ES(. Use the )BD(-i)ES( switch to specify image +filtering, i.e. the same style filtering as is performed by )BD(funcnts)ES(. +Thus, you can perform a quick calculation of counts in regions, using +either the analytic or image filtering method, by specifying the + )BD(-n 0)ES( and optional )BD(-i)ES( switches. These two method often +give different results because of how boundary events are processed: +) 5 46 PR( [sh] funcen snr.ev "cir 505 508 5" + 915 505.00 508.00 345.284038 58.870920 j2000 + + [sh] funcen -i snr.ev "cir 505 508 5" + 798 505.00 508.00 345.284038 58.870920 j2000)RP( +See )0 58 1 A(Region Boundaries)58 0 TN TL()Ec /AF f D( for more information +about how boundaries are calculated using these two methods. + + + + +)0 2 8 H(funcnts)WB 71 Sn()WB 5 Sn( - count photons in specified regions, with bkgd subtraction)EA()EH( + + +)BD() 1 86 PR(funcnts [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_value])RP()ES( + + +)0 P() 16 79 PR( -e "source_exposure[;bkgd_exposure]" + # source \201bkgd\202 FITS exposure image using matching files + -w "source_exposure[;bkgd_exposure]" + # source \201bkgd\202 FITS exposure image using WCS transform + -t "source_timecorr[;bkgd_timecorr]" + # source \201bkgd\202 time correction value or header parameter name + -g # output using nice g format + -G # output using %.14g format \201maximum precision\202 + -i "[column;]int1;int2..." # column-based intervals + -m # match individual source and bkgd regions + -p # output in pixels, even if wcs is present + -r # output inner/outer radii \201and angles\202 for annuli \201and pandas\202 + -s # output summed values + -v "scol[;bcol]" # src and bkgd value columns for tables + -T # output in starbase/rdb format + -z # output regions with zero area)ES()RP( + + +)0 P()BD(funcnts)ES( counts photons in the specified source regions and +reports the results for each region. Regions are specified using the +)0 54 1 A(Spatial Region Filtering)54 0 TN TL()Ec /AF f D( mechanism. +Photons are also counted in the specified bkgd regions applied to the +same data file or a different data file. \201Alternatively, a constant +background value in counts/pixel**2 can be specified.\202 The bkgd regions +are either paired one-to-one with source regions or pooled and +normalized by area, and then subtracted from the source counts in each +region. Displayed results include the bkgd-subtracted counts in each +region, as well as the error on the counts, the area in +each region, and the surface brightness \201cnts/area**2\202 calculated for +each region. + +)0 P(The first argument to the program specifies the FITS input image, array, or +raw event file to process. If "stdin" is specified, data are read from +the standard input. Use )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions, image sections, and filters. + +)0 P(The optional second argument is the source region descriptor. If no +region is specified, the entire field is used. + +)0 P(The background arguments can take one of two forms, depending on +whether a separate background file is specified. If the source +file is to be used for background as well, the third argument can be +either the background region, or a constant value denoting background +cnts/pixel. Alternatively, the third argument can be a background +data file, in which case the fourth argument is the background region. +If no third argument is specified, a constant value of 0 is used +\201i.e., no background\202. + +)0 P(In summary, the following command arguments are valid: +) 5 77 PR( [sh] funcnts sfile # counts in source file + [sh] funcnts sfile sregion # counts in source region + [sh] funcnts sfile sregion bregion # bkgd reg. is from source file + [sh] funcnts sfile sregion bvalue # bkgd reg. is constant + [sh] funcnts sfile sregion bfile bregion # bkgd reg. is from separate file)RP( + +)0 P(NB: unlike other Funtools programs, source and background regions are +specified as separate arguments on the command line, rather than being +placed inside brackets as part of the source and background filenames. +This is because regions in funcnts are not simply used as data +filters, but also are used to calculate areas, exposure, etc. If you +put the source region inside the brackets \201i.e. use it simply as a +filter\202 rather than specifying it as argument two, the program still +will only count photons that pass the region filter. However, the area +calculation will be performed on the whole field, since field\201\202 is the +default source region. This rarely is the desired behavior. On the +other hand, with FITS binary tables, it often is useful to put a column +filter in the filename brackets, so that only events matching the +column filter are counted inside the region. + +)0 P(For example, to extract the counts within a radius of 22 pixels from the +center of the FITS binary table snr.ev and subtract the background determined +from the same image within an annulus of radii 50-100 pixels: +) 33 82 PR( [sh] funcnts snr.ev "circle\201502,512,22\202" "annulus\201502,512,50,100\202" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001 + + + # the following source and background components were used: + source region\201s\202 + ---------------- + circle\201502,512,22\202 + + reg counts pixels + ---- ------------ --------- + 1 4382.000 1513 + + background region\201s\202 + -------------------- + annulus\201502,512,50,100\202 + + reg counts pixels)WR( + ---- ------------ --------- + all 8656.000 23572)RP( +The area units for the output columns labeled "area", "surf_bri" +\201surface brightness\202 and "surf_err" will be given either in +arc-seconds \201if appropriate WCS information is in the data file +header\201s\202\202 or in pixels. If the data file has WCS info, but you do not +want arc-second units, use the )BD(-p)ES( switch to force output in +pixels. Also, regions having zero area are not normally included in +the primary \201background-subtracted\202 table, but are included in the +secondary source and bkgd tables. If you want these regions to be +included in the primary table, use the )BD(-z)ES( switch. + +)0 P(Note that a simple sed command will extract the background-subtracted results +for further analysis: +) 6 79 PR( [sh] cat funcnts.sed + 1,/---- .*/d + /^$/,$d + + [sh] sed -f funcnts.sed funcnts.out + 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001)RP( + +)0 P(If separate source and background files are specified, )BD(funcnts)ES( will +attempt to normalize the the background area so that the background +pixel size is the same as the source pixel size. This normalization +can only take place if the appropriate WCS information is contained in +both files \201e.g. degrees/pixel values in CDELT\202. If either +file does not contain the requisite size information, the normalization +is not performed. In this case, it is the user's responsibility to +ensure that the pixel sizes are the same for the two files. + +)0 P(Normally, if more than one background region is specified, )BD(funcnts)ES( +will combine them all into a single region and use this background +region to produce the background-subtracted results for each source +region. The )BD(-m)ES( \201match multiple backgrounds\202 switch tells +)BD(funcnts)ES( to make a one to one correspondence between background and +source regions, instead of using a single combined background region. +For example, the default case is to combine 2 background +regions into a single region and then apply that region to each of the +source regions: + +) 35 82 PR( [sh] funcnts snr.ev "annulus\201502,512,0,22,n=2\202" "annulus\201502,512,50,100,n=2\202" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 + 2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 + + + # the following source and background components were used: + source region\201s\202 + ---------------- + annulus\201502,512,0,22,n=2\202 + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region\201s\202 + -------------------- + annulus\201502,512,50,100,n=2\202)WR( + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572)RP( +Note that the basic region filter rule "each photon is counted once +and no photon is counted more than once" still applies when using The +)BD(-m)ES( to match background regions. That is, if two background +regions overlap, the overlapping pixels will be counted in only one of +them. In a worst-case scenario, if two background regions are the same +region, the first will get all the counts and area and the second +will get none. + +)0 P(Using the )BD(-m)ES( switch causes )BD(funcnts)ES( to use each of the two +background regions independently with each of the two source regions: + +) 36 82 PR( [sh] funcnts -m snr.ev "annulus\201502,512,0,22,n=2\202" "ann\201502,512,50,100,n=2\202" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3087.015 56.954 150.985 2.395 23872.00 0.129 0.002 + 2 755.959 34.295 388.041 5.672 72959.99 0.010 0.000 + + + # the following source and background components were used: + source region\201s\202 + ---------------- + annulus\201502,512,0,22,n=2\202 + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region\201s\202 + -------------------- + ann\201502,512,50,100,n=2\202)WR( + + reg counts pixels + ---- ------------ --------- + 1 3975.000 9820 + 2 4681.000 13752)RP( + +)0 P(Note that most floating point quantities are displayed using "f" +format. You can change this to "g" format using the )BD(-g)ES( +switch. This can be useful when the counts in each pixel is very +small or very large. If you want maximum precision and don't care +about the columns lining up nicely, use )BD(-G)ES(, which outputs +all floating values as %.14g. + +)0 P(When counting photons using the annulus and panda \201pie and annuli\202 +shapes, it often is useful to have access to the radii \201and panda +angles\202 for each separate region. The )BD(-r)ES( switch will add radii +and angle columns to the output table: + +) 37 122 PR( [sh] funcnts -r snr.ev "annulus\201502,512,0,22,n=2\202" "ann\201502,512,50,100,n=2\202" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + # radii: arcsecs + # angles: degrees + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err radius1 radius2 angle1 angle2 + ---- ------------ --------- ------------ --------- --------- --------- --------- --------- --------- --------- --------- + 1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 0.00 88.00 NA NA + 2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 88.00 176.00 NA NA + + + # the following source and background components were used: + source region\201s\202 + ---------------- + annulus\201502,512,0,22,n=2\202 + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region\201s\202)WR( + -------------------- + ann\201502,512,50,100,n=2\202 + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572)RP( + +)0 P(Radii are given in units of pixels or arc-seconds \201depending on the +presence of WCS info\202, while the angle values \201when present\202 are in +degrees. These columns can be used to plot radial profiles. For +example, the script )BD(funcnts.plot)ES( in the funtools +distribution\202 will plot a radial profile using gnuplot \201version 3.7 or +above\202. A simplified version of this script is shown below: + +) 74 79 PR( #!/bin/sh + + if [ x"$1" = xgnuplot ]; then + if [ x`which gnuplot 2>/dev/null` = x ]; then + echo "ERROR: gnuplot not available" + exit 1 + fi + awk ' + BEGIN{HEADER=1; DATA=0; FILES=""; XLABEL="unknown"; YLABEL="unknown"} + HEADER==1{ + if\201 $1 == "#" && $2 == "data" && $3 == "file:" \202{ + if\201 FILES != "" \202 FILES = FILES "," + FILES = FILES $4 + } + else if\201 $1 == "#" && $2 == "radii:" \202{ + XLABEL = $3 + } + else if\201 $1 == "#" && $2 == "surf_bri:" \202{ + YLABEL = $3 + } + else if\201 $1 == "----" \202{ + printf "set nokey; set title \200"funcnts\201%s\202\200"\200n", FILES + printf "set xlabel \200" radius\201%s\202\200"\200n", XLABEL + printf "set ylabel \200"surf_bri\201%s\202\200"\200n", YLABEL + print "plot \200"-\200" using 3:4:6:7:8 with boxerrorbars" + HEADER = 0 + DATA = 1 + next + } + } + DATA==1{)WR( + if\201 NF == 12 \202{ + print $9, $10, \201$9+$10\202/2, $7, $8, $7-$8, $7+$8, $10-$9 + } + else{ + exit + } + } + ' | gnuplot -persist - 1>/dev/null 2>&1 + + elif [ x"$1" = xds9 ]; then + awk ' + BEGIN{HEADER=1; DATA=0; XLABEL="unknown"; YLABEL="unknown"} + HEADER==1{ + if\201 $1 == "#" && $2 == "data" && $3 == "file:" \202{ + if\201 FILES != "" \202 FILES = FILES "," + FILES = FILES $4 + } + else if\201 $1 == "#" && $2 == "radii:" \202{ + XLABEL = $3 + } + else if\201 $1 == "#" && $2 == "surf_bri:" \202{ + YLABEL = $3 + } + else if\201 $1 == "----" \202{ + printf "funcnts\201%s\202 radius\201%s\202 surf_bri\201%s\202 3\200n", FILES, XLABEL, YLABEL + HEADER = 0 + DATA = 1 + next + } + })WR( + DATA==1{ + if\201 NF == 12 \202{ + print $9, $7, $8 + } + else{ + exit + } + } + ' + else + echo "funcnts -r ... | funcnts.plot [ds9|gnuplot]" + exit 1 + fi)RP( + +Thus, to run )BD(funcnts)ES( and plot the results using gnuplot \201version 3.7 +or above\202, use: +) 1 75 PR( funcnts -r snr.ev "annulus\201502,512,0,50,n=5\202" ... | funcnts.plot gnuplot)RP( + +)0 P(The )BD(-s)ES( \201sum\202 switch causes )BD(funcnts)ES( to produce an +additional table of summed \201integrated\202 background subtracted values, +along with the default table of individual values: + +) 51 82 PR( [sh] funcnts -s snr.ev "annulus\201502,512,0,50,n=5\202" "annulus\201502,512,50,100\202" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # summed background-subtracted results + upto net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003 + 2 3776.817 65.254 457.183 4.914 79679.98 0.047 0.001 + 3 4025.492 71.972 1031.508 11.087 179775.96 0.022 0.000 + 4 4185.149 80.109 1840.851 19.786 320831.94 0.013 0.000 + 5 4415.540 90.790 2873.460 30.885 500799.90 0.009 0.000 + + + # background-subtracted results + reg counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003 + 2 895.818 35.423 345.182 3.710 60159.99 0.015 0.001 + 3 248.675 29.345 574.325 6.173 100095.98 0.002 0.000 + 4 159.657 32.321 809.343 8.699 141055.97 0.001 0.000 + 5 230.390 37.231 1032.610 11.099 179967.96 0.001 0.000 + + )WR( + # the following source and background components were used: + source region\201s\202 + ---------------- + annulus\201502,512,0,50,n=5\202 + + reg counts pixels sumcnts sumpix + ---- ------------ --------- ------------ --------- + 1 2993.000 305 2993.000 305 + 2 1241.000 940 4234.000 1245 + 3 823.000 1564 5057.000 2809 + 4 969.000 2204 6026.000 5013 + 5 1263.000 2812 7289.000 7825 + + background region\201s\202 + -------------------- + annulus\201502,512,50,100\202 + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572)RP( + +)0 P(The )BD(-t)ES( and )BD(-e)ES( switches can be used to apply timing and +exposure corrections, respectively, to the data. Please note that +these corrections are meant to be used qualitatively, since +application of more accurate correction factors is a complex and +mission-dependent effort. The algorithm for applying these simple +corrections is as follows: +) 9 63 PR( C = Raw Counts in Source Region + Ac= Area of Source Region + Tc= Exposure time for Source Data + Ec= Average exposure in Source Region, from exposure map + + B= Raw Counts in Background Region + Ab= Area of Background Region + Tb= \201Exposure\202 time for Background Data + Eb= Average exposure in Background Region, from exposure map)RP( +Then, Net Counts in Source region is +) 1 37 PR( Net= C - B * \201Ac*Tc*Ec\202/\201Ab*Tb*Eb\202)RP( +with the standard propagation of errors for the Error on Net. +The net rate would then be +) 1 27 PR( Net Rate = Net/\201Ac*Tc*Ec\202)RP( +The average exposure in each region is calculated by summing up the +pixel values in the exposure map for the given region and then +dividing by the number of pixels in that region. Exposure maps often +are generated at a block factor > 1 \201e.g., block 4 means that each +exposure pixel contains 4x4 pixels at full resolution\202 and +)BD(funcnts)ES( will deal with the blocking automatically. Using the +)BD(-e)ES( switch, you can supply both source and background exposure +files \201separated by ";"\202, if you have separate source and background +data files. If you do not supply a background exposure file to go with +a separate background data file, )BD(funcnts)ES( assumes that exposure +already has been applied to the background data file. In addition, it +assumes that the error on the pixels in the background data file is +zero. + +)0 P(NB: The )BD(-e)ES( switch assumes that the exposure map overlays the +image file )BD(exactly)ES(, except for the block factor. Each pixel in +the image is scaled by the block factor to access the corresponding +pixel in the exposure map. If your exposure map does not line up +exactly with the image, )BD(do not use)ES( the )BD(-e)ES( exposure +correction. In this case, it still is possible to perform exposure +correction )BD(if)ES( both the image and the exposure map have valid +WCS information: use the )BD(-w)ES( switch so that the transformation +from image pixel to exposure pixel uses the WCS information. That is, +each pixel in the image region will be transformed first from image +coordinates to sky coordinates, then from sky coordinates to exposure +coordinates. Please note that using )BD(-w)ES( can increase the time +required to process the exposure correction considerably. + +)0 P(A time correction can be applied to both source and +background data using the )BD(-t)ES( switch. The value for the correction can +either be a numeric constant or the name of a header parameter in +the source \201or background\202 file: +) 2 74 PR( [sh] funcnts -t 23.4 ... # number for source + [sh] funcnts -t "LIVETIME;23.4" ... # param for source, numeric for bkgd)RP( +When a time correction is specified, it is applied to the net counts +as well \201see algorithm above\202, so that the units of surface brightness +become cnts/area**2/sec. + +)0 P(The )BD(-i)ES( \201interval\202 switch is used to run )BD(funcnts)ES( on multiple +column-based intervals with only a single pass through the data. It is +equivalent to running )BD(funcnts)ES( several times with a different column +filter added to the source and background data each time. For each +interval, the full )BD(funcnts)ES( output is generated, with a linefeed +character \201^L\202 inserted between each run. In addition, the output for +each interval will contain the interval specification in its header. +Intervals are very useful for generating X-ray hardness ratios +efficiently. Of course, they are only supported when the input data +are contained in a table. + +)0 P(Two formats are supported for interval specification. The most general +format is semi-colon-delimited list of filters to be used as intervals: +) 1 73 PR( funcnts -i "pha=1:5;pha=6:10;pha=11:15" snr.ev "circle\201502,512,22\202" ...)RP( +Conceptually, this will be equivalent to running )BD(funcnts)ES( three times: +) 3 50 PR( funcnts snr.ev'[pha=1:5]' "circle\201502,512,22\202" + funcnts snr.ev'[pha=6:10]' "circle\201502,512,22\202" + funcnts snr.ev'[pha=11:15]' "circle\201502,512,22\202")RP( +However, using the )BD(-i)ES( switch will require only one pass through +the data. + +)0 P(Note that complex filters can be used to specify intervals: +) 1 64 PR( funcnts -i "pha=1:5&)SY(\160)ES(=4;pha=6:10&)SY(\160)ES(=5;pha=11:15&)SY(\160)ES(=6" snr.ev ...)RP( +The program simply runs the data through each filter in turn and generates +three )BD(funcnts)ES( outputs, separated by the line-feed character. + +)0 P(In fact, although the intent is to support intervals for hardness ratios, +the specified filters do not have to be intervals at all. Nor does one +"interval" filter have to be related to another. For example: +) 1 75 PR( funcnts -i "pha=1:5;pi=6:10;energy=11:15" snr.ev "circle\201502,512,22\202" ...)RP( +is equivalent to running )BD(funcnts)ES( three times with unrelated filter +specifications. + +)0 P(A second interval format is supported for the simple case in which a +single column is used to specify multiple homogeneous intervals for +that column. In this format, a column name is specified first, +followed by intervals: +) 1 65 PR( funcnts -i "pha;1:5;6:10;11:15" snr.ev "circle\201502,512,22\202" ...)RP( +This is equivalent to the first example, but requires less typing. The +)BD(funcnts)ES( program will simply prepend "pha=" before each of the specified +intervals. \201Note that this format does not contain the "=" character in +the column argument.\202 + +)0 P(Ordinarily, when )BD(funcnts)ES( is run on a FITS binary table \201or a +raw event table\202, one integral count is accumulated for each row +\201event\202 contained within a given region. The )BD(-v "scol[;bcol]")ES( +\201value column\202 switch will accumulate counts using the value from the +specified column for the given event. If only a single column is +specified, it is used for both the source and background regions. Two +separate columns, separated by a semi-colon, can be specified for source +and background. The special token '$none' can be used to specify that +a value column is to be used for one but not the other. For example, +'pha;$none' will use the pha column for the source but use integral +counts for the background, while '$none;pha' will do the converse. +If the value column is of type logical, then the value used will be 1 +for T and 0 for F. Value columns are used, for example, to integrate +probabilities instead of integral counts. + +)0 P(If the )BD(-T)ES( \201rdb table\202 switch is used, the output will conform +to starbase/rdb data base format: tabs will be inserted between +columns rather than spaces and line-feed will be inserted between +tables. + +)0 P(Finally, note that )BD(funcnts)ES( is an image program, even though it +can be run directly on FITS binary tables. This means that image +filtering is applied to the rows in order to ensure that the same +results are obtained regardless of whether a table or the equivalent +binned image is used. Because of this, however, the number of counts +found using )BD(funcnts)ES( can differ from the number of events found +using row-filter programs such as )BD(fundisp)ES( or )BD(funtable)ES( +For more information about these difference, see the discussion of +)0 58 1 A(Region Boundaries)58 0 TN TL()Ec /AF f D(. + + + + +)0 2 9 H(funcone)WB 72 Sn()WB 6 Sn( - cone search of a binary table containing RA, Dec columns)EA()EH( + + +)BD() 1 81 PR(funcone <switches> <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns])RP()ES( + + +)0 P() 9 84 PR( -d deccol:[hdr] # Dec column name, units \201def: DEC:d\202 + -j # join columns from list file + -J # join columns from list file, output all rows + -l listfile # read centers and radii from a list + -L listfile # read centers and radii from a list, output list rows + -n # don't use cone limits as a filter + -r racol:[hdr] # RA column name, units \201def: RA:h\202 + -x # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols + -X # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols, output all rows)RP( + + +)0 P(Funcone performs a cone search on the RA and Dec columns of a FITS +binary table. The distance from the center RA, Dec position to the RA, +Dec in each row in the table is calculated. Rows whose distance is +less than the specified radius are output. + +)0 P(The first argument to the program specifies the FITS file, raw event +file, or raw array file. If "stdin" is specified, data are read from +the standard input. Use )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions, and filters. The second +argument is the output FITS file. If "stdout" is specified, the FITS +binary table is written to the standard output. + +)0 P( +The third and fourth required arguments are the RA and Dec center +position. By default, RA is specified in hours while Dec is specified +in degrees. You can change the units of either of these by appending +the character "d" \201degrees\202, "h" \201hours\202 or "r" \201radians\202. Sexagesimal +notation is supported, with colons or spaces separating hms and dms. +\201When using spaces, please ensure that the entire string is quoted.\202 + +)0 P(The fifth required argument is the radius of the cone search. By default, +the radius value is given in degrees. The units can be changed by appending +the character "d" \201degrees\202, "r" \201radians\202, "'" \201arc minutes\202 or +'"' \201arc seconds\202. + +)0 P(By default, all +columns of the input file are copied to the output file. Selected +columns can be output using an optional sixth argument in the form: +) 1 31 PR( "column1 column1 ... columnN")RP( +A seventh argument allows you to output selected columns from the list +file when )BD(-j)ES( switch is used. Note that the RA and Dec columns +used in the cone calculation must not be de-selected. + +)0 P(Also by default, the RA and Dec column names are named "RA" and "Dec", +and are given in units of hours and degrees respectively. You can +change both the name and the units using the -r [RA] and/or -d [Dec] +switches. Once again, one of "h", "d", or "r" is appended to the +column name to specify units but in this case, there must be a colon ":" +between the name and the unit specification. + +)0 P(If the )BD(-l [listfile])ES( switch is used, then one or more of the +center RA, center Dec, and radius can be taken from a list file \201which +can be a FITS table or an ASCII column text file\202. In this case, the +third \201center RA\202, fourth \201center Dec\202, and fifth \201radius\202 command +line arguments can either be a column name in the list file \201if that +parameter varies\202 or else a numeric value \201if that parameter is +static\202. When a column name is specified for the RA, Dec, or radius, +you can append a colon followed by "h", "d", or "r" to specify units +\201also ' and " for radius\202. The cone search algorithm is run once for +each row in the list, taking RA, Dec, and radius values from the +specified columns or from static numeric values specified on the +command line. + +)0 P(When using a list, all valid rows from each iteration are written to a +single output file. Use the )BD(-x)ES( switch to help delineate which +line of the list file was used to produce the given output row\201s\202. +This switch causes the values for the center RA, Dec, radius, and row +number to be appended to the output file, in columns called RA_CEN, +DEC_CEN, RAD_CEN and CONE_KEY, respectively. Alternatively, the +)BD(-j)ES( \201join\202 switch will append all columns from the list row to +the output row \201essentially a join of the list row and input row\202, +along with the CONE_KEY row number. These two switches are mutually +exclusive. + +)0 P(The )BD(-X)ES( and )BD(-J)ES( switches write out the same data as their +lower case counterparts for each row satisfying a cone search. In +addition, these switches also write out rows from the event file that +do not satisfy any cone search. In such cases, that CONE_KEY column +will be given a value of -1 and the center and list position information +will be set to zero for the given row. Thus, all rows of the input +event file are guaranteed to be output, with rows satisfying at least +one cone search having additional search information. + +)0 P(The )BD(-L)ES( switch acts similarly to the )BD(-l)ES( switch in that it +takes centers from a list file. However, it also implicitly sets the +-j switch, so that output rows are the join of the input event row and +the center position row. In addition, this switch also writes out all +center position rows for which no event satisfies the cone search +criteria of that row. The CONE_KEY column will be given a value of -2 +for center rows that were not close to any data row and the event +columns will be zeroed out for such rows. In this way, all centers +rows are guaranteed to be output at least once. + +)0 P(If any of "all row" switches \201)BD(-X)ES(, )BD(-J)ES(, or )BD(-L)ES(\202 are +specified, then a new column named JSTAT is added to the output table. +The positive values in this column indicate the center position row number +\201starting from 1\202 in the list file that this data row successful matched +in a cone search. A value of -1 means that the data row did not match +any center position. A value of -2 means that the center position was +not matched by any data row. + +)0 P(Given a center position and radius, the cone search algorithm +calculates limit parameters for a box enclosing the specified cone, +and only tests rows whose positions values lie within those limits. +For small files, the overhead associated with this cone limit +filtering can cause the program to run more slowly than if all events +were tested. You can turn off cone limit filtering using the )BD(-n)ES( +switch to see if this speeds up the processing \201especially useful when +processing a large list of positions\202. + +)0 P(For example, the default cone search uses columns "RA" and "Dec" in hours +and degrees \201respectively\202 and RA position in hours, Dec and radius in degrees: +) 1 42 PR( funone in.fits out.fits 23.45 34.56 0.01)RP( +To specify the RA position in degrees: +) 1 44 PR( funcone in.fits out.fits 23.45d 34.56 0.01)RP( +To get RA and Dec from a list but use a static value for radius \201and +also write identifying info for each row in the list\202: +) 1 57 PR( funcone -x -l list.txt in.fits out.fits MYRA MYDec 0.01)RP( +User specified columns in degrees, RA position in hours \201sexagesimal +notation\202, Dec position in degrees \201sexagesimal notation\202 and radius +in arc minutes: +) 1 66 PR( funcone -r myRa:d -d myDec in.fits out.fits 12:30:15.5 30:12 15')RP( + + + + +)0 2 10 H(fundisp)WB 73 Sn()WB 7 Sn( - display data in a Funtools data file)EA()EH( + + +)BD() 1 62 PR(fundisp [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n])RP()ES( + + +)0 P() 5 68 PR( -f # format string for display + -l # display image as a list containing the columns X, Y, VAL + -n # don't output header + -F [c] # use specified character as column separator \201def: space\202 + -T # output in rdb/starbase format \201tab separators\202)RP( + + +)0 P()BD(fundisp)ES( displays the data in the specified +)0 42 1 A(FITS Extension)42 0 TN TL()Ec /AF f D( +and/or +)0 49 1 A(Image Section)49 0 TN TL()Ec /AF f D( +of a FITS file, or in a +)0 49 1 A(Section)49 0 TN TL()Ec /AF f D( +of a non-FITS array or raw event file. +)0 P(The first argument to the program specifies the FITS input image, array, or +raw event file to display. If "stdin" is specified, data are read from +the standard input. Use )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions, image sections, and filters. + +)0 P(If the data being displayed are columns \201either in a FITS binary table +or a raw event file\202, the individual rows are listed. Filters can be +added using bracket notation. Thus: +) 13 75 PR( [sh] fundisp "test.ev[time-\201int\202time>.15]" + X Y PHA PI TIME DX DY + ------- ------- ------- --------- ---------------- ---------- ---------- + 10 8 10 8 17.1600 8.50 10.50 + 9 9 9 9 17.1600 9.50 9.50 + 10 9 10 9 18.1600 9.50 10.50 + 10 9 10 9 18.1700 9.50 10.50 + 8 10 8 10 17.1600 10.50 8.50 + 9 10 9 10 18.1600 10.50 9.50 + 9 10 9 10 18.1700 10.50 9.50 + 10 10 10 10 19.1600 10.50 10.50 + 10 10 10 10 19.1700 10.50 10.50 + 10 10 10 10 19.1800 10.50 10.50)RP( +[NB: The FITS binary table test file test.ev, as well as the FITS +image test.fits, are contained in the funtools funtest directory.] + +)0 P(When a table is being displayed using )BD(fundisp)ES(, a second optional +argument can be used to specify the columns to display. For example: +) 12 56 PR( [sh] fundisp "test.ev[time-\201int\202time>=.99]" "x y time" + X Y TIME + -------- -------- --------------------- + 5 -6 40.99000000 + 4 -5 59.99000000 + -1 0 154.99000000 + -2 1 168.99000000 + -3 2 183.99000000 + -4 3 199.99000000 + -5 4 216.99000000 + -6 5 234.99000000 + -7 6 253.99000000)RP( + +)0 P(The special column )BD($REGION)ES( can be specified to display the +region id of each row: +) 12 89 PR( [sh $] fundisp "test.ev[time-\201int\202time>=.99&&annulus\2010 0 0 10 n=3\202]" 'x y time $REGION' + X Y TIME REGION + -------- -------- --------------------- ---------- + 5 -6 40.99000000 3 + 4 -5 59.99000000 2 + -1 0 154.99000000 1 + -2 1 168.99000000 1 + -3 2 183.99000000 2 + -4 3 199.99000000 2 + -5 4 216.99000000 2 + -6 5 234.99000000 3 + -7 6 253.99000000 3)RP( +)0 P(Here only rows with the proper fractional time and whose position also is +within one of the three annuli are displayed. +)0 P(Columns can be excluded from display using a minus sign before the +column: +) 12 64 PR( [sh $] fundisp "test.ev[time-\201int\202time>=.99]" "-time" + X Y PHA PI DX DY + -------- -------- -------- ---------- ----------- ----------- + 5 -6 5 -6 5.50 -6.50 + 4 -5 4 -5 4.50 -5.50 + -1 0 -1 0 -1.50 0.50 + -2 1 -2 1 -2.50 1.50 + -3 2 -3 2 -3.50 2.50 + -4 3 -4 3 -4.50 3.50 + -5 4 -5 4 -5.50 4.50 + -6 5 -6 5 -6.50 5.50 + -7 6 -7 6 -7.50 6.50)RP( +All columns except the time column are displayed. +)0 P(The special column )BD($N)ES( can be specified to display the +ordinal value of each row. Thus, continuing the previous example: +) 12 74 PR( fundisp "test.ev[time-\201int\202time>=.99]" '-time $n' + X Y PHA PI DX DY N + ------- -------- -------- ---------- ----------- ----------- ---------- + 5 -6 5 -6 5.50 -6.50 337 + 4 -5 4 -5 4.50 -5.50 356 + -1 0 -1 0 -1.50 0.50 451 + -2 1 -2 1 -2.50 1.50 465 + -3 2 -3 2 -3.50 2.50 480 + -4 3 -4 3 -4.50 3.50 496 + -5 4 -5 4 -5.50 4.50 513 + -6 5 -6 5 -6.50 5.50 531 + -7 6 -7 6 -7.50 6.50 550)RP( +Note that the column specification is enclosed in single quotes to protect +'$n' from begin expanded by the shell. + +)0 P(In general, the rules for activating and de-activating columns are: +)UL()-1 LI( If only exclude columns are specified, then all columns but +the exclude columns will be activated. +)-1 LI( If only include columns are specified, then only the specified columns +are activated. +)-1 LI( If a mixture of include and exclude columns are specified, then +all but the exclude columns will be active; this last case +is ambiguous and the rule is arbitrary.)LU( +In addition to specifying columns names explicitly, the special +symbols )BD(+)ES( and )BD(-)ES( can be used to activate and +de-activate )BD(all)ES( columns. This is useful if you want to +activate the $REGION column along with all other columns. According +to the rules, the syntax "$REGION" only activates the region column +and de-activates the rest. Use "+ $REGION" to activate all +columns as well as the region column. + +)0 P(If the data being displayed are image data \201either in a FITS primary +image, a FITS image extension, or an array file\202, an mxn pixel display +is produced, where m and n are the dimensions of the image. By +default, pixel values are displayed using the same data type as in the +file. However, for integer data where the BSCALE and BZERO header parameters +are present, the data is displayed as floats. In either case, the +display data type can be overridden using an optional second argument +of the form: +) 1 10 PR( bitpix=n)RP( +where n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, +respectively. + +)0 P(Of course, running )BD(fundisp)ES( on anything but the smallest image +usually results in a display whose size makes it unreadable. +Therefore, one can uses bracket notation \201see below\202 +to apply section and/or blocking to the image before generating a +display. For example: +) 9 66 PR( [sh] fundisp "test.fits[2:6,2:7]" bitpix=-32 + 2 3 4 5 6 + ---------- ---------- ---------- ---------- ---------- + 2: 3.00 4.00 5.00 6.00 7.00 + 3: 4.00 5.00 6.00 7.00 8.00 + 4: 5.00 6.00 7.00 8.00 9.00 + 5: 6.00 7.00 8.00 9.00 10.00 + 6: 7.00 8.00 9.00 10.00 11.00 + 7: 8.00 9.00 10.00 11.00 12.00)RP( + +)0 P(Note that is is possible to display a FITS binary table as an image +simply by passing the table through )BD(funimage)ES( first: +) 9 68 PR( [sh] ./funimage test.ev stdout | fundisp "stdin[2:6,2:7]" bitpix=8 + 2 3 4 5 6 + ------- ------- ------- ------- ------- + 2: 3 4 5 6 7 + 3: 4 5 6 7 8 + 4: 5 6 7 8 9 + 5: 6 7 8 9 10 + 6: 7 8 9 10 11 + 7: 8 9 10 11 12)RP( + +If the )BD(-l)ES( \201list\202 switch is used, then an image is displayed as a +list containing the columns: X, Y, VAL. For example: +) 33 45 PR( fundisp -l "test1.fits[2:6,2:7]" bitpix=-32 + X Y VAL + ---------- ---------- ----------- + 2 2 6.00 + 3 2 1.00 + 4 2 1.00 + 5 2 1.00 + 6 2 1.00 + 2 3 1.00 + 3 3 5.00 + 4 3 1.00 + 5 3 1.00 + 6 3 1.00 + 2 4 1.00 + 3 4 1.00 + 4 4 4.00 + 5 4 1.00 + 6 4 1.00 + 2 5 1.00 + 3 5 1.00 + 4 5 1.00 + 5 5 3.00 + 6 5 1.00 + 2 6 1.00 + 3 6 1.00 + 4 6 1.00 + 5 6 1.00 + 6 6 2.00 + 2 7 1.00 + 3 7 1.00 + 4 7 1.00)WR( + 5 7 1.00 + 6 7 1.00)RP( + +)0 P(If the )BD(-n)ES( \201nohead\202 switch is used, then no header is output for +tables. This is useful, for example, when fundisp output is being +directed into gnuplot. + +)0 P(The )BD(fundisp)ES( program uses a default set of display formats: +) 10 33 PR( datatype TFORM format + -------- ----- -------- + double D "%21.8f" + float E "%11.2f" + int J "%10d" + short I "%8d" + byte B "%6d" + string A "%12.12s" + bits X "%8x" + logical L "%1x")RP( +Thus, the default display of 1 double and 2 shorts gives: +) 7 42 PR( [sh] fundisp snr.ev "time x y" + + TIME X Y + --------------------- -------- -------- + 79494546.56818075 546 201 + 79488769.94469175 548 201 + ...)RP( +You can change the display format for individual columns or for all +columns of a given data types by means of the -f switch. The format +string that accompanies -f is a space-delimited list of keyword=format +values. The keyword values can either be column names \201in which case +the associated format pertains only to that column\202 or FITS table +TFORM specifiers \201in which case the format pertains to all columns +having that data type\202. For example, you can change the double and +short formats for all columns like this: +) 7 53 PR( [sh] fundisp -f "D=%22.11f I=%3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- --- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ...)RP( + +)0 P(Alternatively, you can change the format of the time and x columns like this: +) 7 56 PR( [sh] fundisp -f "time=%22.11f x=%3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- -------- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ...)RP( +Note that there is a potential conflict if a column has the same name +as one of the TFORM specifiers. In the examples above, the the "X" +column in the table has the same name as the X \201bit\202 datatype. To +resolve this conflict, the format string is processed such that +TFORM datatype specifiers are checked for first, using a +case-sensitive comparison. If the specified format value is not an +upper case TFORM value, then a case-insensitive check is made on the +column name. This means that, in the examples above, "X=%3d" will refer +to the X \201bit\202 datatype, while "x=%3d" will refer to the X column: +) 15 38 PR( [sh] fundisp -f "X=%3d" snr.ev "x y" + + X Y + -------- -------- + 546 201 + 548 201 + ... + + [sh] fundisp -f "x=%3d" snr.ev "x y" + + X Y + --- -------- + 546 201 + 548 201 + ...)RP( +As a rule, therefore, it is best always to specify the column name in +lower case and TFORM data types in upper case. + +)0 P(The )BD(-f [format])ES( will change the format for a single execution +of fundisp. You also can use the )BD(FUN_FORMAT)ES( envronment variable +to change the format for all invocations of fundisp. The format of this +environment variable's value is identical to that used with +the )BD(-f)ES( switch. This global value can be overridden in +individual cases by use of the )BD(-f [format])ES( switch. + +)0 P(Caveats: Please also note that it is the user's responsibility to +match the format specifier to the column data type correctly. Also +note that, in order to maintain visual alignment between names and +columns, the column name will be truncated \201on the left\202 if the +format width is less than the length of the name. However, truncation +is not performed if the output is in RDB format \201using the -T switch\202. + +)0 P([An older-style format string is supported but deprecated. It +consists of space-delimited C format statements for all data types, +specified in the following order: +) 1 40 PR( double float int short byte string bit.)RP( +This order of the list is based on the assumption that people generally +will want to change the float formats. +)0 P(If "-" is entered instead of a format statement for a given data type, the +default format is used. Also, the format string can be terminated without +specifying all formats, and defaults will be used for the rest of the +list. Note that you must supply a minimum field width, i.e., "%6d" and +"%-6d" are legal, "%d" is not legal. + +By using -f [format], you can change the double and short formats like this: +) 7 51 PR( [sh] fundisp -f "22.11f - - 3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- --- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ...)RP( +NB: This format is deprecated and will be removed in a future release.] + +)0 P(The )BD(-F[c])ES( switch can be used to specify a \201single-character\202 +column separator \201where the default is a space\202. Note that column +formatting will almost certainly also add spaces to pad individual +columns to the required width. These can be removed with a program +such as sed, at the cost of generating unaligned columns. For example: +) 26 117 PR(fundisp -F',' snr.ev'[cir 512 512 .1]' + X, Y, PHA, PI, TIME, DX, DY +--------,--------,--------,--------,---------------------,--------,-------- + 512, 512, 6, 7, 79493997.45854475, 578, 574 + 512, 512, 8, 9, 79494575.58943175, 579, 573 + 512, 512, 5, 6, 79493631.03866175, 578, 575 + 512, 512, 5, 5, 79493290.86521725, 578, 575 + 512, 512, 8, 9, 79493432.00990875, 579, 573 + +fundisp -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g' + X,Y,PHA,PI,TIME,DX,DY +--------,--------,--------,--------,---------------------,--------,-------- + 512,512,6,7,79493997.45854475,578,574 + 512,512,8,9,79494575.58943175,579,573 + 512,512,5,6,79493631.03866175,578,575 + 512,512,5,5,79493290.86521725,578,575 + 512,512,8,9,79493432.00990875,579,573 + +fundisp -f "x=%3d y=%3d pi=%1d pha=%1d time=%20.11f dx=%3d dy=%3d" -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g' + X,Y,A,I,TIME,DX,DY +---,---,-,-,--------------------,---,--- +512,512,6,7,79493997.45854474604,578,574 +512,512,8,9,79494575.58943174779,579,573 +512,512,5,6,79493631.03866174817,578,575 +512,512,5,5,79493290.86521725357,578,575 +512,512,8,9,79493432.00990875065,579,573)RP( + +)0 P(If the )BD(-T)ES( \201rdb table\202 switch is used, the output will conform +to starbase/rdb data base format: tabs will be inserted between +columns rather than spaces. This format is not available when +displaying image pixels \201except in conjunction with the )BD(-l)ES( +switch\202. + +)0 P(Finally, note that )BD(fundisp)ES( can be used to create column filters from +the auxiliary tables in a FITS file. For example, the following shell code +will generate a good-time interval \201GTI\202 filter for X-ray data files that +contain a standard GTI extension: +) 3 68 PR( #!/bin/sh + sed '1,/---- .*/d + /^$/,$d' | awk 'tot>0{printf "||"};{printf "time="$1":"$2; tot++}')RP( +If this script is placed in a file called "mkgti", it can be used in a +command such as: +) 1 46 PR( fundisp foo.fits"[GTI]" | mkgti > gti.filter)RP( +The resulting filter file can then be used in various funtools programs: +) 1 37 PR( funcnts foo.fits"[@gti.filter]" ...)RP( +to process only the events in the good-time intervals. + + + + +)0 2 11 H(funhead)WB 74 Sn()WB 8 Sn( - display a header in a Funtools file)EA()EH( + + +)BD() 1 50 PR(funhead [-a] [-s] [-t] [-L] <iname> [oname ename])RP()ES( + + +)0 P() 4 60 PR( -a # display all extension headers + -s # display 79 chars instead of 80 before the new-line + -t # prepend data type char to each line of output + -L # output in rdb/starbase list format)RP( + + +)0 P()BD(funhead)ES( displays the FITS header parameters in the specified +)0 42 1 A(FITS Extension)42 0 TN TL()Ec /AF f D(. +)0 P(The first argument to the program specifies the Funtools input file +to display. If "stdin" is specified, data are read from +the standard input. )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( is used to specify particular FITS extension to process. +Normally, the full 80 characters of each header card is output, +followed by a new-line. + +)0 P(If the )BD(-a)ES( switch is specified, the header from each FITS +extensions in the file is displayed. Note, however, that the )BD(-a)ES( +switch does not work with FITS files input via stdin. We hope to +remove this restriction in a future release. + +)0 P(If the )BD(-s)ES( switch is specified, only 79 characters are output +before the new-line. This helps the display on 80 character terminals. + +)0 P(If the )BD(-t)ES( switch is specified, the data type of the parameter +is output as a one character prefix, followed by 77 characters of the +param. The parameter data types are defined as: FUN_PAR_UNKNOWN +\201'u'\202, FUN_PAR_COMMENT \201'c'\202, FUN_PAR_LOGICAL \201'l'\202, FUN_PAR_INTEGER +\201'i'\202, FUN_PAR_STRING \201's'\202, FUN_PAR_REAL \201'r'\202, FUN_PAR_COMPLEX \201'x'\202. + +)0 P(If the )BD(-L)ES( \201rdb table\202 switch is used, the output will conform +to starbase/rdb data base list format. + +)0 P(For example to display the EVENTS extension \201binary table\202: +) 17 65 PR( [sh] funhead "foo.fits[EVENTS]" + XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE + BITPIX = 8 / Binary data + NAXIS = 2 / Table is a matrix + NAXIS1 = 20 / Width of table in bytes + NAXIS2 = 30760 / Number of entries in table + PCOUNT = 0 / Random parameter count + GCOUNT = 1 / Group count + TFIELDS = 7 / Number of fields in each row + EXTNAME = 'EVENTS ' / Table name + EXTVER = 1 / Version number of table + TFORM1 = '1I ' / Data type for field + TTYPE1 = 'X ' / Label for field + TUNIT1 = ' ' / Physical units for field + TFORM2 = '1I ' / Data type for field + etc. ... + END)RP( + +)0 P(To display the third header: +) 14 65 PR( [sh] funhead "foo.fits[3]" + XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE + BITPIX = 8 / Binary data + NAXIS = 2 / Table is a matrix + NAXIS1 = 32 / Width of table in bytes + NAXIS2 = 40 / Number of entries in table + PCOUNT = 0 / Random parameter count + GCOUNT = 1 / Group count + TFIELDS = 7 / Number of fields in each row + EXTNAME = 'TGR ' / Table name + EXTVER = 1 / Version number of table + TFORM1 = '1D ' / Data type for field + etc. ... + END)RP( + +)0 P(To display the primary header \201i.e., extension 0\202: +) 8 59 PR( sh> funhead "coma.fits[0]" + SIMPLE = T /STANDARD FITS FORMAT + BITPIX = 16 /2-BYTE TWOS-COMPL INTEGER + NAXIS = 2 /NUMBER OF AXES + NAXIS1 = 800 / + NAXIS2 = 800 / + DATATYPE= 'INTEGER*2' /SHORT INTEGER + END)RP( + +)0 P(The funhead program also can edit \201i.e. add, delete, or modify\202 or +display individual headers parameters. Edit mode is signalled by the +presence of two additional command-line arguments: output file and +edit command file, in that order. Edit mode acts as a filter: the +output file will contain the entire input FITS file, including other +extensions. The edit command file can be "stdin", in which case edit +command are read from the standard input. + +)0 P(The edit command file contains parameter comments \201having '#' in the +first column\202 and delete and assignment\201modify or add\202 operations. A +delete operation is specified by preceding the parameter name with a +minus sign "-". A display operation \201very useful in interactive +sessions, i.e., where the edit commands are taken from stdin\202 is +specified by preceding the parameter name with a question mark "?". In +either case, a parameter value need not be specified. An assignment +operation is specified in the same two ways that a parameter is +specified in a text header \201but without the comment character that +precedes header params\202, i.e.: + +)UL()-1 LI( FITS-style comments have an equal sign "=" between the keyword and +value and an optional slash "/" to signify a comment. The strict FITS +rules on column positions are not enforced. + +)-1 LI( Free-form comments can have an optional colon separator between the +keyword and value. In the absence of quote, all tokens after the +keyword are part of the value, i.e. no comment is allowed.)LU( + +)0 P(For example, the following interactive session checks for the +existence of parameters, adds new parameters, modifies them, and +modifies and deletes existing parameters: +) 20 55 PR( sh$ ./funhead snr.ev foo.fits - + # look for FOO1 + ? FOO1 + WARNING: FOO1 not found + # add new foo1 + FOO1 = 100 + # add foo2 + FOO2 = 200 + # reset foo1 to a different value + FOO1 -1 + # delete foo2 + -FOO2 + # change existing value + EXTVER 2 + ? XS-SORT + XS-SORT = 'EOF ' / type of event sort + # delete existing value + -XS-SORT + # exit + ^D)RP( + +)0 P(See )0 47 1 A(Column-based Text Files)47 0 TN TL()Ec /AF f D( +for more information about header parameter format. + +)0 P( +)0 P( + + +)0 2 12 H(funhist)WB 75 Sn()WB 9 Sn( - create a 1D histogram of a column \201from a FITS binary table or raw event file\202 or an image)EA()EH( + + +)BD() 1 51 PR(funhist [-n|-w|-T] <iname> [column] [[lo:hi:]bins])RP()ES( + + +)0 P() 3 61 PR( -n # normalize bin value by the width of each bin + -w # specify bin width instead of number of bins in arg3 + -T # output in rdb/starbase format \201tab separators\202)RP( + + +)0 P()BD(funhist)ES( creates a one-dimensional histogram from the specified +columns of a )0 42 1 A(FITS Extension)42 0 TN TL()Ec /AF f D( +binary table of a FITS file \201or from a non-FITS raw event file\202, or +from a FITS image or array, and writes that histogram as an ASCII +table. Alternatively, the program can perform a 1D projection of one +of the image axes. + +)0 P(The first argument to the program is required, and specifies the +Funtools file: FITS table or image, raw event file, or array. If +"stdin" is specified, data are read from the standard input. Use +)0 42 1 A(Funtools Bracket Notation)42 0 TN TL()Ec /AF f D( to specify FITS +extensions, and filters. + +)0 P(For a table, the second argument also is required. It specifies the +column to use in generating the histogram. If the data file is of +type image \201or array\202, the column is optional: if "x" \201or "X"\202, "y" +\201or "Y"\202 is specified, then a projection is performed over the x +\201dim1\202 or y \201dim2\202 axes, respectively. \201That is, this projection will +give the same results as a histogram performed on a table containing +the equivalent x,y event rows.\202 If no column name is specified or +"xy" \201or "XY"\202 is specified for the image, then a histogram is +performed on the values contained in the image pixels. + +)0 P(The argument that follows is optional and specifies the number of bins +to use in creating the histogram and, if desired, the range of bin +values. For image and table histograms, the range should specify the +min and max data values. For image histograms on the x and y axes, +the range should specify the min and max image bin values. If this +argument is omitted, the number of output bins for a table is +calculated either from the TLMIN/TLMAX headers values \201if these exist +in the table FITS header for the specified column\202 or by going through +the data to calculate the min and max value. For an image, the number +of output bins is calculated either from the DATAMIN/DATAMAX header +values, or by going through the data to calculate min and max value. +\201Note that this latter calculation might fail if the image cannot be +fit in memory.\202 If the data are floating point \201table or image\202 and +the number of bins is not specified, an arbitrary default of 128 is +used. + +)0 P(For binary table processing, the )BD(-w)ES( \201bin width\202 switch can be used +to specify the width of each bin rather than the number of bins. Thus: +) 1 29 PR( funhist test.ev pha 1:100:5)RP( +means that 5 bins of width 20 are used in the histogram, while: +) 1 32 PR( funhist -w test.ev pha 1:100:5)RP( +means that 20 bins of width 5 are used in the histogram. + +)0 P(The data are divvied up into the specified number of bins and the +resulting 1D histogram \201or projection\202 is output in ASCII table +format. For a table, the output displays the low_edge \201inclusive\202 and +hi_edge \201exclusive\202 values for the data. For example, a 15-row table +containing a "pha" column whose values range from -7.5 to 7.5 +can be processed thus: + +) 47 62 PR( [sh] funhist test.ev pha + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: -7.5 7.5 15 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 22 -7.50000000 -6.50000000 + 2 21 -6.50000000 -5.50000000 + 3 20 -5.50000000 -4.50000000 + 4 19 -4.50000000 -3.50000000 + 5 18 -3.50000000 -2.50000000 + 6 17 -2.50000000 -1.50000000 + 7 16 -1.50000000 -0.50000000 + 8 30 -0.50000000 0.50000000 + 9 16 0.50000000 1.50000000 + 10 17 1.50000000 2.50000000 + 11 18 2.50000000 3.50000000 + 12 19 3.50000000 4.50000000 + 13 20 4.50000000 5.50000000 + 14 21 5.50000000 6.50000000 + 15 22 6.50000000 7.50000000 + + [sh] funhist test.ev pha 1:6 + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: 0.5 6.5 6 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 16 0.50000000 1.50000000)WR( + 2 17 1.50000000 2.50000000 + 3 18 2.50000000 3.50000000 + 4 19 3.50000000 4.50000000 + 5 20 4.50000000 5.50000000 + 6 21 5.50000000 6.50000000 + + [sh] funhist test.ev pha 1:6:3 + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: 0.5 6.5 3 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 33 0.50000000 2.50000000 + 2 37 2.50000000 4.50000000 + 3 41 4.50000000 6.50000000)RP( + +)0 P(For a table histogram, the )BD(-n)ES(\201normalize\202 switch can be used to +normalize the bin value by the width of the bin \201i.e., hi_edge-lo_edge\202: +) 11 74 PR( [sh] funhist -n test.ev pha 1:6:3 + # data file: test.ev + # column: pha + # min,max,bins: 0.5 6.5 3 + # width normalization \201val/\201hi_edge-lo_edge\202\202 is applied + + bin value lo_edge hi_edge + ------ --------------------- --------------------- --------------------- + 1 16.50000000 0.50000000 2.50000000 + 2 6.16666667 2.50000000 4.50000000 + 3 4.10000000 4.50000000 6.50000000)RP( +This could used, for example, to produce a light curve with values +having units of counts/second instead of counts. + +)0 P(For an image histogram, the output displays the low and high image +values \201both inclusive\202 used to generate the histogram. For example, +in the following example, 184 pixels had a value of 1, 31 had a value +of 2, while only 2 had a value of 3,4,5,6, or 7: +) 13 74 PR( [sh] funhist test.fits + # data file: /home/eric/data/test.fits + # min,max,bins: 1 7 7 + + bin value lo_val hi_val + ------ --------------------- --------------------- --------------------- + 1 184.00000000 1.00000000 1.00000000 + 2 31.00000000 2.00000000 2.00000000 + 3 2.00000000 3.00000000 3.00000000 + 4 2.00000000 4.00000000 4.00000000 + 5 2.00000000 5.00000000 5.00000000 + 6 2.00000000 6.00000000 6.00000000 + 7 2.00000000 7.00000000 7.00000000)RP( + +)0 P(For the axis projection of an image, the output displays the low and +high image bins \201both inclusive\202 used to generate the projection. For +example, in the following example, 21 counts had their X bin value of +2, etc.: +) 23 74 PR( [sh] funhist test.fits x 2:7 + # data file: /home/eric/data/test.fits + # column: X + # min,max,bins: 2 7 6 + + bin value lo_bin hi_bin + ------ --------------------- --------------------- --------------------- + 1 21.00000000 2.00000000 2.00000000 + 2 20.00000000 3.00000000 3.00000000 + 3 19.00000000 4.00000000 4.00000000 + 4 18.00000000 5.00000000 5.00000000 + 5 17.00000000 6.00000000 6.00000000 + 6 16.00000000 7.00000000 7.00000000 + + [sh] funhist test.fits x 2:7:2 + # data file: /home/eric/data/test.fits + # column: X + # min,max,bins: 2 7 2 + + bin value lo_bin hi_bin + ------ --------------------- --------------------- --------------------- + 1 60.00000000 2.00000000 4.00000000 + 2 51.00000000 5.00000000 7.00000000)RP( + +)0 P(You can use gnuplot or other plotting programs to graph the +results, using a script such as: +) 7 119 PR( #!/bin/sh + sed -e '1,/---- .*/d + /^$/,$d' | \200 + awk '\200 + BEGIN{print "set nokey; set title \200"funhist\200"; set xlabel \200"bin\200"; set ylabel \200"counts\200"; plot \200"-\200" with boxes"} \200 + {print $3, $2, $4-$3}' | \200 + gnuplot -persist - 1>/dev/null 2>&1)RP( + +Similar plot commands are supplied in the script )BD(funhist.plot)ES(: +) 1 49 PR( funhist test.ev pha ... | funhist.plot gnuplot)RP( + + + + +)0 2 13 H(funimage)WB 76 Sn()WB 10 Sn( - create a FITS image from a Funtools data file)EA()EH( + + +)BD() 3 73 PR(funimage [-a] <iname> <oname> [bitpix=n] +funimage [-l] <iname> <oname> <xcol:xdims> <ycol:ydims> <vcol> [bitpix=n] +funimage [-p x|y] <iname> <oname> [bitpix=n])RP()ES( + + +)0 P() 3 65 PR( -a # append to existing output file as an image extension + -l # input is a list file containing xcol, ycol, value + -p [x|y] # project along x or y axis to create a 1D image)RP( + + +)0 P()BD(funimage)ES( creates a primary FITS image from the specified +)0 42 1 A(FITS Extension)42 0 TN TL()Ec /AF f D( +and/or +)0 49 1 A(Image Section)49 0 TN TL()Ec /AF f D( +of a FITS file, or from an +)0 49 1 A(Image Section)49 0 TN TL()Ec /AF f D( +of a non-FITS array, or from a raw event file. +)0 P(The first argument to the program specifies the FITS input image, +array, or raw event file to process. If "stdin" is specified, data are +read from the standard input. Use )0 42 1 A(Funtools +Bracket Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions, image sections, and +filters. The second argument is the output FITS file. If "stdout" is +specified, the FITS image is written to the standard output. By +default, the output pixel values are of the same data type as those of the +input file \201or type "int" when binning a table\202, but this can be +overridden using an optional third argument of the form: +) 1 10 PR( bitpix=n)RP( +where n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, +respectively. + +)0 P(If the input data are of type image, the appropriate section is +extracted and blocked \201based on how the +)0 49 1 A(Image Section)49 0 TN TL()Ec /AF f D( is specified\202, and +the result is written to the FITS primary image. When an integer +image containing the BSCALE and BZERO keywords is converted to float, +the pixel values are scaled and the scaling keywords are deleted from the +output header. When converting integer scaled data to integer +\201possibly of a different size\202, the pixels are not scaled and the +scaling keywords are retained. + +)0 P(If the input data is a binary table or raw event file, these are +binned into an image, from which a section is extracted and blocked, +and written to a primary FITS image. In this case, it is necessary to +specify the two columns that will be used in the 2D binning. This can +be done on the command line using the )BD(bincols=\201x,y\202)ES( keyword: +) 1 46 PR( funcnts "foo.ev[EVENTS,bincols=\201detx,dety\202]")RP( +The full form of the )BD(bincols=)ES( specifier is: +) 1 77 PR( bincols=\201[xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]\202)RP( +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: +) 2 56 PR( dim = \201tlmax - tlmin\202/binsiz \201floating point data\202 + dim = \201tlmax - tlmin\202/binsiz + 1 \201integer data\202)RP( +Using this syntax, it is possible to bin any two columns of a binary +table at any bin size. Note that the tlmin, tlmax, and binsiz +specifiers can be omitted if TLMIN, TLMAX, and TDBIN header parameters +\201respectively\202 are present in the FITS binary table header for the +column in question. Note also that if only one parameter is specified, +it is assumed to be tlmax, and tlmin defaults to 1. If two parameters +are specified, they are assumed to be tlmin and tlmax. +See )0 50 1 A(Binning FITS Binary Tables and Non-FITS +Event Files)50 0 TN TL()Ec /AF f D( for more information about binning parameters. + +)0 P(By default, a new 2D FITS image file is created and the image is written +to the primary HDU. If the )BD(-a)ES( \201append\202 switch is specified, +the image is appended to an existing FITS file as an IMAGE extension. +\201If the output file does not exist, the switch is effectively ignored +and the image is written to the primary HDU.\202 This can be useful in a +shell programming environment when processing multiple FITS images +that you want to combine into a single final FITS file. + +)0 P()BD(funimage)ES( also can take input from a table containing columns of +x, y, and value \201e.g., the output from )BD(fundisp -l)ES( which +displays each image x and y and the number of counts at that +position.\202 When the )BD(-l)ES( \201list\202 switch is used, the input file is +taken to be a FITS or ASCII table containing \201at least\202 three columns +that specify the x and y image coordinates and the value of that +image pixel. In this case, )BD(funimage)ES( requires four extra +arguments: xcolumn:xdims, ycolumn:ydims, vcolumn and bitpix=n. The x +and y col:dim information takes the form: +) 3 55 PR( name:dim # values range from 1 to dim + name:min:max # values range from min to max + name:min:max:binsiz # dimensions scaled by binsize)RP( +In particular, the min value should be used whenever the +minimum coordinate value is something other than one. For example: +) 2 69 PR( + funimage -l foo.lst foo.fits xcol:0:512 ycol:0:512 value bitpix=-32)RP( + +)0 P(The list feature also can be used to read unnamed columns from standard +input: simply replace the column name with a null string. Note +that the dimension information is still required: +) 5 60 PR( funimage -l stdin foo.fits "":0:512 "":0:512 "" bitpix=-32 + 240 250 1 + 255 256 2 + ... + ^D)RP( + +)0 P(The list feature provides a simple way to generate a blank image. +If you pass a )0 47 1 A(Column-based Text File)47 0 TN TL()Ec /AF f D( +to funimage in which the text header contains the required image +information, then funimage will correctly make a blank image. For +example, consider the following text file \201called foo.txt\202: +) 3 20 PR( x:I:1:10 y:I:1:10 + ------ ------ + 0 0)RP( +This text file defines two columns, x and y, each of data type 32-bit int and +image dimension 10. The command: +) 1 36 PR( funimage foo.txt foo.fits bitpix=8)RP( +will create an empty FITS image called foo.fits containing a 10x10 +image of unsigned char: +) 13 75 PR( fundisp foo.fits + 1 2 3 4 5 6 7 8 9 10 + ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ + 10: 0 0 0 0 0 0 0 0 0 0 + 9: 0 0 0 0 0 0 0 0 0 0 + 8: 0 0 0 0 0 0 0 0 0 0 + 7: 0 0 0 0 0 0 0 0 0 0 + 6: 0 0 0 0 0 0 0 0 0 0 + 5: 0 0 0 0 0 0 0 0 0 0 + 4: 0 0 0 0 0 0 0 0 0 0 + 3: 0 0 0 0 0 0 0 0 0 0 + 2: 0 0 0 0 0 0 0 0 0 0 + 1: 1 0 0 0 0 0 0 0 0 0)RP( + +Note that the text file must contain at least +one row of data. However, in the present example, event position 0,0 is +outside the limits of the image and will be ignored. \201You can, of course, +use real x,y values to seed the image with data.\202 + +)0 P(Furthermore, you can use the TEXT filter specification to obviate the need for +an input text file altogether. The following command will create the same +10x10 char image without an actual input file: +) 3 69 PR( funimage stdin'[TEXT\201x:I:10,y:I:10\202]' foo.fits bitpix=8 < /dev/null +or + funimage /dev/null'[TEXT\201x:I:10,y:I:10\202]' foo.fits bitpix=8)RP( + +)0 P(You also can use either of these methods to generate a region mask simply +by appending a region inside the filter brackets and specfying )BD(mask=all)ES( +along with the bitpix. For example, the following command will generate a +10x10 char mask using 3 regions: +) 2 76 PR( funimage stdin'[TEXT\201x:I:10,y:I:10\202,cir\2015,5,4\202,point\20110,1\202,-cir\2015,5,2\202]' \200 + foo.fits bitpix=8,mask=all < /dev/null)RP( +The resulting mask looks like this: +) 13 75 PR( fundisp foo.fits + 1 2 3 4 5 6 7 8 9 10 + ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ + 10: 0 0 0 0 0 0 0 0 0 0 + 9: 0 0 0 0 0 0 0 0 0 0 + 8: 0 0 1 1 1 1 1 0 0 0 + 7: 0 1 1 1 1 1 1 1 0 0 + 6: 0 1 1 0 0 0 1 1 0 0 + 5: 0 1 1 0 0 0 1 1 0 0 + 4: 0 1 1 0 0 0 1 1 0 0 + 3: 0 1 1 1 1 1 1 1 0 0 + 2: 0 0 1 1 1 1 1 0 0 0 + 1: 0 0 0 0 0 0 0 0 0 2)RP( + +)0 P(You can use )BD(funimage)ES( to create 1D image projections along the x +or y axis using the )BD(-p [x|y])ES( switch. This capability works for +both images and tables. For example consider a FITS table named ev.fits +containing the following rows: +) 17 19 PR( X Y + -------- -------- + 1 1 + 1 2 + 1 3 + 1 4 + 1 5 + 2 2 + 2 3 + 2 4 + 2 5 + 3 3 + 3 4 + 3 5 + 4 4 + 4 5 + 5 5)RP( +A corresponding 5x5 image, called dim2.fits, would therefore contain: +) 7 59 PR( 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 5: 1 1 1 1 1 + 4: 1 1 1 1 0 + 3: 1 1 1 0 0 + 2: 1 1 0 0 0 + 1: 1 0 0 0 0)RP( +A projection along the y axis can be performed on either the table or +the image: +) 9 59 PR( funimage -p y ev.fits stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 + + funimage -p y dim2.fits stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5)RP( + +)0 P(Furthermore, you can create a 1D image projection along any column of +a table by using the )BD(bincols=[column])ES( filter specification and +specifying a single column. For example, the following command +projects the same 1D image along the y axis of a table as use of +the )BD(-p y)ES( switch: +) 4 59 PR( funimage ev.fits'[bincols=y]' stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5)RP( + +)0 P(Examples: +)0 P(Create a FITS image from a FITS binary table: +) 1 33 PR( [sh] funimage test.ev test.fits)RP( + +)0 P(Display the FITS image generated from a blocked section of FITS binary table: +) 5 60 PR( [sh] funimage "test.ev[2:8,3:7,2]" stdout | fundisp stdin + 1 2 3 + --------- --------- --------- + 1: 20 28 36 + 2: 28 36 44)RP( + + + + +)0 2 14 H(funindex)WB 77 Sn()WB 11 Sn( - create an index for a column of a FITS binary table)EA()EH( + + +)BD() 1 42 PR(funindex <switches> <iname> <key> [oname])RP()ES( + + +)0 P() 7 71 PR( NB: these options are not compatible with Funtools processing. Please + use the defaults instead. + -c # compress output using gzip" + -a # ASCII output, ignore -c \201default: FITS table\202" + -f # FITS table output \201default: FITS table\202" + -l # long output, i.e. with key value\201s\202 \201default: long\202" + -s # short output, i.e. no key value\201s\202 \201default: long\202")RP( + + +)0 P(The funindex script creates an index for the specified column \201key\202 by +running funtable -s \201sort\202 and then saving the column value and the +record number for each sorted row. This index will be used automatically + by funtools filtering of that column, provided the index file's modification +date is later than that of the data file. + +)0 P( +The first required argument is the name of the FITS binary table +to index. Please note that text files cannot be indexed at this time. +The second required argument is the column \201key\202 name to index. While +multiple keys can be specified in principle, the funtools index processing +assume a single key and will not recognize files containing multiple keys. + +)0 P(By default, the output index file name is [root]_[key].idx, where [root] +is the root of the input file. Funtools looks for this specific file name +when deciding whether to use an index for faster filtering. Therefore, the +optional third argument \201output file name\202 should not be used for funtools +processing. + +)0 P(For example, to create an index on column Y for a given FITS file, use: +) 1 21 PR( funindex foo.fits Y)RP( +This will generate an index named foo_y.idx, which will be used by funtools +for filters involving the Y column. + + + + +)0 2 15 H(funjoin)WB 78 Sn()WB 12 Sn( - join two or more FITS binary tables on specified columns)EA()EH( + + +)BD() 1 57 PR(funjoin [switches] <ifile1> <ifile2> ... <ifilen> <ofile>)RP()ES( + + +)0 P() 11 74 PR( -a cols # columns to activate in all files + -a1 cols ... an cols # columns to activate in each file + -b 'c1:bvl,c2:bv2' # blank values for common columns in all files + -bn 'c1:bv1,c2:bv2' # blank values for columns in specific files + -j col # column to join in all files + -j1 col ... jn col # column to join in each file + -m min # min matches to output a row + -M max # max matches to output a row + -s # add 'jfiles' status column + -S col # add col as status column + -t tol # tolerance for joining numeric cols [2 files only])RP( + + +)BD(funjoin)ES( joins rows from two or more \201up to 32\202 +)0 42 1 A(FITS Binary Table files)42 0 TN TL()Ec /AF f D(, based on the values +of specified join columns in each file. NB: the join columns must have +an index file associated with it. These files are generated using the +)BD(funindex)ES( program. + +)0 P(The first argument to the program specifies the first input FITS table +or raw event file. If "stdin" is specified, data are read from the +standard input. Subsequent arguments specify additional event files +and tables to join. The last argument is the output FITS file. + +)0 P(NB: Do )BD(not)ES( use )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions and row filters when running +funjoin or you will get wrong results. Rows are accessed and joined +using the index files directly, and this bypasses all filtering. + +)0 P(The join columns are specified using the )BD(-j col)ES( switch \201which +specifies a column name to use for all files\202 or with )BD(-j1 col1)ES(, +)BD(-j2 col2)ES(, ... )BD(-jn coln)ES( switches \201which specify a column +name to use for each file\202. A join column must be specified for each file. +If both )BD(-j col)ES( and )BD(-jn coln)ES( are specified for a given +file, then the latter is used. Join columns must either be of type +string or type numeric; it is illegal to mix numeric and string +columns in a given join. For example, to join three files using the +same key column for each file, use: +) 1 52 PR( funjoin -j key in1.fits in2.fits in3.fits out.fits)RP( +A different key can be specified for the third file in this way: +) 1 65 PR( funjoin -j key -j3 otherkey in1.fits in2.fits in3.fits out.fits)RP( + +)0 P(The )BD(-a "cols")ES( switch \201and )BD(-a1 "col1")ES(, +)BD(-a2 "cols2")ES( counterparts\202 can be used to specify columns to +activate \201i.e. write to the output file\202 for each input file. By +default, all columns are output. + +)0 P(If two or more columns from separate files have the same name, the +second \201and subsequent\202 columns are renamed to have an underscore +and a numeric value appended. + +)0 P(The )BD(-m min)ES( and )BD(-M max)ES( switches specify the minimum +and maximum number of joins required to write out a row. The default +minimum is 0 joins \201i.e. all rows are written out\202 and the default maximum +is 63 \201the maximum number of possible joins with a limit of 32 input files\202. +For example, to write out only those rows in which exactly two files +have columns that match \201i.e. one join\202: +) 1 66 PR( funjoin -j key -m 1 -M 1 in1.fits in2.fits in3.fits ... out.fits)RP( + +)0 P(A given row can have the requisite number of joins without all of the +files being joined \201e.g. three files are being joined but only two +have a given join key value\202. In this case, all of the columns of the +non-joined file are written out, by default, using blanks \201zeros or NULLs\202. +The )BD(-b c1:bv1,c2:bv2)ES( and +)BD(-b1 'c1:bv1,c2:bv2' -b2 'c1:bv1,c2:bv2' ...)ES( +switches can be used to set the blank value for columns common to all +files and/or columns in a specified file, respectively. Each blank value +string contains a comma-separated list of column:blank_val specifiers. +For floating point values \201single or double\202, a case-insensitive string +value of "nan" means that the IEEE NaN \201not-a-number\202 should be +used. Thus, for example: +) 1 62 PR( funjoin -b "AKEY:???" -b1 "A:-1" -b3 "G:NaN,E:-1,F:-100" ...)RP( +means that a non-joined AKEY column in any file will contain the +string "???", the non-joined A column of file 1 will contain a value +of -1, the non-joined G column of file 3 will contain IEEE NaNs, while +the non-joined E and F columns of the same file will contain values -1 +and -100, respectively. Of course, where common and specific blank values +are specified for the same column, the specific blank value is used. + +)0 P(To distinguish which files are non-blank components of a given row, +the )BD(-s)ES( \201status\202 switch can be used to add a bitmask column named +"JFILES" to the output file. In this column, a bit is set for each +non-blank file composing the given row, with bit 0 corresponds to the +first file, bit 1 to the second file, and so on. The file names +themselves are stored in the FITS header as parameters named JFILE1, +JFILE2, etc. The )BD(-S col)ES( switch allows you to change the name +of the status column from the default "JFILES". + +)0 P(A join between rows is the Cartesian product of all rows in one file +having a given join column value with all rows in a second file having +the same value for its join column and so on. Thus, if file1 has 2 +rows with join column value 100, file2 has 3 rows with the same value, +and file3 has 4 rows, then the join results in 2*3*4=24 rows being output. + +)0 P(The join algorithm directly processes the index file associated with +the join column of each file. The smallest value of all the current +columns is selected as a base, and this value is used to join +equal-valued columns in the other files. In this way, the index files +are traversed exactly once. + +)0 P(The )BD(-t tol)ES( switch specifies a tolerance value for numeric +columns. At present, a tolerance value can join only two files at a +time. \201A completely different algorithm is required to join more than +two files using a tolerance, somethng we might consider implementing +in the future.\202 + +)0 P(The following example shows many of the features of funjoin. The input files +t1.fits, t2.fits, and t3.fits contain the following columns: +) 28 49 PR( [sh] fundisp t1.fits + AKEY KEY A B + ----------- ------ ------ ------ + aaa 0 0 1 + bbb 1 3 4 + ccc 2 6 7 + ddd 3 9 10 + eee 4 12 13 + fff 5 15 16 + ggg 6 18 19 + hhh 7 21 22 + +fundisp t2.fits + AKEY KEY C D + ----------- ------ ------ ------ + iii 8 24 25 + ggg 6 18 19 + eee 4 12 13 + ccc 2 6 7 + aaa 0 0 1 + +fundisp t3.fits + AKEY KEY E F G +------------ ------ -------- -------- ----------- + ggg 6 18 19 100.10 + jjj 9 27 28 200.20 + aaa 0 0 1 300.30 + ddd 3 9 10 400.40)RP( + +)0 P(Given these input files, the following funjoin command: +) 3 59 PR( funjoin -s -a1 "-B" -a2 "-D" -a3 "-E" -b \200 + "AKEY:???" -b1 "AKEY:XXX,A:255" -b3 "G:NaN,E:-1,F:-100" \200 + -j key t1.fits t2.fits t3.fits foo.fits)RP( +will join the files on the KEY column, outputting all columns except B +\201in t1.fits\202, D \201in t2.fits\202 and E \201in t3.fits\202, and setting blank +values for AKEY \201globally, but overridden for t1.fits\202 and A \201in file +1\202 and G, E, and F \201in file 3\202. A JFILES column will be output to +flag which files were used in each row: +) 12 105 PR( AKEY KEY A AKEY_2 KEY_2 C AKEY_3 KEY_3 F G JFILES + ------------ ------ ------ ------------ ------ ------ ------------ ------ -------- ----------- -------- + aaa 0 0 aaa 0 0 aaa 0 1 300.30 7 + bbb 1 3 ??? 0 0 ??? 0 -100 nan 1 + ccc 2 6 ccc 2 6 ??? 0 -100 nan 3 + ddd 3 9 ??? 0 0 ddd 3 10 400.40 5 + eee 4 12 eee 4 12 ??? 0 -100 nan 3 + fff 5 15 ??? 0 0 ??? 0 -100 nan 1 + ggg 6 18 ggg 6 18 ggg 6 19 100.10 7 + hhh 7 21 ??? 0 0 ??? 0 -100 nan 1 + XXX 0 255 iii 8 24 ??? 0 -100 nan 2 + XXX 0 255 ??? 0 0 jjj 9 28 200.20 4)RP( + + + +)0 2 16 H(funmerge)WB 79 Sn()WB 13 Sn( - merge one or more Funtools table files)EA()EH( + + +)BD() 1 61 PR(funmerge [-w|-x] -f [colname] <iname1> <iname2> ... <oname>)RP()ES( + + +)0 P() 3 68 PR( -f # output a column specifying file from which this event came + -w # adjust position values using WCS info + -x # adjust position values using WCS info and save old values)RP( + + +)0 P()BD(funmerge)ES( merges FITS data from one or more +)0 42 1 A(FITS Binary Table files)42 0 TN TL()Ec /AF f D( +or raw event files. +)0 P(The first argument to the program specifies the first input FITS table +or raw event file. If "stdin" is specified, data are read from the +standard input. Use )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions and row filters. Subsequent +arguments specify additional event files and tables to merge. \201NB: Stdin +cannot not be used for any of these additional input file arguments.\202 +The last argument is the output FITS file. The columns in each input table +must be identical. + +)0 P(If an input file begins with the '@' character, it is processed as an +include file, i.e., as a text file containing event file names \201as +well as blank lines and/or comment lines starting with the '#' sign\202. +If standard input is specified as an include file \201'@stdin'\202, then +file names are read from the standard input until EOF \201^D\202. Event +files and include files can be mixed on a command line. + +)0 P(Rows from each table are written sequentially to the output +file. If the switch )BD(-f [colname])ES( is specified on the command +line, an additional column is added to each row containing the number +of the file from which that row was taken \201starting from one\202. In +this case, the corresponding file names are stored in the header +parameters having the prefix )BD(FUNFIL)ES(, i.e., FUNFIL01, +FUNFIL02, etc. + +)0 P(Using the )BD(-w)ES( switch \201or )BD(-x)ES( switch as described +below\202, )BD(funmerge)ES( also can adjust the position column values +using the WCS information in each file. \201By position columns, we mean +the columns that the table is binned on, i.e., those columns defined +by the )BD(bincols=)ES( switch, or \201X,Y\202 by default.\202 To perform WCS +alignment, the WCS of the first file is taken as the base WCS. Each +position in subsequent files is adjusted by first converting it to the +sky coordinate in its own WCS coordinate system, then by converting +this sky position to the sky position of the base WCS, and finally +converting back to a pixel position in the base system. Note that in +order to perform WCS alignment, the appropriate WCS and TLMIN/TLMAX +keywords must already exist in each FITS file. +)0 P(When performing WCS alignment, you can save the original positions in +the output file by using the )BD(-x)ES( \201for "xtra"\202 switch instead +of the )BD(-w)ES( switch \201i.e., using this switch also implies using +)BD(-w)ES(\202 The old positions are saved in columns having the same +name as the original positional columns, with the added prefix "OLD_". +)0 P(Examples: + +)0 P(Merge two tables, and preserve the originating file number for +each row in the column called "FILE" \201along with the corresponding +file name in the header\202: +) 1 51 PR( [sh] funmerge -f "FILE" test.ev test2.ev merge.ev)RP( + +)0 P(Merge two tables with WCS alignment, saving the old position values in +2 additional columns: +) 1 44 PR( [sh] funmerge -x test.ev test2.ev merge.ev)RP( + +)0 P(This program only works on raw event files and binary tables. We have +not yet implemented image and array merging. + + + + +)0 2 17 H(funsky)WB 80 Sn()WB 14 Sn( - convert between image and sky coordinates)EA()EH( + + +)BD() 4 72 PR( funsky iname[ext] # RA,Dec \201deg\202 or image pix from stdin + funsky iname[ext] [lname] # RA, Dec \201deg\202 or image pix from list + funsky iname[ext] [col1] [col2] # named cols:units from stdin + funsky iname[ext] [lname] [col1] [col2] # named cols:units from list)RP()ES( + + +)0 P() 5 73 PR( -d # always use integer tlmin conversion \201as ds9 does\202 + -r # convert x,y to RA,Dec \201default: convert RA,Dec to x,y\202 + -o # include offset from the nominal target position \201in arcsec\202 + -v # display input values also \201default: display output only\202 + -T # output display in rdb format \201w/header,tab delimiters\202)RP( + + +)0 P(Funsky converts input sky coordinates \201RA, Dec\202 to image coordinates \201or vice +versa\202 using the WCS information contained in the specified FITS file. Several +calling sequences are supported in order to make it easy to specify +coordinate positions in different ways. + +)0 P(The first required argument is always the input FITS file \201or +extension\202 containing the WCS information in an extension header. Note +that the data from this file is not used. By default, the program +converts input RA and Dec values to X and Y using this WCS +information. If the WCS is associated with a FITS image, then the X,Y +values are image values. If the WCS is associated with a binary table, +then the X, Y values are physical values. To convert X,Y to RA and +Dec, use the )BD(-r)ES( \201reverse\202 switch. + +)0 P(If no other command arguments are supplied, then the input positions +are read from the standard input. Each line is assumed to contain a +single coordinate position consisting of an RA in degrees \201or X in +pixels\202 followed by a Dec in degrees \201or Y in pixels\202. The usual +delimiters are supported \201spaces, commas, tabs\202. For example: +) 9 52 PR( # read from stdin, default column names and units + [sh] funsky snr.ev + 22.982695 58.606523 # input RA \201hrs\202, Dec\201deg\202 + 510.00 510.00 + 22.982127 58.607634 # input + 512.00 510.50 + 22.981700 58.614301 # input + 513.50 513.50 + ^D # end of input)RP( + +)0 P(If a second argument is supplied, this argument is assumed to be +a file containing RA \201X\202 and Dec \201Y\202 positions. The file can either be +an ASCII table or a FITS binary table. The order of columns is +unimportant, if the table has a column header. In this case, the +names of the columns must be one of "RA", "DEC", or "X", "Y" for sky +to image and image to sky conversions, respectively. If the table has +no header, then once again, RA \201X\202 is assumed to first, followed +by DEC \201Y\202. +For example: +) 12 50 PR( # read from file, default column names and units + [sh] cat hd.in + RA DEC + --------- --------- + 22.982695 58.606523 + 22.982127 58.607634 + 22.981700 58.614301 + + [sh] funsky snr.ev hd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50)RP( + +)0 P(If three arguments are supplied, then the input positions again are +read from the standard input. Each line is assumed to contain a single +coordinate position consisting of an RA \201or X in pixels\202 followed by a +Dec \201or Y in pixels\202, with the usual delimiters supported. However, +the second and third arguments now specify the column names and/or +sky units using a colon-delimited syntax: +) 1 19 PR( [colname]:[h|d|r])RP( +If the colname is omitted, the names default to "RA", "DEC", "X", "Y", +"COL1", or "COL2" as above. If the units are omitted, the default is degrees +for both RA and Dec. When the -r switch is used \201convert from image +to sky\202 the units are applied to the output instead of the input. The following +examples will serve to illustrate the options: +) 36 66 PR( # read from stdin, specifying column names \201def. units: degrees\202 + [sh] cat hd.in + MYRA MYDEC + --------- --------- + 22.982695 58.606523 + 22.982127 58.607634 + 22.981700 58.614301 + + [sh] funsky snr.ev MYRA MYDEC < hd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read from stdin, specifying column names and units + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev MYRA:d MYDEC:d < dd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read stdin, convert image to sky, specifying output sky units + [sh] cat im.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50)WR( + + [sh] cat im.in | funsky -r snr.ev :d :d + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301)RP( + +)0 P(Finally, four command arguments specify both and input file and column names +and/or units: +) 22 64 PR( [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev dd.in MYRA:d MYDEC:d + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read file, convert image to sky, specifying output sky units + [sh] cat im.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] funsky -r snr.ev im.in :d :d + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301)RP( + +)0 P(By default, the output of funsky consists only of the converted coordinate +position\201s\202, one per output line. This makes parsing in shell scripts easy. +Use the )BD(-v)ES( \201verbose\202 switch to specify that the input +coordinates should be pre-pended to each line. For example: +) 16 53 PR( [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev dd.in MYRA:d MYDEC:d + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] funsky -v snr.ev dd.in MYRA:d MYDEC:d + 344.740432 58.606523 510.00 510.00 + 344.731900 58.607634 512.00 510.50 + 344.725500 58.614301 513.50 513.50)RP( + +)0 P(In addition, a full starbase table can be output using the )BD(-T)ES( +\201table\202 switch. This switch can be used with or without the -v +switch. If the -T and -v are both specified, then a descriptive header +parameters are output before the table \201mainly to remind you of the +sky units\202: +) 23 62 PR( # output table in non-verbose mode + [sh] funsky -T snr.ev dd.in MYRA:d MYDEC:d + X Y + ------------ ------------ + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # output table in verbose mode + [sh] funsky -T -v snr.ev dd.in MYRA:d MYDEC:d + # IFILE = /Users/eric/data/snr.ev + # ICOL1 = MYRA + # ICOL2 = MYDEC + # IUNITS1 = d + # IUNITS2 = d + # OCOL1 = X + # OCOL2 = Y + + MYRA MYDEC X Y + ------------ ------------ ------------ ------------ + 344.740432 58.606523 510.00 510.00 + 344.731900 58.607634 512.00 510.50 + 344.725500 58.614301 513.50 513.50)RP( + +)0 P(Finally, the )BD(-d)ES( \201ds9\202 switch mimicks ds9's use of integer TLMIN +and TLMAX values for all coordinate transformations. FITS conventions +seem to call for use of floating point TLMIN and TLMAX when the data are +floats. This convention is followed by funsky but results in a +small discrepancy with ds9's converted values for floating point +data. We will remedy this conflict in the future, maybe. + + + + +)0 2 18 H(funtable)WB 81 Sn()WB 15 Sn( - copy selected rows from a Funtools file to a FITS binary table)EA()EH( + + +)BD() 1 62 PR(funtable [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns])RP()ES( + + +)0 P() 5 61 PR( -a # append to existing output file as a table extension + -i # for image data, only generate X and Y columns + -m # for tables, write a separate file for each region + -s "col1 ..." # columns on which to sort + -z # for image data, output zero-valued pixels)RP( + + +)0 P()BD(funtable)ES( selects rows from the specified +)0 42 1 A(FITS Extension)42 0 TN TL()Ec /AF f D( +\201binary table only\202 of a FITS file, or from a non-FITS raw event +file, and writes those rows to a FITS binary table file. It also +will create a FITS binary table from an image or a raw array file. + +)0 P(The first argument to the program specifies the FITS file, raw event +file, or raw array file. If "stdin" is specified, data are read from +the standard input. Use )0 42 1 A(Funtools Bracket +Notation)42 0 TN TL()Ec /AF f D( to specify FITS extensions, and filters. The second +argument is the output FITS file. If "stdout" is specified, the FITS +binary table is written to the standard output. By default, all +columns of the input file are copied to the output file. Selected +columns can be output using an optional third argument in the form: +) 1 31 PR( "column1 column1 ... columnN")RP( + +)0 P(The )BD(funtable)ES( program generally is used to select rows from a +FITS binary table using +)0 52 1 A(Table Filters)52 0 TN TL()Ec /AF f D( +and/or +)0 54 1 A(Spatial Region Filters)54 0 TN TL()Ec /AF f D(. +For example, you can copy only selected rows \201and output only selected +columns\202 by executing in a command such as: +) 13 75 PR( [sh] funtable "test.ev[pha==1&)SY(\160)ES(==10]" stdout "x y pi pha" | fundisp stdin + X Y PHA PI + ------- ------- ------- --------- + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10)RP( +)0 P(The special column )BD($REGION)ES( can be specified to write the +region id of each row: +) 12 113 PR( [sh $] funtable "test.ev[time-\201int\202time>=.99&&annulus\2010 0 0 10 n=3\202]" stdout 'x y time $REGION' | fundisp stdin + X Y TIME REGION + -------- -------- --------------------- ---------- + 5 -6 40.99000000 3 + 4 -5 59.99000000 2 + -1 0 154.99000000 1 + -2 1 168.99000000 1 + -3 2 183.99000000 2 + -4 3 199.99000000 2 + -5 4 216.99000000 2 + -6 5 234.99000000 3 + -7 6 253.99000000 3)RP( +)0 P(Here only rows with the proper fractional time and whose position also is +within one of the three annuli are written. +)0 P(Columns can be excluded from display using a minus sign before the +column: +) 12 79 PR( [sh $] funtable "test.ev[time-\201int\202time>=.99]" stdout "-time" | fundisp stdin + X Y PHA PI DX DY + -------- -------- -------- ---------- ----------- ----------- + 5 -6 5 -6 5.50 -6.50 + 4 -5 4 -5 4.50 -5.50 + -1 0 -1 0 -1.50 0.50 + -2 1 -2 1 -2.50 1.50 + -3 2 -3 2 -3.50 2.50 + -4 3 -4 3 -4.50 3.50 + -5 4 -5 4 -5.50 4.50 + -6 5 -6 5 -6.50 5.50 + -7 6 -7 6 -7.50 6.50)RP( +All columns except the time column are written. +)0 P(In general, the rules for activating and de-activating columns are: +)UL()-1 LI( If only exclude columns are specified, then all columns but +the exclude columns will be activated. +)-1 LI( If only include columns are specified, then only the specified columns +are activated. +)-1 LI( If a mixture of include and exclude columns are specified, then +all but the exclude columns will be active; this last case +is ambiguous and the rule is arbitrary.)LU( +In addition to specifying columns names explicitly, the special +symbols )EM(+)ES( and )EM(-)ES( can be used to activate and +de-activate )EM(all)ES( columns. This is useful if you want to +activate the $REGION column along with all other columns. According +to the rules, the syntax "$REGION" only activates the region column +and de-activates the rest. Use "+ $REGION" to activate all +columns as well as the region column. + +)0 P(Ordinarily, only the selected table is copied to the output file. In +a FITS binary table, it sometimes is desirable to copy all of the +other FITS extensions to the output file as well. This can be done by +appending a '+' sign to the name of the extension in the input file +name. For example, the first command below copies only the EVENT table, +while the second command copies other extensions as well: +) 2 64 PR( [sh] funtable "/proj/rd/data/snr.ev[EVENTS]" events.ev + [sh] funtable "/proj/rd/data/snr.ev[EVENTS+]" eventsandmore.ev)RP( + +)0 P(If the input file is an image or a raw array file, then +)BD(funtable)ES( will generate a FITS binary table from the pixel +values in the image. Note that it is not possible to specify the +columns to output \201using command-line argument 3\202. Instead, there are +two ways to create such a binary table from an image. By default, a +3-column table is generated, where the columns are "X", "Y", and +"VALUE". For each pixel in the image, a single row \201event\202 is +generated with the "X" and "Y" columns assigned the dim1 and dim2 +values of the image pixel, respectively and the "VALUE" column +assigned the value of the pixel. With sort of table, running +)BD(funhist)ES( on the "VALUE" column will give the same results as +running )BD(funhist)ES( on the original image. + +)0 P(If the )BD(-i)ES( \201"individual" rows\202 switch is specified, then only +the "X" and "Y" columns are generated. In this case, each positive +pixel value in the image generates n rows \201events\202, where n is equal +to the integerized value of that pixel \201plus 0.5, for floating point +data\202. In effect, )BD(-i)ES( approximately recreates the rows of a +table that would have been binned into the input image. \201Of course, +this is only approximately correct, since the resulting x,y positions +are integerized.\202 + +)0 P(If the )BD(-s [col1 col2 ... coln])ES( \201"sort"\202 switch is specified, +the output rows of a binary table will be sorted using the +specified columns as sort keys. The sort keys must be scalar columns +and also must be part of the output file \201i.e. you cannot sort on a +column but not include it in the output\202. This facility uses the +)BD(_sort)ES( program \201included with funtools\202, which must be accessible +via your path. + +)0 P(For binary tables, the )BD(-m)ES( \201"multiple files"\202 switch will +generate a separate file for each region in the filter specification +i.e. each file contains only the rows from that region. Rows +which pass the filter but are not in any region also are put in a +separate file. + +)0 P(The separate output file names generated by the )BD(-m)ES( switch are +produced automatically from the root output file to contain the region id of +the associated region. \201Note that region ids start at 1, so that the +file name associated with id 0 contains rows that pass the filter but +are not in any given region.\202 Output file names are generated as follows: + +)UL()-1 LI( A $n specification can be used anywhere in the root file name \201suitably +quoted to protect it from the shell\202 and will be expanded to be the id +number of the associated region. For example: +) 1 78 PR( funtable -m input.fits'[cir\201512,512,1\202;cir\201520,520,1\202...]' 'foo.goo_$n.fits')RP( +will generate files named foo.goo_0.fits \201for rows not in any region but +still passing the filter\202, foo.goo_1.fits \201rows in region id #1, the first +region\202, foo.goo_2.fits \201rows in region id #2\202, etc. Note that single quotes +in the output root are required to protect the '$' from the shell. + +)-1 LI( If $n is not specified, then the region id will be placed before +the first dot \201.\202 in the filename. Thus: +) 1 73 PR( funtable -m input.fits'[cir\201512,512,1\202;cir\201520,520,1\202...]' foo.evt.fits)RP( +will generate files named foo0.evt.fits \201for rows not in any region but +still passing the filter\202, foo1.evt.fits \201rows in region id #1\202, +foo2.evt.fits \201rows in region id #2\202, etc. + +)-1 LI( If no dot is specified in the root output file name, then +the region id will be appended to the filename. Thus: +) 1 70 PR( funtable -m input.fits'[cir\201512,512,1\202;cir\201520,520,1\202...]' 'foo_evt')RP( +will generate files named foo_evt0 \201for rows not in any region but +still passing the filter\202, foo_evt1 \201rows in region id #1\202, +foo_evt2 \201rows in region id #2\202, etc.)LU( +The multiple file mechanism provide a simple way to generate +individual source data files with a single pass through the data. + +)0 P(By default, a new FITS file is created and the binary table is written +to the first extension. If the )BD(-a)ES( \201append\202 switch is specified, +the table is appended to an existing FITS file as a BINTABLE extension. +Note that the output FITS file must already exist. + +)0 P(If the )BD(-z)ES( \201"zero" pixel values\202 switch is specified and +)BD(-i)ES( is not specified, then pixels having a zero value will +be output with their "VALUE" column set to zero. Obviously, this +switch does not make sense when individual events are output. + + + + +)0 2 19 H(funtbl)WB 82 Sn()WB 16 Sn( - extract a table from Funtools ASCII output)EA()EH( + + +)BD() 1 61 PR(funtable [-c cols] [-h] [-n table] [-p prog] [-s sep] <iname>)RP()ES( + + +)0 P([NB: This program has been deprecated in favor of the ASCII text processing +support in funtools. You can now perform fundisp on funtools ASCII output +files \201specifying the table using bracket notation\202 to extract tables +and columns.] + +The )BD(funtbl)ES( script extracts a specified table \201without the +header and comments\202 from a funtools ASCII output file and writes the +result to the standard output. The first non-switch argument is the +ASCII input file name \201i.e. the saved output from funcnts, fundisp, +funhist, etc.\202. If no filename is specified, stdin is read. The +-n switch specifies which table \201starting from 1\202 to extract. The +default is to extract the first table. The -c switch is a +space-delimited list of column numbers to output, e.g. -c "1 3 5" +will extract the first three odd-numbered columns. The default is to +extract all columns. The -s switch specifies the separator string to +put between columns. The default is a single space. The -h switch +specifies that column names should be added in a header line before +the data is output. Without the switch, no header is prepended. The +-p program switch allows you to specify an awk-like program to run +instead of the default \201which is host-specific and is determined at +build time\202. The -T switch will output the data in rdb format \201i.e., +with a 2-row header of column names and dashes, and with data columns +separated by tabs\202. The -help switch will print out a message +describing program usage. + +)0 P(For example, consider the output from the following funcnts command: +) 37 82 PR( [sh] funcnts -sr snr.ev "ann 512 512 0 9 n=3" + # source + # data file: /proj/rd/data/snr.ev + # arcsec/pixel: 8 + # background + # constant value: 0.000000 + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # summed background-subtracted results + upto net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 + 2 625.000 25.000 0.000 0.000 6976.00 0.090 0.004 + 3 1442.000 37.974 0.000 0.000 15936.00 0.090 0.002 + + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 + 2 478.000 21.863 0.000 0.000 5376.00 0.089 0.004 + 3 817.000 28.583 0.000 0.000 8960.00 0.091 0.003 + + + # the following source and background components were used: + source_region\201s\202 + ---------------- + ann 512 512 0 9 n=3)WR( + + reg counts pixels sumcnts sumpix + ---- ------------ --------- ------------ --------- + 1 147.000 25 147.000 25 + 2 478.000 84 625.000 109 + 3 817.000 140 1442.000 249)RP( +)0 P(There are four tables in this output. To extract the last one, you +can execute: +) 4 60 PR( [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -n 4 + 1 147.000 25 147.000 25 + 2 478.000 84 625.000 109 + 3 817.000 140 1442.000 249)RP( +Note that the output has been re-formatted so that only a single space +separates each column, with no extraneous header or comment information. + +)0 P(To extract only columns 1,2, and 4 from the last example \201but with a header +prepended and tabs between columns\202, you can execute: +) 5 82 PR( [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -c "1 2 4" -h -n 4 -s "\200t" + #reg counts sumcnts + 1 147.000 147.000 + 2 478.000 625.000 + 3 817.000 1442.000)RP( +)0 P(Of course, if the output has previously been saved in a file named +foo.out, the same result can be obtained by executing: +) 5 48 PR( [sh] funtbl -c "1 2 4" -h -n 4 -s "\200t" foo.out + #reg counts sumcnts + 1 147.000 147.000 + 2 478.000 625.000 + 3 817.000 1442.000)RP( + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 20 H(Last)WB 83 Sn( updated: April 1, 2007)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (ds9.html) D +/Ti (Funtools and DS9 Image Display) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 17 Sn( + + +)0 2 21 H(FunDS9:)WB 85 Sn()WB 84 Sn( Funtools and DS9 Image Display)EA()EH( + + +)0 2 22 H(Summary)WB 86 Sn()EH( +Describes how funtools can be integrated into the ds9 Analysis menu. + + +)0 2 23 H(Description)WB 87 Sn()EH( +)0 P()R1 2 A(SAOImage/DS9)EA( is an astronomical imaging and data visualization +application used by astronomers around the world. DS9 can display +standard astronomical FITS images and binary tables, but also has +support for displaying raw array files, shared memory files, and data +files automatically retrieved via FTP and HTTP. Standard functional +capabilities include multiple frame buffers, colormap and region +manipulation, and many data scaling algorithms. DS9's advanced +features include TrueColor visuals, deep frame buffers, true +PostScript printing, and display of image mosaics. The program's +support of image tiling, "blinking", arbitrary zoom, rotation, and pan +is unparalleled in astronomy. It also has innovative support for +automatic retrieval and display of standard image data such as the +Digital Sky Survey \201using servers at SAO, StScI, or ESO\202. + +)0 P(DS9 can communicate with external programs such as Funtools using the +)R2 2 A(XPA)EA( +messaging system. In addition, programs can be integrated directly +into the DS9 GUI by means of a configurable Analysis menu. By +default, the DS9 Analysis menu contains algorithms deemed essential to +the core functions of DS9, e.g., display cross-cuts of data, +iso-intensity contours, and WCS grids. However, new programs can be +added to DS9 by creating a set-up file which can be loaded into DS9 +to reconfigure the Analysis menu. + +) 7 48 PR(The basic format of the analysis set-up file is: +)0 P( # + # Analysis command descriptions: + # menu label/description + # file templates for this command + # "menu" \201add to menu\202 |"bind" \201bind to key\202 + # analysis command line)RP( + +For example, the funcnts program can be specified in this way: +) 4 69 PR( Funcnts \201counts in source/bkgd regions; options: none\202 + * + menu + funcnts $filename $regions\201source,,\202 $regions\201background,,\202 | $text)RP( +As shown above, DS9 supports a macro facility to provide information +as well as task support to command lines. For example, the $regions +macro is expanded by DS9 to provide the current source and/or +background region to the analysis command. The $text macro is expanded +to generate a text window display. It also is possible to query for +parameters using a $param macro, plot data using a $plot macro, +etc. See the DS9 documentation for further details. + +)0 P(A set-up file called )0 2 A(funtools.ds9)EA( will +load some useful Funtools applications \201counts in regions, radial +profile, X-ray light curve and energy spectrum, 1D histogram\202 into the DS9 +Analysis menu \201version 2.1 and above\202. The file resides in the bin +directory where Funtools programs are installed. It can be manually +loaded into DS9 from the )BD(Load Analysis Commands ...)ES( option of +the )BD(Analysis)ES( menu. Alternatively, you can tell DS9 to load +this file automatically at start-up time by adding the pathname to the +)BD(Edit)ES(->)BD(Preferences)ES(->)BD(Analysis Menu)ES(->)BD(Analysis +File)ES( menu option. \201NB: make sure you select +)BD(Edit)ES(->)BD(Preferences)ES(->)BD(Save Preferences)ES( after setting +the pathname.\202 + +)0 P(The tasks in this setup file generally process the original disk-based +FITS file. Funcnts-based results \201radial profile, counts in regions\202 +are presented in WCS units, if present in the FITS header. For +situations where a disk file is not available \201e.g., image data +generated and sent to DS9's 'fits' XPA access point\202, versions of the +radial profile and counts in regions tasks also are also offered +utilizing DS9's internal image data. Results are presented in pixels. +Aside from the units, the results should be identical to the file-based +results. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 24 H(Last)WB 88 Sn( updated: November 16, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (library.html) D +/Ti (Funtools Programming) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 18 Sn( + + +)0 2 25 H(FunLib:)WB 91 Sn()WB 89 Sn( the Funtools Programming Interface)EA()EH( + + +)0 2 26 H(Summary)WB 92 Sn()EH( +A description of the Funtools library. + + +)0 2 27 H(Introduction)WB 93 Sn()WB 90 Sn( to the Funtools Programming Interface)EH()EA( +)0 P(To create a Funtools application, you need to include +the funtools.h definitions file in your code: +) 1 23 PR( #include <funtools.h>)RP( + +You then call Funtools subroutines and functions to access Funtools data. +The most important routines are: +)UL( +)0 P()-1 LI( )0 25 1 A(FunOpen)25 0 TN TL()Ec /AF f D(: open a Funtools file +)-1 LI()0 35 1 A(FunInfoGet)35 0 TN TL()Ec /AF f D(: get info about an image or table +)-1 LI()0 26 1 A(FunImageGet)26 0 TN TL()Ec /AF f D(: retrieve image data +)-1 LI()0 28 1 A(FunImageRowGet)28 0 TN TL()Ec /AF f D(: retrieve image data by row +)-1 LI()0 27 1 A(FunImagePut)27 0 TN TL()Ec /AF f D(: output image data +)-1 LI()0 29 1 A(FunImageRowPut)29 0 TN TL()Ec /AF f D(: output image data by row +)-1 LI()0 32 1 A(FunColumnSelect)32 0 TN TL()Ec /AF f D(: select columns in a table for access +)-1 LI()0 30 1 A(FunTableRowGet)30 0 TN TL()Ec /AF f D(: retrieve rows from a table +)-1 LI()0 31 1 A(FunTableRowPut)31 0 TN TL()Ec /AF f D(: output rows to a table +)-1 LI( )0 40 1 A(FunClose)40 0 TN TL()Ec /AF f D(: close a Funtools file)LU( + +Your program must be linked against the libfuntools.a library, +along with the math library. The following libraries also might be required +on your system: +)UL()-1 LI( -lsocket -lnsl for socket support +)-1 LI( -ldl for dynamic loading)LU( +)0 P(For example, on a Solaris system using gcc, use the following link line: +) 1 53 PR( gcc -o foo foo.c -lfuntools -lsocket -lnsl -ldl -lm)RP( +On a Solaris system using Solaris cc, use the following link line: +) 1 48 PR( gcc -o foo foo.c -lfuntools -lsocket -lnsl -lm)RP( +On a Linux system using gcc, use the following link line: +) 1 38 PR( gcc -o foo foo.c -lfuntools -ldl -lm)RP( +Once configure has built a Makefile on your platform, the required +"extra" libraries \201aside from -lm, which always is required\202 are +specified in that file's EXTRA_LIBS variable. For example, under +Linux you will find: +) 3 26 PR( grep EXTRA_LIBS Makefile + EXTRA_LIBS = -ldl + ...)RP( + +)0 P(The Funtools library contains both the zlib library +\201http://www.gzip.org/zlib/\202 and Doug Mink's WCS library +\201http://tdc-www.harvard.edu/software/wcstools/\202. It is not necessary +to put these libraries on a Funtools link line. Include files +necessary for using these libraries are installed in the Funtools +include directory. + +)0 2 28 H(Funtools)WB 94 Sn()WB 20 Sn( Programming Tutorial)EA()EH( + +The +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( +function is used to open a FITS file, an array, or a raw event file: +) 4 79 PR( /* open the input FITS file for reading */ + ifun = FunOpen\201iname, "r", NULL\202; + /* open the output FITS file for writing, and connect it to the input file */ + ofun = FunOpen\201iname, "w", ifun\202;)RP( +A new output file can inherit header parameters automatically from +existing input file by passing the input Funtools handle as the last +argument to the new file's +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( +call as shown above. + +)0 P(For image data, you then can call +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D( +to read an image into memory. +) 3 61 PR( float buf=NULL; + /* extract and bin the data section into an image buffer */ + buf = FunImageGet\201fun, NULL, "bitpix=-32"\202;)RP( +If the \201second\202 buf argument to this call is NULL, buffer space is allocated +automatically. The \201third\202 plist argument can be used to specify the +return data type of the array. If NULL is specified, the data type of +the input file is used. + +)0 P(To process an image buffer, you would generally make a call to +)0 35 1 A(FunInfoGet\201\202)35 0 TN TL()Ec /AF f D( to determine the +dimensions of the image \201which may have been changed from the original +file dimensions due to )0 49 1 A(Funtools image +sectioning)49 0 TN TL()Ec /AF f D( on the command line\202. In a FITS image, the index along +the dim1 axis varies most rapidly, followed by the dim2 axis, etc. +Thus, to access each pixel in an 2D image, use a double loop such as:)RP( + buf = FunImageGet\201fun, NULL, "bitpix=-32"\202; + FunInfoGet\201fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0\202; + for\201i=1; i<=dim2; i++\202{ + for\201j=1; j<=dim1; j++\202{ + ... process buf[\201\201i-1\202*dim1\202+\201j-1\202] ... + } + })RP( +or: +) 5 65 PR( buf = FunImageGet\201fun, NULL, "bitpix=-32"\202; + FunInfoGet\201fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0\202; + for\201i=0; i<\201dim1*dim2\202; i++\202{ + ... process buf[i] ... + })RP( +Finally, you can write the resulting image to disk using +)0 27 1 A(FunImagePut\201\202)27 0 TN TL()Ec /AF f D(: +) 1 48 PR( FunImagePut\201fun2, buf, dim1, dim2, -32, NULL\202;)RP( +Note that Funtools automatically takes care of book-keeping tasks such as +reading and writing FITS headers \201although you can, of course, write +your own header or add your own parameters to a header\202. + +)0 P(For binary tables and raw event files, a call to +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( +will be followed by a call to the +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +routine to select columns to be read from the input file and/or +written to the output file: + +) 8 69 PR( typedef struct evstruct{ + double time; + int time2; + } *Ev, EvRec; + FunColumnSelect\201fun, sizeof\201EvRec\202, NULL, + "time", "D", "rw", FUN_OFFSET\201Ev, time\202, + "time2", "J", "w", FUN_OFFSET\201Ev, time2\202, + NULL\202;)RP( +Columns whose \201third\202 mode argument contains an "r" are "readable", +i.e., columns will be read from the input file and converted into the +data type specified in the call's second argument. These columns +values then are stored in the specified offset of the user record +structure. Columns whose mode argument contains a "w" are +"writable", i.e., these values will be written to the output file. +The +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +routine also offers the option of automatically merging user +columns with the original input columns when writing the output +rows. + +)0 P(Once a set of columns has been specified, you can retrieve rows using +)0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D(, +and write the rows using +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D(: +) 17 70 PR( Ev ebuf, ev; + /* get rows -- let routine allocate the array */ + while\201 \201ebuf = \201Ev\202FunTableRowGet\201fun, NULL, MAXROW, NULL, &got\202\202 \202{ + /* process all rows */ + for\201i=0; i<got; i++\202{ + /* point to the i'th row */ + ev = ebuf+i; + /* time2 is generated here */ + ev->time2 = \201int\202\201ev->time+.5\202; + /* change the input time as well */ + ev->time = -\201ev->time/10.0\202; + } + /* write out this batch of rows with the new column */ + FunTableRowPut\201fun2, \201char *\202ebuf, got, 0, NULL\202; + /* free row data */ + if\201 ebuf \202 free\201ebuf\202; + })RP( +The input rows are retrieved into an array of user structs, which +can be accessed serially as shown above. Once again, Funtools +automatically takes care of book-keeping tasks such as reading and writing +FITS headers \201although you can, of course, write your own header or +add your own parameters to a header\202. + +)0 P(When all processing is done, you can call +)0 40 1 A(FunClose\201\202)40 0 TN TL()Ec /AF f D( +to close the file\201s\202: +) 2 17 PR( FunClose\201fun2\202; + FunClose\201fun\202;)RP( + +)0 P(These are the basics of processing FITS files \201and arrays or raw event +data\202 using Funtools. The routines in these examples are described in +more detail below, along with a few other routines that support +parameter access, data flushing, etc. + +)0 2 29 H(Compiling)WB 95 Sn()WB 22 Sn( and Linking)EA()EH( +)0 P(To create a Funtools application, a software developer will include +the funtools.h definitions file in Funtools code: +) 1 23 PR( #include <funtools.h>)RP( +The program is linked against the libfuntools.a library, along with the +math library \201and the dynamic load library, if the latter is available +on your system\202: +) 1 38 PR( gcc -o foo foo.c -lfuntools -ldl -lm)RP( +)0 P(If gcc is used, Funtools filtering can be performed using dynamically +loaded shared objects that are built at run-time. Otherwise, filtering +is performed using a slave process. +)0 P(Funtools has been built on the following systems: +)UL()-1 LI( Sun/Solaris 5.X +)-1 LI( Linux/RedHat Linux 5.X,6.X,7.X +)-1 LI( Dec Alpha/OSF1 V4.X +)-1 LI( WindowsNT/Cygwin 1.0 +)-1 LI( SGI/IRIX64 6.5)LU( + +)0 2 30 H(A)WB 96 Sn()WB 21 Sn( Short Digression on Subroutine Order)EA()EH( +)0 P(There is a natural order for all I/O access libraries. You would not +think of reading a file without first opening it, or writing a file +after closing it. A large part of the experiment in funtools is to use +the idea of "natural order" as a means of making programming +easier. We do this by maintaining the state of processing for a given +funtools file, so that we can do things like write headers and flush +extension padding at the right time, without you having to do it. + +)0 P(For example, if you open a new funtools file for writing using +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(, +then generate an array of image data and call +)0 27 1 A(FunImagePut\201\202)27 0 TN TL()Ec /AF f D(, +funtools knows to write the image header automatically. +There is no need to think about writing a standard header. +Of course, you can add parameters to the file first by +calling one of the +)0 38 1 A(FunParamPut\201\202)38 0 TN TL()Ec /AF f D( +routines, and these parameters will automatically be added +to the header when it is written out. There still is no +need to write the header explicitly. + +)0 P(Maintaining state in this way means that there are certain rules of +order which should be maintained in any funtools program. In particular, +we strongly recommend the following ordering rules be adhered to: + +)UL()-1 LI( When specifying that input extensions be copied to an output file +via a reference handle, open the output file )BD(before)ES( reading the +input file. \201Otherwise the initial copy will not occur\202. + +)-1 LI( Always write parameters to an output file using one of the +)0 38 1 A(FunParamPut\201\202)38 0 TN TL()Ec /AF f D( calls +)BD(before)ES( writing any data. \201This is a good idea for all FITS +libraries, to avoid having to recopy data is the FITS header needs +to be extended by adding a single parameter.\202 + +)-1 LI( If you retrieve an image, and need to know the data +type, use the FUN_SECT_BITPIX option of +)0 35 1 A(FunInfoGet\201\202)35 0 TN TL()Ec /AF f D(, +)BD(after)ES( calling +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D(, since +it is possible to change the value of BITPIX from the latter. + +)-1 LI( When specifying that input extensions be copied to an output file +via a reference handle, close the output file )BD(before)ES( closing +input file, or else use +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( +explicitly on the output file +)BD(before)ES( closing the input file. \201Otherwise the final copy will +not occur\202.)LU( + +)0 P(We believe that these are the natural rules that are implied in most +FITS programming tasks. However, we recognize that making explicit use +of "natural order" to decide what automatic action to take on behalf +of the programmer is experimental. Therefore, if you find that your +needs are not compatible with our preferred order, please let us know +-- it will be most illuminating for us as we evaluate this experiment. + +)0 2 31 H(Funtools)WB 97 Sn()WB 41 Sn( Programming Examples)EA()EH( +)0 P(The following complete coding examples are provided to illustrate the +simplicity of Funtools applications. They can be found in the funtest +subdirectory of the Funtools distribution. In many cases, you should +be able to modify one of these programs to generate your own Funtools +program: +)UL()-1 LI()0 2 A(evread.c)EA(: read and write binary tables +)-1 LI()0 2 A(evcols.c)EA(: add column and rows to binary tables +)-1 LI()0 2 A(evmerge.c)EA(: merge new columns with existing columns +)-1 LI()0 2 A(evnext.c)EA(: manipulate raw data pointers +)-1 LI()0 2 A(imblank.c)EA(: blank out image values below a threshold +)-1 LI()0 2 A(asc2fits.c)EA(: convert a specific ASCII table to FITS binary table)LU( + +)0 2 32 H(The)WB 98 Sn()WB 24 Sn( Funtools Programming Reference Manual)EA()EH( +)0 P() 45 116 PR(#include <funtools.h> + +Fun )0 25 1 A(FunOpen\201char *name, char *mode, Fun ref\202)25 0 TN TL()Ec /AF f D( + +void *)0 26 1 A(FunImageGet\201Fun fun, void *buf, char *plist\202)26 0 TN TL()Ec /AF f D( + +int )0 27 1 A(FunImagePut\201Fun fun, void *buf, int dim1, int dim2, int bitpix, char *plist\202)27 0 TN TL()Ec /AF f D( + +void * )0 28 1 A(FunImageRowGet\201Fun fun, void *buf, int rstart, int rstop, char *plist\202)28 0 TN TL()Ec /AF f D( + +void * )0 29 1 A(FunImageRowPut\201Fun fun, void *buf, int rstart, int rstop, int dim1, int dim2, int bitpix, char *plist\202)29 0 TN TL()Ec /AF f D( + +int )0 32 1 A(FunColumnSelect\201Fun fun, int size, char *plist, ...\202)32 0 TN TL()Ec /AF f D( + +void )0 33 1 A(FunColumnActivate\201Fun fun, char *s, char *plist\202)33 0 TN TL()Ec /AF f D( + +int )0 34 1 A(FunColumnLookup\201Fun fun, char *s, int which, char **name, int *type, int *mode, int *offset, int *n, int *width\202)34 0 TN TL()Ec /AF f D( + +void *)0 30 1 A(FunTableRowGet\201Fun fun, void *rows, int maxrow, char *plist, int *nrow\202)30 0 TN TL()Ec /AF f D( + +int )0 31 1 A(FunTableRowPut\201Fun fun, void *rows, int nev, int idx, char *plist\202)31 0 TN TL()Ec /AF f D( + +int )0 37 1 A(FunParamGetb\201Fun fun, char *name, int n, int defval, int *got\202)37 0 TN TL()Ec /AF f D( + +int )0 37 1 A(FunParamGeti\201Fun fun, char *name, int n, int defval, int *got\202)37 0 TN TL()Ec /AF f D( + +double )0 37 1 A(FunParamGetd\201Fun fun, char *name, int n, double defval, int *got\202)37 0 TN TL()Ec /AF f D( + +char *)0 37 1 A(FunParamGets\201Fun fun, char *name, int n, char *defval, int *got\202)37 0 TN TL()Ec /AF f D( + +int )0 38 1 A(FunParamPutb\201Fun fun, char *name, int n, int value, char *comm, int append\202)38 0 TN TL()Ec /AF f D()WR( + +int )0 38 1 A(FunParamPuti\201Fun fun, char *name, int n, int value, char *comm, int append\202)38 0 TN TL()Ec /AF f D( + +int )0 38 1 A(FunParamPutd\201Fun fun, char *name, int n, double value, int prec, char *comm, int append\202)38 0 TN TL()Ec /AF f D( + +int )0 38 1 A(FunParamPuts\201Fun fun, char *name, int n, char *value, char *comm, int append\202)38 0 TN TL()Ec /AF f D( + +int )0 35 1 A(FunInfoGet\201Fun fun, int type, ...\202)35 0 TN TL()Ec /AF f D( + +int )0 36 1 A(FunInfoPut\201Fun fun, int type, ...\202)36 0 TN TL()Ec /AF f D( + +void )0 39 1 A(FunFlush\201Fun fun, char *plist\202)39 0 TN TL()Ec /AF f D( + +void )0 40 1 A(FunClose\201Fun fun\202)40 0 TN TL()Ec /AF f D()RP( + + + + +)0 2 33 H(FunOpen)WB 99 Sn()WB 25 Sn( - open a Funtools data file)EA()EH( + + +)BD() 3 47 PR( #include <funtools.h> + + Fun FunOpen\201char *name, char *mode, Fun ref\202;)RP()ES( + + +)0 P(The )BD(FunOpen\201\202)ES( routine opens a Funtools data file for reading or +appending, or creates a new FITS file for writing. The )BD(name)ES( +argument specifies the name of the Funtools data file to open. You can +use IRAF-style bracket notation to specify +)0 42 1 A(Funtools Files, Extensions, and Filters)42 0 TN TL()Ec /AF f D(. +A separate call should be made each time a different FITS extension is +accessed: +) 7 65 PR( Fun fun; + char *iname; + ... + if\201 !\201fun = FunOpen\201iname, "r", NULL\202\202 \202{ + fprintf\201stderr, "could not FunOpen input file: %s\200n", iname\202; + exit\2011\202; + })RP( +)0 P(If )BD(mode)ES( is "r", the file is opened for reading, and processing +is set up to begin at the specified extension. For reading, +)BD(name)ES( can be )BD(stdin)ES(, in which case the standard input is read. + +)0 P(If )BD(mode)ES( is "w", the file is created if it does not exist, or +opened and truncated for writing if it does exist. Processing starts +at the beginning of the file. The )BD(name)ES( can be )BD(stdout)ES(, +in which case the standard output is readied for processing. + +)0 P(If )BD(mode)ES( is "a", the file is created if it does not exist, or +opened if it does exist. Processing starts at the end of the file. +The )BD(name)ES( can be )BD(stdout)ES(, in which case the standard +output is readied for processing. + +)0 P(When a Funtools file is opened for writing or appending, a previously +opened )0 23 1 A(Funtools reference +handle)23 0 TN TL()Ec /AF f D( can be specified as the third argument. This handle +typically is associated with the input Funtools file that will be used +to generate the data for the output data. When a reference file is +specified in this way, the output file will inherit the \201extension\202 +header parameters from the input file: +) 8 67 PR( Fun fun, fun2; + ... + /* open input file */ + if\201 !\201fun = FunOpen\201argv[1], "r", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen input file: %s\200n", argv[1]\202; + /* open the output FITS image, inheriting params from input */ + if\201 !\201fun2 = FunOpen\201argv[2], "w", fun\202\202 \202 + gerror\201stderr, "could not FunOpen output file: %s\200n", argv[2]\202;)RP( +Thus, in the above example, the output FITS binary table file will +inherit all of the parameters associated with the input binary table +extension. +)0 P(A file opened for writing with a +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( also +inherits the selected columns \201i.e. those columns chosen for +processing using the +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( routine\202 +from the reference file as its default columns. This makes it easy to +open an output file in such a way that the columns written to the +output file are the same as the columns read in the input file. Of +course, column selection can easily be tailored using the +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( routine. +In particular, it is easy to merge user-defined columns with the input +columns to generate a new file. See the +)0 2 A(evmerge)EA( for a complete example. + +)0 P(In addition, when a +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( +is supplied in a )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( call, +it is possible also to specify that all other extensions from the +reference file \201other than the input extension being processed\202 should +be copied from the reference file to the output file. This is useful, +for example, in a case where you are processing a FITS binary table +or image and you want to copy all of the other extensions to +the output file as well. Copy of other extensions is controlled by +adding a "C" or "c" to the mode string of the +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( call )BD(of the input +reference file)ES(. If "C" is specified, then other extensions are +)BD(always)ES( copied \201i.e., copy is forced by the application\202. If +"c" is used, then other extensions are copied if the user requests +copying by adding a plus sign "+" to the extension name in the bracket +specification. For example, the )BD(funtable)ES( program utilizes +"c" mode, giving users the option of copying all other extensions: +) 6 67 PR( /* open input file -- allow user copy of other extensions */ + if\201 !\201fun = FunOpen\201argv[1], "rc", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen input file: %s\200n", argv[1]\202; + /* open the output FITS image, inheriting params from input */ + if\201 !\201fun2 = FunOpen\201argv[2], "w", fun\202\202 \202 + gerror\201stderr, "could not FunOpen output file: %s\200n", argv[2]\202;)RP( + +Thus, )BD(funtable)ES( supports either of these command lines: +) 4 60 PR( # copy only the EVENTS extension + csh> funtable "test.ev[EVENTS,circle\201512,512,10\202]" foo.ev + # copy ALL extensions + csh> funtable "test.ev[EVENTS+,circle\201512,512,10\202]" foo.ev)RP( + +)0 P(Use of a )0 23 1 A(Funtools reference +handle)23 0 TN TL()Ec /AF f D( implies that the input file is opened before the output +file. However, it is important to note that if copy mode \201"c" or "C"\202 +is specified for the input file, the actual input file open is delayed +until just after the output file is opened, since the copy of prior +extensions to the output file takes place while Funtools is seeking to +the specified input extension. This implies that the output file +should be opened before any I/O is done on the input file or else the +copy will fail. Note also that the copy of subsequent extension will +be handled automatically by +)0 40 1 A(FunClose\201\202)40 0 TN TL()Ec /AF f D( +if the output file is +closed before the input file. Alternatively, it can be done explicitly +by )0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D(, but again, this +assumes that the input file still is open. + +)0 P(Upon success )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( returns a +Fun handle that is used in subsequent Funtools calls. On error, NULL +is returned. + + + + +)0 2 34 H(FunImageGet)WB 100 Sn()WB 26 Sn( - get an image or image section)EA()EH( + + +)BD() 3 52 PR( #include <funtools.h> + + void *FunImageGet\201Fun fun, void *buf, char *plist\202)RP()ES( + + +)0 P(The )BD(FunImageGet\201\202)ES( routine returns an binned image array of the +specified section of a Funtools data file. If the input data are +already of type image, the array is generated by extracting the +specified image section and then binning it according to the specified +bin factor. If the input data are contained in a binary table or raw +event file, the rows are binned on the columns specified by the +)BD(bincols=)ES( keyword \201using appropriate default columns as +necessary\202, after which the image section and bin factors are +applied. In both cases, the data is automatically converted from FITS +to native format, if necessary. +)0 P(The first argument is the Funtools handle returned by +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(. The second )BD(buf)ES( +argument is a pointer to a data buffer to fill. If NULL is specified, +FunImageGet will allocate a buffer of the appropriate size. Generally +speaking, you always want Funtools to allocate the buffer because +the image dimensions will be determined by +)0 49 1 A(Funtools image sectioning)49 0 TN TL()Ec /AF f D( +on the command line. +)0 P(The third )BD(plist)ES( \201i.e., parameter list\202 argument is a string +containing one or more comma-delimited )BD(keyword=value)ES( +parameters. It can be used to specify the return data type using the +)BD(bitpix=)ES( keyword. If no such keyword is specified in the plist +string, the data type of the returned image is the same as the data type +of the original input file, or is of type int for FITS binary tables. + +)0 P(If the )BD(bitpix=)ES( keyword is supplied in the plist string, the data +type of the returned image will be one of the supported FITS image +data types: +)UL()-1 LI( 8 unsigned char +)-1 LI( 16 short +)-1 LI( 32 int +)-1 LI( -32 float +)-1 LI( -64 double)LU( +For example: +) 4 57 PR( void *buf; + /* extract data section into an image buffer */ + if\201 !\201buf = FunImageGet\201fun, NULL, NULL\202\202 \202 + gerror\201stderr, "could not FunImageGet: %s\200n", iname\202;)RP( +will allocate buf and retrieve the image in the file data format. In +this case, you will have to determine the data type \201using the +FUN_SECT_BITPIX value in the +)0 35 1 A(FunInfoGet\201\202)35 0 TN TL()Ec /AF f D( +routine\202 +and then use a switch statement to process each data type: +) 17 65 PR( int bitpix; + void *buf; + unsigned char *cbuf; + short *sbuf; + int *ibuf; + ... + buf = FunImageGet\201fun, NULL, NULL\202; + FunInfoGet\201fun, FUN_SECT_BITPIX, &bitpix, 0\202; + /* set appropriate data type buffer to point to image buffer */ + switch\201bitpix\202{ + case 8: + cbuf = \201unsigned char *\202buf; break; + case 16: + sbuf = \201short *\202buf; break; + case 32: + ibuf = \201int *\202buf; break; + ...)RP( +See the +)0 2 A(imblank example code)EA( +for more details on how to process an image when the data type is not +specified beforehand. + +)0 P(It often is easier to specify the data type directly: +) 4 57 PR( double *buf; + /* extract data section into a double image buffer */ + if\201 !\201buf = FunImageGet\201fun, NULL, "bitpix=-64"\202\202 \202 + gerror\201stderr, "could not FunImageGet: %s\200n", iname\202;)RP( +will extract the image while converting to type double. + +)0 P(On success, a pointer to the image buffer is returned. \201This will be +the same as the second argument, if NULL is not passed to the latter.\202 +On error, NULL is returned. + +)0 P(In summary, to retrieve image or row data into a binned image, you simply +call FunOpen\201\202 followed by +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D(. Generally, you +then will want to call +)0 35 1 A(FunInfoGet\201\202)35 0 TN TL()Ec /AF f D( +to retrieve the +axis dimensions \201and data type\202 of the section you are processing +\201so as to take account of sectioning and blocking of the original data\202: +) 20 73 PR( double *buf; + int i, j; + int dim1, dim2; + ... other declarations, etc. + + /* open the input FITS file */ + if\201 !\201fun = FunOpen\201argv[1], "rc", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen input file: %s\200n", argv[1]\202; + + /* extract and bin the data section into a double float image buffer */ + if\201 !\201buf = FunImageGet\201fun, NULL, "bitpix=-64"\202\202 \202 + gerror\201stderr, "could not FunImageGet: %s\200n", argv[1]\202; + + /* get dimension information from funtools structure */ + FunInfoGet\201fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0\202; + + /* loop through pixels and reset values below limit to value */ + for\201i=0; i<dim1*dim2; i++\202{ + if\201 buf[i] <= blimit \202 buf[i] = bvalue; + })RP( + +)0 P(Another useful plist string value is "mask=all", which returns an +image populated with regions id values. Image pixels within a region +will contain the associated region id \201region values start at 1\202, and +otherwise will contain a 0 value. Thus, the returned image is a +region mask which can be used to process the image data \201which +presumably is retrieved by a separate call to FunImageGet\202 pixel by +pixel. + +)0 P(If a FITS binary table or a non-FITS raw event file is being binned +into an image, it is necessary to specify the two columns that will be +used in the 2D binning. This usually is done on the command line +using the )BD(bincols=\201x,y\202)ES( keyword: +) 1 46 PR( funcnts "foo.ev[EVENTS,bincols=\201detx,dety\202]")RP( + +)0 P(The full form of the )BD(bincols=)ES( specifier is: +) 1 77 PR( bincols=\201[xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]\202)RP( +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: +) 2 56 PR( dim = \201tlmax - tlmin\202/binsiz \201floating point data\202 + dim = \201tlmax - tlmin\202/binsiz + 1 \201integer data\202)RP( +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters \201respectively\202 are present in the +FITS binary table header for the column in question. Note that if +only one parameter is specified, it is assumed to be tlmax, and tlmin +defaults to 1. If two parameters are specified, they are assumed to be +tlmin and tlmax. + +)0 P(If )BD(bincols)ES( is not specified on the command line, Funtools tries +to use appropriate defaults: it looks for the environment variable +FITS_BINCOLS \201or FITS_BINKEY\202. Then it looks for the Chandra +parameters CPREF \201or PREFX\202 in the FITS binary table header. Failing +this, it looks for columns named "X" and "Y" and if these are not +found, it looks for columns containing the characters "X" and "Y". +)0 P(See )0 50 1 A(Binning FITS Binary Tables and +Non-FITS Event Files)50 0 TN TL()Ec /AF f D( for more information. + + + + +)0 2 35 H(FunImagePut)WB 101 Sn()WB 27 Sn( - put an image to a Funtools file)EA()EH( + +)BD() 4 69 PR( #include <funtools.h> + + int FunImagePut\201Fun fun, void *buf, int dim1, int dim2, int bitpix, + char *plist\202)RP()ES( + + +The )BD(FunImagePut\201\202)ES( routine outputs an image array to a FITS +file. The image is written either as a primary header/data unit or as +an image extension, depending on whether other data have already been +written to the file. That is, if the current file position is at the +beginning of the file, a primary HDU is written. Otherwise, an +image extension is written. + +)0 P(The first argument is the Funtools handle returned by +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(. The second )BD(buf)ES( +argument is a pointer to a data buffer to write. The )BD(dim1)ES(and +)BD(dim2)ES( arguments that follow specify the dimensions of the image, +where dim1 corresponds to naxis1 and dim2 corresponds to naxis2. The +)BD(bitpix)ES( argument specifies the data type of the image and can +have the following FITS-standard values: +)UL()-1 LI( 8 unsigned char +)-1 LI( 16 short +)-1 LI( 32 int +)-1 LI( -32 float +)-1 LI( -64 double)LU( + +)0 P(When )0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D( is first +called for a given image, Funtools checks to see if the primary header +has already been written \201by having previously written an image or a +binary table.\202 If not, this image is written to the primary HDU. +Otherwise, it is written to an image extension. +)0 P(Thus, a simple program to generate a FITS image might look like this: +) 16 67 PR( int i; + int dim1=512, dim2=512; + double *dbuf; + Fun fun; + dbuf = malloc\201dim1*dim2*sizeof\201double\202\202; + /* open the output FITS image, preparing to copy input params */ + if\201 !\201fun = FunOpen\201argv[1], "w", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen output file: %s\200n", argv[1]\202; + for\201i=0; i<\201dim1*dim2\202; i++\202{ + ... fill dbuf ... + } + /* put the image \201header will be generated automatically */ + if\201 !FunImagePut\201fun, buf, dim1, dim2, -64, NULL\202 \202 + gerror\201stderr, "could not FunImagePut: %s\200n", argv[1]\202; + FunClose\201fun\202; + free\201dbuf\202;)RP( + +)0 P(In addition, if a +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( +was specified when this table was opened, the +parameters from this +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( +are merged into the new image +header. Furthermore, if a reference image was specified during +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(, the values of +)BD(dim1)ES(, )BD(dim2)ES(, and )BD(bitpix)ES( in the calling sequence +can all be set to 0. In this case, default values are taken from the +reference image section. This is useful if you are reading an image +section in its native data format, processing it, and then writing +that section to a new FITS file. See the +)0 2 A(imblank example code)EA(. + +)0 P(The data are assumed to be in the native machine format and will +automatically be swapped to FITS big-endian format if necessary. This +behavior can be over-ridden with the )BD(convert=[true|false])ES( +keyword in the )BD(plist)ES( param list string. + +)0 P(When you are finished writing the image, you should call +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( to write out the FITS +image padding. However, this is not necessary if you subsequently call +FunClose\201\202 without doing any other I/O to the FITS file. + + + + +)0 2 36 H(FunImageRowGet)WB 102 Sn()WB 28 Sn( - get row\201s\202 of an image)EA()EH( + + +)BD() 4 65 PR( #include <funtools.h> + + void *FunImageRowGet\201Fun fun, void *buf, int rstart, int rstop, + char *plist\202)RP()ES( + + +)0 P(The )BD(FunImageRowGet\201\202)ES( routine returns one or more image rows +from the specified section of a Funtools data file. If the input data +are of type image, the array is generated by extracting the specified +image rows and then binning them according to the specified bin +factor. If the input data are contained in a binary table or raw +event file, the rows are binned on the columns specified by the +)BD(bincols=)ES( keyword \201using appropriate default columns as needed\202, +after which the image section and bin factors are applied. + +)0 P(The first argument is the Funtools handle returned by +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(. The second )BD(buf)ES( +argument is a pointer to a data buffer to fill. If NULL is specified, +FunImageGet\201\202 will allocate a buffer of the appropriate size. + +)0 P(The third and fourth arguments specify the first and last row to +retrieve. Rows are counted starting from 1, up to the value of +FUN_YMAX\201fun\202. The final )BD(plist)ES( \201i.e., parameter list\202 argument +is a string containing one or more comma-delimited +)BD(keyword=value)ES( parameters. It can be used to specify the return +data type using the )BD(bitpix=)ES( keyword. If no such keyword is +specified in the plist string, the data type of the image is the same +as the data type of the original input file, or is of type int for +FITS binary tables. + +)0 P(If the )BD(bitpix=)ES(value is supplied in the plist string, the data +type of the returned image will be one of the supported FITS image +data types: +)UL()-1 LI( 8 unsigned char +)-1 LI( 16 short +)-1 LI( 32 int +)-1 LI( -32 float +)-1 LI( -64 double)LU( + +)0 P(For example: +) 17 65 PR( double *drow; + Fun fun; + ... open files ... + /* get section dimensions */ + FunInfoGet\201fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0\202; + /* allocate one line's worth */ + drow = malloc\201dim1*sizeof\201double\202\202; + /* retrieve and process each input row \201starting at 1\202 */ + for\201i=1; i <= dim2; i++\202{ + if\201 !FunImageRowGet\201fun, drow, i, i, "bitpix=-64"\202 \202 + gerror\201stderr, "can't FunImageRowGet: %d %s\200n", i, iname\202; + /* reverse the line */ + for\201j=1; j<=dim1; j++\202{ + ... process drow[j-1] ... + } + } + ...)RP( + +)0 P(On success, a pointer to the image buffer is returned. \201This will be +the same as the second argument, if NULL is not passed to the latter.\202 +On error, NULL is returned. Note that the considerations described +above for specifying binning columns in +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D( also apply to +)BD(FunImageRowGet\201\202)ES(. + + + + +)0 2 37 H(FunImageRowPut)WB 103 Sn()WB 29 Sn( - put row\201s\202 of an image)EA()EH( + + +)BD() 4 67 PR( #include <funtools.h> + + void *FunImageRowPut\201Fun fun, void *buf, int rstart, int rstop, + int dim1, int dim2, int bitpix, char *plist\202)RP()ES( + + +)0 P(The )BD(FunImageRowPut\201\202)ES( routine writes one or more image rows to +the specified FITS image file. The first argument is the Funtools +handle returned by )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(. +The second )BD(buf)ES( argument is a pointer to the row data buffer, +while the third and fourth arguments specify the starting and ending +rows to write. Valid rows values range from 1 to dim2, i.e., row is +one-valued. + +)0 P(The )BD(dim1)ES(and )BD(dim2)ES( arguments that follow specify the +dimensions, where dim1 corresponds to naxis1 and dim2 corresponds to +naxis2. The )BD(bitpix)ES( argument data type of the image and can +have the following FITS-standard values: +)UL()-1 LI( 8 unsigned char +)-1 LI( 16 short +)-1 LI( 32 int +)-1 LI( -32 float +)-1 LI( -64 double)LU( + +For example: +) 16 65 PR( double *drow; + Fun fun, fun2; + ... open files ... + /* get section dimensions */ + FunInfoGet\201fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0\202; + /* allocate one line's worth */ + drow = malloc\201dim1*sizeof\201double\202\202; + /* retrieve and process each input row \201starting at 1\202 */ + for\201i=1; i <= dim2; i++\202{ + if\201 !FunImageRowGet\201fun, drow, i, i, "bitpix=-64"\202 \202 + gerror\201stderr, "can't FunImageRowGet: %d %s\200n", i, iname\202; + ... process drow ... + if\201 !FunImageRowPut\201fun2, drow, i, i, 64, NULL\202 \202 + gerror\201stderr, "can't FunImageRowPut: %d %s\200n", i, oname\202; + } + ...)RP( + +)0 P(The data are assumed to be in the native machine format and will +automatically be swapped to big-endian FITS format if necessary. This +behavior can be over-ridden with the )BD(convert=[true|false])ES( +keyword in the )BD(plist)ES( param list string. + +)0 P(When you are finished writing the image, you should call +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( to write out the FITS +image padding. However, this is not necessary if you subsequently call +FunClose\201\202 without doing any other I/O to the FITS file. + + + + +)0 2 38 H(FunColumnSelect)WB 104 Sn()WB 32 Sn( - select Funtools columns)EA()EH( + +)BD() 11 73 PR( #include <funtools.h> + + int FunColumnSelect\201Fun fun, int size, char *plist, + char *name1, char *type1, char *mode1, int offset1, + char *name2, char *type2, char *mode2, int offset2, + ..., + NULL\202 + + int FunColumnSelectArr\201Fun fun, int size, char *plist, + char **names, char **types, char **modes, + int *offsets, int nargs\202;)RP()ES( + + +The )BD(FunColumnSelect\201\202)ES( routine is used to select the columns +from a Funtools binary table extension or raw event file for +processing. This routine allows you to specify how columns in a file +are to be read into a user record structure or written from a user +record structure to an output FITS file. + +)0 P(The first argument is the Fun handle associated with this set of +columns. The second argument specifies the size of the user record +structure into which columns will be read. Typically, the sizeof\201\202 +macro is used to specify the size of a record structure. The third +argument allows you to specify keyword directives for the selection +and is described in more detail below. + +)0 P(Following the first three required arguments is a variable length list of +column specifications. Each column specification will consist of four +arguments: +)UL()-1 LI( )BD(name)ES(: the name of the column + +)-1 LI( )BD(type)ES(: the data type of the column as it will be stored in +the user record struct \201not the data type of the input file\202. The +following basic data types are recognized: +)UL()-1 LI(A: ASCII characters +)-1 LI(B: unsigned 8-bit char +)-1 LI(I: signed 16-bit int +)-1 LI(U: unsigned 16-bit int \201not standard FITS\202 +)-1 LI(J: signed 32-bit int +)-1 LI(V: unsigned 32-bit int \201not standard FITS\202 +)-1 LI(E: 32-bit float +)-1 LI(D: 64-bit float)LU( +The syntax used is similar to that which defines the TFORM parameter +in FITS binary tables. That is, a numeric repeat value can precede +the type character, so that "10I" means a vector of 10 short ints, "E" +means a single precision float, etc. Note that the column value from +the input file will be converted to the specified data type as the +data is read by +)0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D(. + +)0 P([ A short digression regarding bit-fields: Special attention is +required when reading or writing the FITS bit-field type +\201"X"\202. Bit-fields almost always have a numeric repeat character +preceding the 'X' specification. Usually this value is a multiple of 8 +so that bit-fields fit into an integral number of bytes. For all +cases, the byte size of the bit-field B is \201N+7\202/8, where N is the +numeric repeat character. + +)0 P(A bit-field is most easily declared in the user struct as an array of +type char of size B as defined above. In this case, bytes are simply +moved from the file to the user space. If, instead, a short or int +scalar or array is used, then the algorithm for reading the bit-field +into the user space depends on the size of the data type used along +with the value of the repeat character. That is, if the user data +size is equal to the byte size of the bit-field, then the data is +simply moved \201possibly with endian-based byte-swapping\202 from one to +the other. If, on the other hand, the data storage is larger than the +bit-field size, then a data type cast conversion is performed to move +parts of the bit-field into elements of the array. Examples will help +make this clear: + +)UL()-1 LI( If the file contains a 16X bit-field and user space specifies a 2B +char array[2], then the bit-field is moved directly into the char array. + +)-1 LI( If the file contains a 16X bit-field and user space specifies a 1I +scalar short int, then the bit-field is moved directly into the short int. + +)-1 LI( If the file contains a 16X bit-field and user space specifies a 1J +scalar int, then the bit-field is type-cast to unsigned int before +being moved \201use of unsigned avoids possible sign extension\202. + +)-1 LI( If the file contains a 16X bit-field and user space specifies a 2J +int array[2], then the bit-field is handled as 2 chars, each of which +are type-cast to unsigned int before being moved \201use of unsigned +avoids possible sign extension\202. + +)-1 LI( If the file contains a 16X bit-field and user space specifies a 1B +char, then the bit-field is treated as a char, i.e., truncation will +occur. + +)-1 LI( If the file contains a 16X bit-field and user space specifies a 4J +int array[4], then the results are undetermined. +)LU( +For all user data types larger than char, the bit-field is byte-swapped +as necessary to convert to native format, so that bits in the +resulting data in user space can be tested, masked, etc. in the same +way regardless of platform.] + +)0 P(In addition to setting data type and size, the )BD(type)ES( +specification allows a few ancillary parameters to be set, using the +full syntax for )BD(type)ES(: +) 1 53 PR( [@][n]<type>[[['B']poff]][:[tlmin[:tlmax[:binsiz]]]])RP( + +)0 P(The special character "@" can be prepended to this specification to +indicated that the data element is a pointer in the user record, +rather than an array stored within the record. + +)0 P(The [n] value is an integer that specifies the +number of elements that are in this column \201default is 1\202. TLMIN, +TLMAX, and BINSIZ values also can be specified for this column after +the type, separated by colons. If only one such number is specified, +it is assumed to be TLMAX, and TLMIN and BINSIZ are set to 1. + +)0 P(The [poff] value can be used to specify the offset into an +array. By default, this offset value is set to zero and the data +specified starts at the beginning of the array. The offset usually +is specified in terms of the data type of the column. Thus an offset +specification of [5] means a 20-byte offset if the data type is a +32-bit integer, and a 40-byte offset for a double. If you want to +specify a byte offset instead of an offset tied to the column data type, +precede the offset value with 'B', e.g. [B6] means a 6-bye offset, +regardless of the column data type. + +The [poff] is especially useful in conjunction with the pointer @ +specification, since it allows the data element to anywhere stored +anywhere in the allocated array. For example, a specification such as +"@I[2]" specifies the third \201i.e., starting from 0\202 element in the +array pointed to by the pointer value. A value of "@2I[4]" specifies +the fifth and sixth values in the array. For example, consider the +following specification: ) 12 67 PR( typedef struct EvStruct{ + short x[4], *atp; + } *Event, EventRec; + /* set up the \201hardwired\202 columns */ + FunColumnSelect\201 fun, sizeof\201EventRec\202, NULL, + "2i", "2I ", "w", FUN_OFFSET\201Event, x\202, + "2i2", "2I[2]", "w", FUN_OFFSET\201Event, x\202, + "at2p", "@2I", "w", FUN_OFFSET\201Event, atp\202, + "at2p4", "@2I[4]", "w", FUN_OFFSET\201Event, atp\202, + "atp9", "@I[9]", "w", FUN_OFFSET\201Event, atp\202, + "atb20", "@I[B20]", "w", FUN_OFFSET\201Event, atb\202, + NULL\202;)RP( +Here we have specified the following columns: +)UL()-1 LI( 2i: two short ints in an array which is stored as part the +record +)-1 LI( 2i2: the 3rd and 4th elements of an array which is stored +as part of the record +)-1 LI( an array of at least 10 elements, not stored in the record but +allocated elsewhere, and used by three different columns: +)UL()-1 LI( at2p: 2 short ints which are the first 2 elements of the allocated array +)-1 LI( at2p4: 2 short ints which are the 5th and 6th elements of +the allocated array +)-1 LI( atp9: a short int which is the 10th element of the allocated array)LU( +)-1 LI( atb20: a short int which is at byte offset 20 of another allocated array)LU( +In this way, several columns can be specified, all of which are in a +single array. )BD(NB)ES(: it is the programmer's responsibility to +ensure that specification of a positive value for poff does not point +past the end of valid data. + +)-1 LI( )BD(read/write mode)ES(: "r" means that the column is read from an +input file into user space by +)0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D(, "w" means that +the column is written to an output file. Both can specified at the same +time. + +)-1 LI( )BD(offset)ES(: the offset into the user data to store +this column. Typically, the macro FUN_OFFSET\201recname, colname\202 is used +to define the offset into a record structure.)LU( + +)0 P(When all column arguments have been specified, a final NULL argument +must added to signal the column selection list. + +)0 P(As an alternative to the varargs +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +routine, a non-varargs routine called +)0 32 1 A(FunColumnSelectArr\201\202)32 0 TN TL()Ec /AF f D( +also is available. The first three arguments \201fun, size, plist\202 of this +routine are the same as in +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D(. +Instead of a variable +argument list, however, +)0 32 1 A(FunColumnSelectArr\201\202)32 0 TN TL()Ec /AF f D( +takes 5 additional arguments. The first 4 arrays arguments contain the +names, types, modes, and offsets, respectively, of the columns being +selected. The final argument is the number of columns that are +contained in these arrays. It is the user's responsibility to free +string space allocated in these arrays. + +)0 P(Consider the following example: +) 12 54 PR( typedef struct evstruct{ + int status; + float pi, pha, *phas; + double energy; + } *Ev, EvRec; + + FunColumnSelect\201fun, sizeof\201EvRec\202, NULL, + "status", "J", "r", FUN_OFFSET\201Ev, status\202, + "pi", "E", "r", FUN_OFFSET\201Ev, pi\202, + "pha", "E", "r", FUN_OFFSET\201Ev, pha\202, + "phas", "@9E", "r", FUN_OFFSET\201Ev, phas\202, + NULL\202;)RP( +)0 P(Each time a row is read into the Ev struct, the "status" column is +converted to an int data type \201regardless of its data type in the +file\202 and stored in the status value of the struct. Similarly, "pi" +and "pha", and the phas vector are all stored as floats. Note that the +"@" sign indicates that the "phas" vector is a pointer to a 9 element +array, rather than an array allocated in the struct itself. The row +record can then be processed as required: +) 9 70 PR( /* get rows -- let routine allocate the row array */ + while\201 \201ebuf = \201Ev\202FunTableRowGet\201fun, NULL, MAXROW, NULL, &got\202\202 \202{ + /* process all rows */ + for\201i=0; i<got; i++\202{ + /* point to the i'th row */ + ev = ebuf+i; + ev->pi = \201ev->pi+.5\202; + ev->pha = \201ev->pi-.5\202; + })RP( + +)0 P()0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +can also be called to define "writable" columns in order to generate a FITS +Binary Table, without reference to any input columns. For +example, the following will generate a 4-column FITS binary table when +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D( is used to +write Ev records: + +) 12 56 PR( typedef struct evstruct{ + int status; + float pi, pha + double energy; + } *Ev, EvRec; + + FunColumnSelect\201fun, sizeof\201EvRec\202, NULL, + "status", "J", "w", FUN_OFFSET\201Ev, status\202, + "pi", "E", "w", FUN_OFFSET\201Ev, pi\202, + "pha", "E", "w", FUN_OFFSET\201Ev, pha\202, + "energy", "D", "w", FUN_OFFSET\201Ev, energy\202, + NULL\202;)RP( +All columns are declared to be write-only, so presumably the column +data is being generated or read from some other source. + +)0 P(In addition, +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +can be called to define )BD(both)ES( "readable" and "writable" columns. +In this case, the "read" columns +are associated with an input file, while the "write" columns are +associated with the output file. Of course, columns can be specified as both +"readable" and "writable", in which case they are read from input +and \201possibly modified data values are\202 written to the output. +The +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +call itself is made by passing the input Funtools handle, and it is +assumed that the output file has been opened using this input handle +as its +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D(. + +)0 P(Consider the following example: +) 13 54 PR( typedef struct evstruct{ + int status; + float pi, pha, *phas; + double energy; + } *Ev, EvRec; + + FunColumnSelect\201fun, sizeof\201EvRec\202, NULL, + "status", "J", "r", FUN_OFFSET\201Ev, status\202, + "pi", "E", "rw", FUN_OFFSET\201Ev, pi\202, + "pha", "E", "rw", FUN_OFFSET\201Ev, pha\202, + "phas", "@9E", "rw", FUN_OFFSET\201Ev, phas\202, + "energy", "D", "w", FUN_OFFSET\201Ev, energy\202, + NULL\202;)RP( +As in the "read" example above, each time an row is read into the Ev +struct, the "status" column is converted to an int data type +\201regardless of its data type in the file\202 and stored in the status +value of the struct. Similarly, "pi" and "pha", and the phas vector +are all stored as floats. Since the "pi", "pha", and "phas" variables +are declared as "writable" as well as "readable", they also will be +written to the output file. Note, however, that the "status" variable +is declared as "readable" only, and hence it will not be written to +an output file. Finally, the "energy" column is declared as +"writable" only, meaning it will not be read from the input file. In +this case, it can be assumed that "energy" will be calculated in the +program before being output along with the other values. + +)0 P(In these simple cases, only the columns specified as "writable" will +be output using +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D(. However, +it often is the case that you want to merge the user columns back in +with the input columns, even in cases where not all of the input +column names are explicitly read or even known. For this important +case, the )BD(merge=[type])ES( keyword is provided in the plist string. + +)0 P(The )BD(merge=[type])ES( keyword tells Funtools to merge the columns from +the input file with user columns on output. It is normally used when +an input and output file are opened and the input file provides the +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( +for the output file. In this case, each time +)0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D( is called, the +raw input rows are saved in a special buffer. If +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D( then is called +\201before another call to +)0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D(\202, the contents +of the raw input rows are merged with the user rows according to the +value of )BD(type)ES( as follows: + +)UL()-1 LI( )BD(update)ES(: add new user columns, and update value of existing ones \201maintaining the input data type\202 + +)-1 LI( )BD(replace)ES(: add new user columns, and replace the data type +and value of existing ones. \201Note that if tlmin/tlmax values are not +specified in the replacing column, but are specified in the original +column being replaced, then the original tlmin/tlmax values are used +in the replacing column.\202 + +)-1 LI( )BD(append)ES(: only add new columns, do not "replace" or "update" existing ones)LU( + +)0 P(Consider the example above. If )BD(merge=update)ES( is specified in the +plist string, then "energy" will be added to the input columns, and +the values of "pi", "pha", and "phas" will be taken from the user +space \201i.e., the values will be updated from the original values, if +they were changed by the program\202. The data type for "pi", "pha", and +"phas" will be the same as in the original file. If +)BD(merge=replace)ES( is specified, both the data type and value of +these three input columns will be changed to the data type and value +in the user structure. If )BD(merge=append)ES( is specified, none of +these three columns will be updated, and only the "energy" column will +be added. Note that in all cases, "status" will be written from the +input data, not from the user record, since it was specified as read-only. + +)0 P(Standard applications will call +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +to define user columns. However, if this routine is not called, the +default behavior is to transfer all input columns into user space. For +this purpose a default record structure is defined such that each data +element is properly aligned on a valid data type boundary. This +mechanism is used by programs such as fundisp and funtable to process +columns without needing to know the specific names of those columns. +It is not anticipated that users will need such capabilities \201contact +us if you do!\202 + +)0 P(By default, )0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +reads/writes rows to/from an "array of structs", where each struct contains +the column values for a single row of the table. This means that the +returned values for a given column are not contiguous. You can +set up the IO to return a "struct of arrays" so that each of the +returned columns are contiguous by specifying )BD(org=structofarrays)ES( +\201abbreviation: )BD(org=soa)ES(\202 in the plist. +\201The default case is )BD(org=arrayofstructs)ES( or )BD(org=aos)ES(.\202 + +)0 P(For example, the default setup to retrieve rows from a table would be +to define a record structure for a single event and then call + )0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +as follows: +) 14 73 PR( typedef struct evstruct{ + short region; + double x, y; + int pi, pha; + double time; + } *Ev, EvRec; + + got = FunColumnSelect\201fun, sizeof\201EvRec\202, NULL, + "x", "D:10:10", mode, FUN_OFFSET\201Ev, x\202, + "y", "D:10:10", mode, FUN_OFFSET\201Ev, y\202, + "pi", "J", mode, FUN_OFFSET\201Ev, pi\202, + "pha", "J", mode, FUN_OFFSET\201Ev, pha\202, + "time", "1D", mode, FUN_OFFSET\201Ev, time\202, + NULL\202;)RP( +Subsequently, each call to +FunTableRowGet\201\202)EA( +will return an array of structs, one for each returned row. If instead you +wanted to read columns into contiguous arrays, you specify )BD(org=soa)ES(: +) 14 72 PR( typedef struct aevstruct{ + short region[MAXROW]; + double x[MAXROW], y[MAXROW]; + int pi[MAXROW], pha[MAXROW]; + double time[MAXROW]; + } *AEv, AEvRec; + + got = FunColumnSelect\201fun, sizeof\201AEvRec\202, "org=soa", + "x", "D:10:10", mode, FUN_OFFSET\201AEv, x\202, + "y", "D:10:10", mode, FUN_OFFSET\201AEv, y\202, + "pi", "J", mode, FUN_OFFSET\201AEv, pi\202, + "pha", "J", mode, FUN_OFFSET\201AEv, pha\202, + "time", "1D", mode, FUN_OFFSET\201AEv, time\202, + NULL\202;)RP( +Note that the only modification to the call is in the plist string. + +)0 P(Of course, instead of using staticly allocated arrays, you also can specify +dynamically allocated pointers: +) 16 75 PR( /* pointers to arrays of columns \201used in struct of arrays\202 */ + typedef struct pevstruct{ + short *region; + double *x, *y; + int *pi, *pha; + double *time; + } *PEv, PEvRec; + + got = FunColumnSelect\201fun, sizeof\201PEvRec\202, "org=structofarrays", + "$region", "@I", mode, FUN_OFFSET\201PEv, region\202, + "x", "@D:10:10", mode, FUN_OFFSET\201PEv, x\202, + "y", "@D:10:10", mode, FUN_OFFSET\201PEv, y\202, + "pi", "@J", mode, FUN_OFFSET\201PEv, pi\202, + "pha", "@J", mode, FUN_OFFSET\201PEv, pha\202, + "time", "@1D", mode, FUN_OFFSET\201PEv, time\202, + NULL\202;)RP( +Here, the actual storage space is either allocated by the user or by the +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( call\202. + +)0 P(In all of the above cases, the same call is made to retrieve rows, e.g.: +) 1 64 PR( buf = \201void *\202FunTableRowGet\201fun, NULL, MAXROW, NULL, &got\202;)RP( +However, the individual data elements are accessed differently. +For the default case of an "array of structs", the +individual row records are accessed using: +) 5 69 PR( for\201i=0; i<got; i++\202{ + ev = \201Ev\202buf+i; + fprintf\201stdout, "%.2f\200t%.2f\200t%d\200t%d\200t%.4f\200t%.4f\200t%21.8f\200n", + ev->x, ev->y, ev->pi, ev->pha, ev->dx, ev->dy, ev->time\202; + })RP( +For a struct of arrays or a struct of array pointers, we have a single struct +through which we access individual columns and rows using: +) 6 63 PR( aev = \201AEv\202buf; + for\201i=0; i<got; i++\202{ + fprintf\201stdout, "%.2f\200t%.2f\200t%d\200t%d\200t%.4f\200t%.4f\200t%21.8f\200n", + aev->x[i], aev->y[i], aev->pi[i], aev->pha[i], + aev->dx[i], aev->dy[i], aev->time[i]\202; + })RP( +Support for struct of arrays in the +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D( +call is handled analogously. + +)0 P(See the )0 2 A(evread example code)EA( +and +)0 2 A(evmerge example code)EA( +for working examples of how +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( is used. + + + + +)0 2 39 H(FunColumnActivate)WB 105 Sn()WB 33 Sn( - activate Funtools columns)EA()EH( + + +)BD() 3 55 PR( #include <funtools.h> + + void FunColumnActivate\201Fun fun, char *s, char *plist\202)RP()ES( + + +)0 P(The )BD(FunColumnActivate\201\202)ES( routine determines which columns \201set up +by )0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D(\202 +ultimately will be read and/or written. By default, all columns that +are selected using +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +are activated. The +)0 33 1 A(FunColumnActivate\201\202)33 0 TN TL()Ec /AF f D( +routine can be used to turn off/off activation of specific columns. + +)0 P(The first argument is the Fun handle associated with this set of +columns. The second argument is a space-delimited list of columns to +activate or de-activate. Columns preceded by "+" are activated and +columns preceded by a "-" are de-activated. If a column is named +without "+" or "-", it is activated. The reserved strings "$region" +and '$n' are used to activate a special columns containing the filter +region value and row value, respectively, associated with +this row. For example, if a filter containing two circular regions is +specified as part of the Funtools file name, this column will contain +a value of 1 or 2, depending on which region that row was in. The +reserved strings "$x" and "$y" are used to activate the current +binning columns. Thus, if the columns DX and DY are specified as +binning columns: +) 1 42 PR( [sh $] fundisp foo.fits[bincols=\201DX,DY\202])RP( +then "$x" and "$y" will refer to these columns in a call to +)0 33 1 A(FunColumnActivate\201\202)33 0 TN TL()Ec /AF f D(. + +)0 P(In addition, if the activation string contains only columns to be +activated, then the routine will de-activate all other columns. +Similarly, if the activation string contains only +columns to de-activate, then the routine will activate all other columns +before activating the list. This makes it simple to change the +activation state of all columns without having to know all of the +column names. For example: +)UL()-1 LI( )BD("pi pha time")ES( # only these three columns will be active +)-1 LI( )BD("-pi -pha -time")ES( # all but these columns will be active +)-1 LI( )BD("pi -pha")ES( # only pi is active, pha is not, others are not +)-1 LI( )BD("+pi -pha")ES( # same as above +)-1 LI( )BD("pi -pha -time")ES( # only pi is active, all others are not +)-1 LI( )BD("pi pha")ES( # pha and pi are active, all others are not +)-1 LI( )BD("pi pha -x -y")ES( # pha and pi are active, all others are not)LU( + +)0 P(You can use the column activation list to reorder columns, since +columns are output in the order specified. For example: +) 19 77 PR( # default output order + fundisp snr.ev'[cir 512 512 .1]' + X Y PHA PI TIME DX DY + -------- -------- -------- -------- --------------------- -------- -------- + 512 512 6 7 79493997.45854475 578 574 + 512 512 8 9 79494575.58943175 579 573 + 512 512 5 6 79493631.03866175 578 575 + 512 512 5 5 79493290.86521725 578 575 + 512 512 8 9 79493432.00990875 579 573 + + # re-order the output by specifying explicit order + fundisp snr.ev'[cir 512 512 .1]' "time x y dy dx pi pha" + TIME X Y DY DX PI PHA + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 512 512 574 578 7 6 + 79494575.58943175 512 512 573 579 9 8 + 79493631.03866175 512 512 575 578 6 5 + 79493290.86521725 512 512 575 578 5 5 + 79493432.00990875 512 512 573 579 9 8)RP( + +)0 P(A "+" sign by itself means to activate all columns, so that you can reorder +just a few columns without specifying all of them: +) 9 77 PR( # reorder 3 columns and then output the rest + fundisp snr.ev'[cir 512 512 .1]' "time pi pha +" + TIME PI PHA Y X DX DY + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 7 6 512 512 578 574 + 79494575.58943175 9 8 512 512 579 573 + 79493631.03866175 6 5 512 512 578 575 + 79493290.86521725 5 5 512 512 578 575 + 79493432.00990875 9 8 512 512 579 573)RP( +The column activation/deactivation is performed in the order of the +specified column arguments. This means you can mix "+", "-" \201which +de-activates all columns\202 and specific column names to reorder and +select columns in one command. For example, consider the following: +) 9 59 PR( # reorder and de-activate + fundisp snr.ev'[cir 512 512 .1]' "time pi pha + -x -y" + TIME PI PHA DX DY + --------------------- -------- -------- -------- -------- + 79493997.45854475 7 6 578 574 + 79494575.58943175 9 8 579 573 + 79493631.03866175 6 5 578 575 + 79493290.86521725 5 5 578 575 + 79493432.00990875 9 8 579 573)RP( +We first activate "time", "pi", and "pha" so that they are output first. +We then activate all of the other columns, and then de-activate "x" and "y". +Note that this is different from: +) 9 77 PR( # probably not what you want ... + fundisp snr.ev'[cir 512 512 .1]' "time pi pha -x -y +" + TIME PI PHA Y X DX DY + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 7 6 512 512 578 574 + 79494575.58943175 9 8 512 512 579 573 + 79493631.03866175 6 5 512 512 578 575 + 79493290.86521725 5 5 512 512 578 575 + 79493432.00990875 9 8 512 512 579 573)RP( +Here, "x" and "y" are de-activated, but then all columns including "x" and +"y" are again re-activated. + +)0 P(Typically, +)0 33 1 A(FunColumnActivate\201\202)33 0 TN TL()Ec /AF f D( uses a +list of columns that are passed into the program from the command line. For +example, the code for funtable contains the following: +) 9 66 PR( char *cols=NULL; + + /* open the input FITS file */ + if\201 !\201fun = FunOpen\201argv[1], "rc", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen input file: %s\200n", argv[1]\202; + + /* set active flag for specified columns */ + if\201 argc >= 4 \202 cols = argv[3]; + FunColumnActivate\201fun, cols, NULL\202;)RP( + +The )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( call sets the +default columns to be all columns in the input file. The +)0 33 1 A(FunColumnActivate\201\202)33 0 TN TL()Ec /AF f D( call +then allows the user to control which columns ultimately will be +activated \201i.e., in this case, written to the new file\202. For example: +) 1 39 PR( funtable test.ev foo.ev "pi pha time")RP( +will process only the three columns mentioned, while: +) 1 33 PR( funtable test.ev foo.ev "-time")RP( +will process all columns except "time". + +)0 P(If )0 33 1 A(FunColumnActivate\201\202)33 0 TN TL()Ec /AF f D( +is called with a null string, then the environment variable +)BD(FUN_COLUMNS)ES( will be used to provide a global value, if present. +This is the reason why we call the routine even if no columns +are specified on the command line \201see example above\202, instead +of calling it this way: +) 4 45 PR( /* set active flag for specified columns */ + if\201 argc >= 4 \202{ + FunColumnActivate\201fun, argv[3], NULL\202; + })RP( + + + + +)0 2 40 H(FunColumnLookup)WB 106 Sn()WB 34 Sn( - lookup a Funtools column)EA()EH( + +)BD() 5 56 PR( #include <funtools.h> + + int FunColumnLookup\201Fun fun, char *s, int which, + char **name, int *type, int *mode, + int *offset, int *n, int *width\202)RP()ES( + + +)0 P(The )BD(FunColumnLookup\201\202)ES( routine returns information about a named +\201or indexed\202 column. The first argument is the Fun handle associated +with this set of columns. The second argument is the name of the +column to look up. If the name argument is NULL, the argument that +follows is the zero-based index into the column array of the column +for which information should be returned. The next argument is a +pointer to a char *, which will contain the name of the column. The +arguments that follow are the addresses of int values into which +the following information will be returned: +)UL()-1 LI( )BD(type)ES(: data type of column: +)UL()-1 LI(A: ASCII characters +)-1 LI(B: unsigned 8-bit char +)-1 LI(I: signed 16-bit int +)-1 LI(U: unsigned 16-bit int \201not standard FITS\202 +)-1 LI(J: signed 32-bit int +)-1 LI(V: unsigned 32-bit int \201not standard FITS\202 +)-1 LI(E: 32-bit float +)-1 LI(D: 64-bit float)LU( +)-1 LI( )BD(mode)ES(: bit flag status of column, including: +)UL()-1 LI( COL_ACTIVE 1 is column activated? +)-1 LI( COL_IBUF 2 is column in the raw input data? +)-1 LI( COL_PTR 4 is column a pointer to an array? +)-1 LI( COL_READ 010 is read mode selected? +)-1 LI( COL_WRITE 020 is write mode selected? +)-1 LI( COL_REPLACEME 040 is this column being replaced by user data?)LU( +)-1 LI( )BD(offset)ES(: byte offset in struct +)-1 LI( )BD(n)ES(: number of elements \201i.e. size of vector\202 in this column +)-1 LI( )BD(width)ES(: size in bytes of this column)LU( +If the named column exists, the routine returns a positive integer, +otherwise zero is returned. \201The positive integer is the index+1 into +the column array where this column was located.\202 + +If NULL is passed as the return address of one \201or more\202 of these +values, no data is passed back for that information. For +example: +) 2 76 PR( if\201 !FunColumnLookup\201fun, "phas", 0, NULL NULL, NULL, NULL, &npha, NULL\202 \202 + gerror\201stderr, "can't find phas column\200n"\202;)RP( +only returns information about the size of the phas vector. + + + + +)0 2 41 H(FunTableRowGet)WB 107 Sn()WB 30 Sn( - get Funtools rows)EA()EH( + + +)BD() 4 68 PR( #include <funtools.h> + + void *FunTableRowGet\201Fun fun, void *rows, int maxrow, char *plist, + int *nrow\202)RP()ES( + + +)0 P(The )BD(FunTableRowGet\201\202)ES( routine retrieves rows from a Funtools +binary table or raw event file, and places the values of columns +selected by )0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( +into an array of user structs. Selected column values are +automatically converted to the specified user data type \201and to native +data format\202 as necessary. + +)0 P(The first argument is the Fun handle associated with this row data. +The second )BD(rows)ES( argument is the array of user structs into +which the selected columns will be stored. If NULL is passed, the +routine will automatically allocate space for this array. \201This +includes proper allocation of pointers within each struct, if the "@" +pointer type is used in the selection of columns. Note that if you +pass NULL in the second argument, you should free this space using the +standard free\201\202 system call when you are finished with the array of +rows.\202 The third )BD(maxrow)ES( argument specifies the maximum number +of rows to be returned. Thus, if )BD(rows)ES( is allocated by the +user, it should be at least of size maxrow*sizeof\201evstruct\202. + +)0 P(The fourth )BD(plist)ES( argument is a param list string. Currently, +the keyword/value pair "mask=transparent" is supported in the plist +argument. If this string is passed in the call's plist argument, then +all rows are passed back to the user \201instead of just rows passing +the filter\202. This is only useful when +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( also is +used to specify "$region" as a column to return for each row. In +such a case, rows found within a region have a returned region value +greater than 0 \201corresponding to the region id of the region in which +they are located\202, rows passing the filter but not in a region have +region value of -1, and rows not passing any filter have region +value of 0. Thus, using "mask=transparent" and the returned region +value, a program can process all rows and decide on an action based +on whether a given row passed the filter or not. + +)0 P(The final argument is a pointer to an int variable that will return +the actual number of rows returned. The routine returns a pointer to +the array of stored rows, or NULL if there was an error. \201This pointer +will be the same as the second argument, if the latter is non-NULL\202. +) 16 69 PR( /* get rows -- let routine allocate the row array */ + while\201 \201buf = \201Ev\202FunTableRowGet\201fun, NULL, MAXROW, NULL, &got\202\202 \202{ + /* process all rows */ + for\201i=0; i<got; i++\202{ + /* point to the i'th row */ + ev = buf+i; + /* rearrange some values. etc. */ + ev->energy = \201ev->pi+ev->pha\202/2.0; + ev->pha = -ev->pha; + ev->pi = -ev->pi; + } + /* write out this batch of rows */ + FunTableRowPut\201fun2, buf, got, 0, NULL\202; + /* free row data */ + if\201 buf \202 free\201buf\202; + })RP( +As shown above, successive calls to +)0 30 1 A(FunTableRowGet\201\202)30 0 TN TL()Ec /AF f D( will return the +next set of rows from the input file until all rows have been read, +i.e., the routine behaves like sequential Unix I/O calls such as +fread\201\202. See )0 2 A(evmerge example code)EA( for a +more complete example. + +)0 P(Note that FunTableRowGet\201\202 also can be called as FunEventsGet\201\202, for +backward compatibility. + + + + +)0 2 42 H(FunTableRowPut)WB 108 Sn()WB 31 Sn( - put Funtools rows)EA()EH( + + +) 1 70 PR()BD(int FunTableRowPut\201Fun fun, void *rows, int nev, int idx, char *plist\202)ES()RP( + + +The )BD(FunTableRowPut\201\202)ES( routine writes rows to a FITS binary +table, taking its input from an array of user structs that contain +column values selected by a previous call to +)0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D(. Selected +column values are automatically converted from native data format to +FITS data format as necessary. + +)0 P(The first argument is the Fun handle associated with this row data. +The second )BD(rows)ES( argument is the array of user structs to +output. The third )BD(nrow)ES( argument specifies the number number of +rows to write. The routine will write )BD(nrow)ES( records, starting +from the location specified by )BD(rows)ES(. + +)0 P(The fourth )BD(idx)ES( argument is the index of the first raw input +row to write, in the case where rows from the user buffer are +being merged with their raw input row counterparts \201see below\202. Note +that this )BD(idx)ES( value is has nothing to do with the +row buffer specified in argument 1. It merely matches the row +being written with its corresponding \201hidden\202 raw row. Thus, if you +read a number of rows, process them, and then write them out all at +once starting from the first user row, the value of )BD(idx)ES( +should be 0: +) 14 70 PR( Ev ebuf, ev; + /* get rows -- let routine allocate the row array */ + while\201 \201ebuf = \201Ev\202FunTableRowGet\201fun, NULL, MAXROW, NULL, &got\202\202 \202{ + /* process all rows */ + for\201i=0; i<got; i++\202{ + /* point to the i'th row */ + ev = ebuf+i; + ... + } + /* write out this batch of rows, starting with the first */ + FunTableRowPut\201fun2, \201char *\202ebuf, got, 0, NULL\202; + /* free row data */ + if\201 ebuf \202 free\201ebuf\202; + })RP( + +)0 P(On the other hand, if you write out the rows one at a time \201possibly +skipping rows\202, then, when writing the i'th row from the input +array of rows, set )BD(idx)ES( to the value of i: +) 14 70 PR( Ev ebuf, ev; + /* get rows -- let routine allocate the row array */ + while\201 \201ebuf = \201Ev\202FunTableRowGet\201fun, NULL, MAXROW, NULL, &got\202\202 \202{ + /* process all rows */ + for\201i=0; i<got; i++\202{ + /* point to the i'th row */ + ev = ebuf+i; + ... + /* write out the current \201i.e., i'th\202 row */ + FunTableRowPut\201fun2, \201char *\202ev, 1, i, NULL\202; + } + /* free row data */ + if\201 ebuf \202 free\201ebuf\202; + })RP( + +)0 P(The final argument is a param list string that is not currently used. +The routine returns the number of rows output. This should be equal +to the value passed in the third )BD(nrowFunParamGet)WB 37 Sn( - get a Funtools param value)EA()EH( + + +)BD() 9 74 PR( #include <funtools.h> + + int FunParamGetb\201Fun fun, char *name, int n, int defval, int *got\202 + + int FunParamGeti\201Fun fun, char *name, int n, int defval, int *got\202 + + double FunParamGetd\201Fun fun, char *name, int n, double defval, int *got\202 + + char *FunParamGets\201Fun fun, char *name, int n, char *defval, int *got\202)RP()ES( + + +)0 P(The four routines )BD(FunParamGetb\201\202)ES(, )BD(FunParamGeti\201\202)ES(, +)BD(FunParamGetd\201\202)ES(, and )BD(FunParamGets\201\202)ES(, return the value of +a FITS header parameter as a boolean, int, double, and string, +respectively. The string returned by )BD(FunParamGets\201\202)ES( is a malloc'ed +copy of the header value and should be freed when no longer needed. + +)0 P(The first argument is the Fun handle associated with the FITS header +being accessed. Normally, the header is associated with the FITS +extension that you opened with )BD(FunOpen\201\202)ES(. However, you can use +FunInfoPut\201\202 to specify access of the primary header. In particular, +if you set the FUN_PRIMARYHEADER parameter to 1, then the primary +header is used for all parameter access until the value is reset to +0. For example: +) 9 75 PR( int val; + FunParamGeti\201fun, "NAXIS", 1, 0, &got\202; # current header + val=1; + FunInfoPut\201fun, FUN_PRIMARYHEADER, &val, 0\202; # switch to ... + FunParamGeti\201fun, "NAXIS", 1, 0, &got\202; # ... primary header + FunParamGeti\201fun, "NAXIS", 2, 0, &got\202; # ... primary header + val=0; + FunInfoPut\201fun, FUN_PRIMARYHEADER, &val, 0\202; # switch back to ... + FunParamGeti\201fun, "NAXIS", 2, 0, &got\202; # current header)RP( + +)0 P(Alternatively, you can use the FUN_PRIMARY macro to access parameters +from the primary header on a per-parameter basis: +) 2 72 PR( FunParamGeti\201fun, "NAXIS1", 0, 0, &got\202; # current header + FunParamGeti\201FUN_PRIMARY\201fun\202, "NAXIS1", 0, 0, &got\202; # primary header)RP( +)BD(NB: FUN_PRIMARY is deprecated.)ES( +It makes use of a global parameter and therefore will not not +appropriate for threaded applications, when we make funtools +thread-safe. We recommend use of FunInfoPut\201\202 to switch between the +extension header and the primary header. + +)0 P(For output data, access to the primary header is only possible until +the header is written out, which usually takes place when the first +data are written. + +)0 P(The second argument is the name of the parameter to access. The third +)BD(n)ES( argument, if non-zero, is an integer that will be added as a +suffix to the parameter name. This makes it easy to use a simple loop +to process parameters having the same root name. For example, to +gather up all values of TLMIN and TLMAX for each column in a binary +table, you can use: +) 4 74 PR( for\201i=0, got=1; got; i++\202{ + fun->cols[i]->tlmin = \201int\202FunParamGeti\201fun, "TLMIN", i+1, 0.0, &got\202; + fun->cols[i]->tlmax = \201int\202FunParamGeti\201fun, "TLMAX", i+1, 0.0, &got\202; + })RP( + +)0 P(The fourth )BD(defval)ES( argument is the default value to return if +the parameter does not exist. Note that the data type of this +parameter is different for each specific FunParamGet\201\202 call. The final +)BD(got)ES( argument will be 0 if no param was found. Otherwise the +data type of the parameter is returned as follows: FUN_PAR_UNKNOWN +\201'u'\202, FUN_PAR_COMMENT \201'c'\202, FUN_PAR_LOGICAL \201'l'\202, FUN_PAR_INTEGER +\201'i'\202, FUN_PAR_STRING \201's'\202, FUN_PAR_REAL \201'r'\202, FUN_PAR_COMPLEX \201'x'\202. + +)0 P(These routines return the value of the header parameter, or the +specified default value if the header parameter does not exist. The +returned value is a malloc'ed string and should be freed when no +longer needed. + +)0 P(By default, )BD(FunParamGets\201\202)ES( returns the string value of the +named parameter. However, you can use FunInfoPut\201\202 to retrieve the +raw 80-character FITS card instead. In particular, if you set the +FUN_RAWPARAM parameter to 1, then card images will be returned by +FunParamGets\201\202 until the value is reset to 0. + +)0 P(Alternatively, if the FUN_RAW macro is applied to the name, then the +80-character raw FITS card is returned instead. +)BD(NB: FUN_RAW is deprecated.)ES( +It makes use of a global parameter and therefore will not not +appropriate for threaded applications, when we make funtools +thread-safe. We recommend use of FunInfoPut\201\202 to switch between the +extension header and the primary header. + +)0 P(Note that in addition to the behaviors described above, the +routine )BD(FunParamGets\201\202)ES( will return the 80 raw characters of the +)BD(nth)ES( FITS card \201including the comment\202 if )BD(name)ES( is specified as +NULL and )BD(n)ES( is positive. For example, to loop through all FITS +header cards in a given extension and print out the raw card, use: +) 9 55 PR( for\201i=1; ;i++\202{ + if\201 \201s = FunParamGets\201fun, NULL, i, NULL, &got\202\202 \202{ + fprintf\201stdout, "%.80s\200n", s\202; + free\201s\202; + } + else{ + break; + } + })RP( + + + + +)0 2 43 H(FunParamPut)WB 109 Sn()WB 38 Sn( - put a Funtools param value)EA()EH( + + +)BD() 13 71 PR( #include <funtools.h> + + int FunParamPutb\201Fun fun, char *name, int n, int value, char *comm, + int append\202 + + int FunParamPuti\201Fun fun, char *name, int n, int value, char *comm, + int append\202 + + int FunParamPutd\201Fun fun, char *name, int n, double value, int prec, + char *comm, int append\202 + + int FunParamPuts\201Fun fun, char *name, int n, char *value, char *comm, + int append\202)RP()ES( + + +)0 P(The four routines )BD(FunParamPutb\201\202)ES(, )BD(FunParamPuti\201\202)ES(, +)BD(FunParamPutd\201\202)ES(, and )BD(FunParamPuts\201\202)ES(, will set the value +of a FITS header parameter as a boolean, int, double, and string, +respectively. + +)0 P(The first argument is the Fun handle associated with the FITS header +being accessed. Normally, the header is associated with the FITS +extension that you opened with )BD(FunOpen\201\202)ES(. +However, you can use FunInfoPut\201\202 to specify that use of the primary +header. In particular, if you set the FUN_PRIMARYHEADER parameter to +1, then the primary header is used for all parameter access until the +value is reset to 0. For example: +) 5 69 PR( int val; + FunParamPuti\201fun, "NAXIS1", 0, 10, NULL, 1\202; # current header + val=1; + FunInfoPut\201fun, FUN_PRIMARYHEADER, &val, 0\202; # switch to ... + FunParamPuti\201fun, "NAXIS1", 0, 10, NULL, 1\202; # primary header)RP( +\201You also can use the deprecated FUN_PRIMARY macro, to access +parameters from the primary header.\202 + +)0 P(The second argument is the )BD(name)ES( of the parameter. \201 +In accordance with FITS standards, the special names )BD(COMMENT)ES( +and )BD(HISTORY)ES(, as well as blank names, are output without the "= " +value indicator in columns 9 and 10. + +)0 P(The third )BD(n)ES( argument, if non-zero, is an integer that will be +added as a suffix to the parameter name. This makes it easy to use a +simple loop to process parameters having the same root name. For +example, to set the values of TLMIN and TLMAX for each column in a +binary table, you can use: +) 4 70 PR( for\201i=0; i<got; i++\202{ + FunParamPutd\201fun, "TLMIN", i+1, tlmin[i], 7, "min column val", 1\202; + FunParamPutd\201fun, "TLMAX", i+1, tlmax[i], 7, "max column val", 1\202; + })RP( + +)0 P(The fourth )BD(defval)ES( argument is the value to set. Note that the +data type of this argument is different for each specific +FunParamPut\201\202 call. The )BD(comm)ES( argument is the comment +string to add to this header parameter. Its value can be NULL. The +final )BD(append)ES( argument determines whether the parameter is added +to the header if it does not exist. If set to a non-zero value, the +header parameter will be appended to the header if it does not exist. +If set to 0, the value will only be used to change an existing parameter. + +)0 P(Note that the double precision routine FunParamPutd\201\202 supports an +extra )BD(prec)ES( argument after the )BD(value)ES( argument, in order +to specify the precision when converting the double value to ASCII. In +general a 20.[prec] format is used \201since 20 characters are alloted to +a floating point number in FITS\202 as follows: if the double value being +put to the header is less than 0.1 or greater than or equal to +10**\20120-2-[prec]\202, then %20.[prec]e format is used \201i.e., scientific +notation\202; otherwise %20.[prec]f format is used \201i.e., numeric +notation\202. + +)0 P(As a rule, parameters should be set before writing the table or image. +It is, however, possible to update the value of an )BD(existing)ES( +parameter after writing an image or table \201but not to add a new +one\202. Such updating only works if the parameter already exists and if +the output file is seekable, i.e. if it is a disk file or is stdout +being redirected to a disk file. + +)0 P(It is possible to add a new parameter to a header after the data has +been written, but only if space has previously been reserved. To reserve +space, add a blank parameter whose value is the name of the parameter you +eventually will update. Then, when writing the new parameter, specify a +value of 2 for the append flag. The parameter writing routine will +first look to update an existing parameter, as usual. If an existing +parameter is not found, an appropriately-valued blank parameter will be +searched for and replaced. For example: +) 8 71 PR( /* add blank card to be used as a place holder for IPAR1 update */ + FunParamPuts\201fun, NULL, 0, "IPAR1", "INTEGER Param", 0\202; + ... + /* write header and data */ + FunTableRowPut\201fun, events, got, 0, NULL\202; + ... + /* update param in file after writing data -- note append = 2 here */ + FunParamPuti\201fun, "IPAR", 1, 400, "INTEGER Param", 2\202;)RP( + +)0 P(The parameter routines return a 1 if the routine was successful and a 0 on +failure. In general, the major reason for failure is that you did not +set the append argument to a non-zero value and the parameter did not +already exist in the file. + + + + +)0 2 44 H(FunInfoGet)WB 110 Sn()WB 35 Sn( - get information from Funtools struct)EA()EH( + + +)BD() 3 52 PR( #include <funtools.h> + + int FunInfoGet\201Fun fun, int type, char *addr, ...\202)RP()ES( + + +)0 P(The )BD(FunInfoGet\201\202)ES( routine returns information culled from the +Funtools structure. The first argument is the Fun handle from which +information is to be retrieved. This first required argument is followed +by a variable length list of pairs of arguments. Each pair consists +of an integer representing the type of information to retrieve and the +address where the information is to be stored. The list is terminated by a 0. +The routine returns the number of get actions performed. + +)0 P(The full list of available information is described below. Please note +that only a few of these will be useful to most application developers. +For imaging applications, the most important types are: +) 3 47 PR( FUN_SECT_DIM1 int /* dim1 for section */ + FUN_SECT_DIM2 int /* dim2 for section */ + FUN_SECT_BITPIX int /* bitpix for section */)RP( +These would be used to determine the dimensions and data type of image +data retrieved using the +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D( routine. For +example: +) 17 68 PR( /* extract and bin the data section into an image buffer */ + buf = FunImageGet\201fun, NULL, NULL\202; + /* get required information from funtools structure. + this should come after the FunImageGet\201\202 call, in case the call + changed sect_bitpix */ + FunInfoGet\201fun, + FUN_SECT_BITPIX, &bitpix, + FUN_SECT_DIM1, &dim1, + FUN_SECT_DIM2, &dim2, + 0\202; + /* loop through pixels and reset values below limit to value */ + for\201i=0; i<dim1*dim2; i++\202{ + switch\201bitpix\202{ + case 8: + if\201 cbuf[i] <= blimit \202 cbuf[i] = bvalue; + ... + })RP( +It is important to bear in mind that the call to +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D( +can change the value of FUN_SECT_BITPIX \201e.g. if "bitpix=n" is passed +in the param list\202. Therefore, a call to +)0 35 1 A(FunInfoGet\201\202)35 0 TN TL()Ec /AF f D( +should be made )BD(after)ES( the call to +)0 26 1 A(FunImageGet\201\202)26 0 TN TL()Ec /AF f D(, +in order to retrieve the updated bitpix value. +See the )0 2 A(imblank example code)EA( for more +details. + +)0 P(It also can be useful to retrieve the World Coordinate System +information from the Funtools structure. Funtools uses the the WCS +Library developed by Doug Mink at SAO, which is available +)R3 2 A(here)EA(. +\201More information about the WCSTools project in general can be found +)R4 2 A(here)EA(.\202 +The )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( routine initializes +two WCS structures that can be used with this WCS Library. +Applications can retrieve either of these two WCS structures using +)BD(FunInfoGet\201\202)ES(: +) 2 75 PR( FUN_WCS struct WorldCoor * /* wcs structure, for image coordinates*/ + FUN_WCS0 struct WorldCoor * /* wcs structure, for physical coordinates */)RP( +The structure retrieved by FUN_WCS is a WCS library handle containing +parameters suitable for use with image coordinates, regardless of whether the +data are images or tables. For this structure, the WCS reference point +\201CRPIX\202 has been converted to image coordinates if the underlying file +is a table \201and therefore in physical coordinates\202. You therefore must +ensure that the positions being passed to a routine like pix2wcs are in +image coordinates. The FUN_WCS0 structure has not had its WCS +reference point converted to image coordinates. It therefore is useful +when passing processing physical coordinates from a table. + +)0 P(Once a WCS structure has been retrieved, it can be used as the first +argument to the WCS library routines. \201If the structure is NULL, no +WCS information was contained in the file.\202 The two important WCS routines +that Funtools uses are: +) 5 66 PR( #include <wcs.h> + void pix2wcs \201wcs,xpix,ypix,xpos,ypos\202 + struct WorldCoor *wcs; /* World coordinate system structure */ + double xpix,ypix; /* x and y coordinates in pixels */ + double *xpos,*ypos; /* RA and Dec in degrees \201returned\202 */)RP( + +which converts pixel coordinates to sky coordinates, and: + +) 5 67 PR( void wcs2pix \201wcs, xpos, ypos, xpix, ypix, offscl\202 + struct WorldCoor *wcs; /* World coordinate system structure */ + double xpos,ypos; /* World coordinates in degrees */ + double *xpix,*ypix; /* coordinates in pixels */ + int *offscl; /* 0 if within bounds, else off scale */)RP( +which converts sky coordinates to pixel coordinates. Again, please note +that the wcs structure returned by FUN_WCS assumes that image coordinates +are passed to the pix2wcs routine, while FUN_WCS0 assumes that physical +coordinates are passed. + +)0 P(Note that funtools.h file automatically includes wcs.h. An example +program that utilizes these WCS structure to call WCS Library routines +is )0 2 A(twcs.c)EA(. + +)0 P(The following is the complete list of information that can be returned: +) 52 79 PR( name type comment + --------- -------- --------------------------------------------- + FUN_FNAME char * /* file name */ + FUN_GIO GIO /* gio handle */ + FUN_HEADER FITSHead /* fitsy header struct */ + FUN_TYPE int /* TY_TABLE,TY_IMAGE,TY_EVENTS,TY_ARRAY */ + FUN_BITPIX int /* bits/pixel in file */ + FUN_MIN1 int /* tlmin of axis1 -- tables */ + FUN_MAX1 int /* tlmax of axis1 -- tables */ + FUN_MIN2 int /* tlmin of axis2 -- tables */ + FUN_MAX2 int /* tlmax of axis2 -- tables */ + FUN_DIM1 int /* dimension of axis1 */ + FUN_DIM2 int /* dimension of axis2 */ + FUN_ENDIAN int /* 0=little, 1=big endian */ + FUN_FILTER char * /* supplied filter */ + FUN_IFUN FITSHead /* pointer to reference header */ + FUN_IFUN0 FITSHead /* same as above, but no reset performed */ + /* image information */ + FUN_DTYPE int /* data type for images */ + FUN_DLEN int /* length of image in bytes */ + FUN_DPAD int /* padding to end of extension */ + FUN_DOBLANK int /* was blank keyword defined? */ + FUN_BLANK int /* value for blank */ + FUN_SCALED int /* was bscale/bzero defined? */ + FUN_BSCALE double /* bscale value */ + FUN_BZERO double /* bzero value */ + /* table information */ + FUN_NROWS int /* number of rows in file \201naxis2\202 */ + FUN_ROWSIZE int /* size of user row struct */ + FUN_BINCOLS char * /* specified binning columns */ + FUN_OVERFLOW int /* overflow detected during binning? */)WR( + /* array information */ + FUN_SKIP int /* bytes to skip in array header */ + /* section information */ + FUN_SECT_X0 int /* low dim1 value of section */ + FUN_SECT_X1 int /* hi dim1 value of section */ + FUN_SECT_Y0 int /* low dim2 value of section */ + FUN_SECT_Y1 int /* hi dim2 value of section */ + FUN_SECT_BLOCK int /* section block factor */ + FUN_SECT_BTYPE int /* 's' \201sum\202, 'a' \201average\202 for binning */ + FUN_SECT_DIM1 int /* dim1 for section */ + FUN_SECT_DIM2 int /* dim2 for section */ + FUN_SECT_BITPIX int /* bitpix for section */ + FUN_SECT_DTYPE int /* data type for section */ + FUN_RAWBUF char * /* pointer to raw row buffer */ + FUN_RAWSIZE int /* byte size of raw row records */ + /* column information */ + FUN_NCOL int /* number of row columns defined */ + FUN_COLS FunCol /* array of row columns */ + /* WCS information */ + FUN_WCS struct WorldCoor * /* wcs structure, converted for images*/ + FUN_WCS0 struct WorldCoor * /* wcs structure, not converted */)RP( + +)0 P(Row applications would not normally need any of this information. +An example of how these values can be used in more complex programs is +the )0 2 A(evnext example code)EA(. In this program, the +time value for each row is changed to be the value of the succeeding +row. The program thus reads the time values for a batch of rows, +changes the time values to be the value for the succeeding row, and +then merges these changed time values back with the other columns to +the output file. It then reads the next batch, etc. + +)0 P(This does not work for the last row read in each batch, since there +is no succeeding row until the next batch is read. Therefore, the +program saves that last row until it has read the next batch, then +processes the former before starting on the new batch. In order to +merge the last row successfully, the code uses FUN_RAWBUF to save +and restore the raw input data associated with each batch of +rows. Clearly, this requires some information about how funtools +works internally. We are happy to help you write such programs as the +need arises. + + + + +)0 2 45 H(FunInfoPut)WB 111 Sn()WB 36 Sn( - put information into a Funtools struct)EA()EH( + + +)BD() 3 52 PR( #include <funtools.h> + + int FunInfoPut\201Fun fun, int type, char *addr, ...\202)RP()ES( + + +)0 P(The )BD(FunInfoPut\201\202)ES( routine puts information into a Funtools +structure. The first argument is the Fun handle from which +information is to be retrieved. After this first required argument +comes a variable length list of pairs of arguments. Each pair consists +of an integer representing the type of information to store and the +address of the new information to store in the struct. The variable +list is terminated by a 0. The routine returns the number of put +actions performed. + +)0 P(The full list of available information is described above with the +)0 36 1 A(FunInfoPut\201\202)36 0 TN TL()Ec /AF f D( routine. Although +use of this routine is expected to be uncommon, there is one +important situation in which it plays an essential part: writing +multiple extensions to a single output file. + +)0 P(For input, multiple extensions are handled by calling +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( for each extension to be +processed. When opening multiple inputs, it sometimes is the case that +you will want to process them and then write them \201including their +header parameters\202 to a single output file. To accomplish this, you +open successive input extensions using +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( and then call +)BD(FunInfoPut\201\202)ES( to set the +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( +of the output file to that of the newly opened input extension: +) 4 71 PR( /* open a new input extension */ + ifun=FunOpen\201tbuf, "r", NULL\202\202 \202 + /* make the new extension the reference handle for the output file */ + FunInfoPut\201ofun, FUN_IFUN, &ifun, 0\202;)RP( + +Resetting FUN_IFUN has same effect as when a funtools handle is passed +as the final argument to +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(. The state of the output +file is reset so that a new extension is ready to be written. +Thus, the next I/O call on the output extension will output the +header, as expected. + +)0 P(For example, in a binary table, after resetting FUN_IFUN you can then +call )0 32 1 A(FunColumnSelect\201\202)32 0 TN TL()Ec /AF f D( to +select the columns for output. When you then call +)0 27 1 A(FunImagePut\201\202)27 0 TN TL()Ec /AF f D( or )0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D(, a new +extension will be written that contains the header parameters from the +reference extension. Remember to call +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( to complete output of a +given extension. + +)0 P(A complete example of this capability is given +in the )0 2 A(evcol example code)EA(. +The central algorithm is: +)UL()-1 LI( open the output file without a reference handle +)-1 LI( loop: open each input extension in turn +)UL()-1 LI( set the reference handle for output to the newly opened input extension +)-1 LI( read the input rows or image and perform processing +)-1 LI( write new rows or image to the output file +)-1 LI( flush the output +)-1 LI( close input extension)LU( +)-1 LI( close output file)LU( +Note that )0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( is called +after processing each input extension in order to ensure that the +proper padding is written to the output file. A call to +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( also ensures that the +extension header is written to the output file in the case where there +are no rows to output. + +)0 P(If you wish to output a new extension without using a +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D(, you can +call FunInfoPut\201\202 to reset the FUN_OPS value directly. For a binary +table, you would then call FunColumnSelect\201\202 to set up the columns for +this new extension. +) 6 59 PR( /* reset the operations performed on this handle */ + int ops=0; + FunInfoPut\201ofun, FUN_OPS, &ops, 0\202; + FunColumnSelect\201fun, sizeof\201EvRec\202, NULL, + "MYCOL", "J", "w", FUN_OFFSET\201Ev, mycol\202, + NULL\202;)RP( +Once the FUN_OPS variable has been reset, the next I/O call on the +output extension will output the header, as expected. + + + + +)0 2 46 H(FunFlush)WB 112 Sn()WB 39 Sn( - flush data to output file)EA()EH( + + +)BD() 3 37 PR( #include <funtools.h> + + void FunFlush\201Fun fun, char *plist\202)RP()ES( + + +)0 P(The )BD(FunFlush)ES( routine will flush data to a FITS output file. In +particular, it can be called after all rows have been written \201using +the )0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D( routine\202 +in order to add the null padding that is required to complete a FITS +block. It also should be called after completely writing an image using +)0 27 1 A(FunImagePut\201\202)27 0 TN TL()Ec /AF f D( or after writing +the final row of an image using +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D(. + +)0 P(The )BD(plist)ES( \201i.e., parameter list\202 argument is a string +containing one or more comma-delimited )BD(keyword=value)ES( +parameters. If the plist string contains the parameter +"copy=remainder" and the file was opened with a reference file, which, +in turn, was opened for extension copying \201i.e. the input +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( mode also was "c" or "C"\202, +then FunFlush also will copy the remainder of the FITS extensions from +the input reference file to the output file. This normally would be +done only at the end of processing. + +)0 P(Note that )0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( is called +with "copy=remainder" in the mode string by FunClose\201\202. This means +that if you close the output file before the reference input file, it +is not necessary to call +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( explicitly, unless you +are writing more than one extension. See the +)0 2 A(evmerge example code)EA(. However, it is safe to +call )0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( more than once +without fear of re-writing either the padding or the copied +extensions. + +)0 P(In addition, if )0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( is +called on an output file with the plist set to "copy=reference" and if +the file was opened with a reference file, the reference extension is +written to the output file. This mechanism provides a simple way to +copy input extensions to an output file without processing the former. +For example, in the code fragment below, an input extension is set to +be the reference file for a newly opened output extension. If that +reference extension is not a binary table, it is written to the output +file: +) 22 73 PR( /* process each input extension in turn */ + for\201ext=0; ;ext++\202{ + /* get new extension name */ + sprintf\201tbuf, "%s[%d]", argv[1], ext\202; + /* open input extension -- if we cannot open it, we are done */ + if\201 !\201ifun=FunOpen\201tbuf, "r", NULL\202\202 \202 + break; + /* make the new extension the reference handle for the output file */ + FunInfoPut\201ofun, FUN_IFUN, &ifun, 0\202; + /* if its not a binary table, just write it out */ + if\201 !\201s=FunParamGets\201ifun, "XTENSION", 0, NULL, &got\202\202 || + strcmp\201s, "BINTABLE"\202\202{ + if\201 s \202 free\201s\202; + FunFlush\201ofun, "copy=reference"\202; + FunClose\201ifun\202; + continue; + } + else{ + /* process binary table */ + .... + } + })RP( + + + + +)0 2 47 H(FunClose)WB 113 Sn()WB 40 Sn( - close a Funtools data file)EA()EH( + + +)BD() 3 24 PR( #include <funtools.h> + + void FunClose\201Fun fun\202)RP()ES( + + +)0 P(The )BD(FunClose\201\202)ES( routine closes a previously-opened Funtools data +file, freeing control structures. If a +)0 23 1 A(Funtools reference handle)23 0 TN TL()Ec /AF f D( +was passed to +the )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( call for this file, +and if copy mode also was specified for that file, then +)0 40 1 A(FunClose\201\202)40 0 TN TL()Ec /AF f D( also will copy the +remaining extensions from the input file to the output file \201if the +input file still is open\202. Thus, we recommend always closing the +output Funtools file )BD(before)ES( the input file. \201Alternatively, +you can call )0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( +explicitly\202. + + + + +)0 2 48 H(FunRef:)WB 114 Sn()WB 23 Sn( the Funtools Reference Handle)EA()EH( + + +)0 2 49 H(Summary)WB 115 Sn()EH( +A description of how to use a Funtools reference handle to connect a +Funtools input file to an output file. + + +)0 2 50 H(Description)WB 116 Sn()EH( +)0 P(The Funtools reference handle connects a Funtools input file to a +Funtools output file so that parameters \201or even whole extensions\202 can +be copied from the one to the other. To make the connection, the Funtools +handle of the input file is passed to the +final argument of the +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( call for the output file: +) 4 67 PR( if\201 !\201ifun = FunOpen\201argv[1], "r", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen input file: %s\200n", argv[1]\202; + if\201 !\201ofun = FunOpen\201argv[2], "w", ifun\202\202 \202 + gerror\201stderr, "could not FunOpen output file: %s\200n", argv[2]\202;)RP( +It does not matter what type of input or output file \201or extension\202 is +opened, or whether they are the same type. When the output image or +binary table is written using +)0 27 1 A(FunImagePut\201\202)27 0 TN TL()Ec /AF f D( +or +)0 31 1 A(FunTableRowPut\201\202)31 0 TN TL()Ec /AF f D( +an appropriate header will be written first, with parameters copied +from the input extension. Of course, invalid parameters will be +removed first, e.g., if the input is a binary table and the output is +an image, then binary table parameters such as TFORM, TUNIT, +etc. parameters will not be copied to the output. + +)0 P(Use of a reference handle also allows default values to be passed +to +)0 27 1 A(FunImagePut\201\202)27 0 TN TL()Ec /AF f D( in order to +write out an output image with the same dimensions and data type +as the input image. To use the defaults from the input, a value +of 0 is entered for dim1, dim2, and bitpix. For example: +) 5 40 PR( fun = FunOpen\201argv[1], "r", NULL\202; + fun2 = FunOpen\201argv[2], "w", fun\202; + buf = FunImageGet\201fun, NULL, NULL\202; + ... process image data ... + FunImagePut\201fun2, buf, 0, 0, 0, NULL\202;)RP( +Of course, you often want to get information about the data type +and dimensions of the image for processing. The above code +is equivalent to the following: +) 7 61 PR( fun = FunOpen\201argv[1], "r", NULL\202; + fun2 = FunOpen\201argv[2], "w", fun\202; + buf = FunImageGet\201fun, NULL, NULL\202; + FunInfoGet\201fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, + FUN_SECT_BITPIX, &bitpix, 0\202; + ... process image data ... + FunImagePut\201fun2, buf, dim1, dim2, bitpix, NULL\202;)RP( + +)0 P(It is possible to change the reference handle for a given output Funtools +handle using the +)0 36 1 A(FunInfoPut\201\202)36 0 TN TL()Ec /AF f D( routine: +) 2 71 PR( /* make the new extension the reference handle for the output file */ + FunInfoPut\201fun2, FUN_IFUN, &fun, 0\202;)RP( +When this is done, Funtools specially resets the output file to start +a new output extension, which is connected to the new input reference +handle. You can use this mechanism to process multiple input extensions +into a single output file, by successively opening the former and +setting the reference handle for the latter. For example: +) 18 73 PR( /* open a new output FITS file */ + if\201 !\201fun2 = FunOpen\201argv[2], "w", NULL\202\202 \202 + gerror\201stderr, "could not FunOpen output file: %s\200n", argv[2]\202; + /* process each input extension in turn */ + for\201ext=0; ;ext++\202{ + /* get new extension name */ + sprintf\201tbuf, "%s[%d]", argv[1], ext\202; + /* open it -- if we cannot open it, we are done */ + if\201 !\201fun=FunOpen\201tbuf, "r", NULL\202\202 \202 + break; + /* make the new extension the reference handle for the output file */ + FunInfoPut\201fun2, FUN_IFUN, &fun, 0\202; + ... process ... + /* flush output extension \201write padding, etc.\202 */ + FunFlush\201fun2, NULL\202; + /* close the input extension */ + FunClose\201fun\202; + })RP( +In this example, the output file is opened first. Then each successive +input extension is opened, and the output reference handle is set to +the newly opened input handle. After data processing is performed, the +output extension is flushed and the input extension is closed, in +preparation for the next input extension. +)0 P(Finally, a reference handle can be used to copy other extensions from +the input file to the output file. Copy of other extensions is +controlled by adding a "C" or "c" to the mode string of the +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( +call )BD(of the input reference file)ES(. If "C" is specified, then +other extensions are )BD(always)ES( copied \201i.e., copy is forced by the +application\202. If "c" is used, then other extensions are copied if the +user requests copying by adding a plus sign "+" to the extension name +in the bracket specification. For example, the )BD(funtable)ES( +program utilizes user-specified "c" mode so that the second example +below will copy all extensions: +) 4 60 PR( # copy only the EVENTS extension + csh> funtable "test.ev[EVENTS,circle\201512,512,10\202]" foo.ev + # copy ALL extensions + csh> funtable "test.ev[EVENTS+,circle\201512,512,10\202]" foo.ev)RP( +When extension copy is specified in the input file, the call to +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( +on the input file delays the actual file open until the output file +also is opened \201or until I/O is performed on the input file, which +ever happens first\202. Then, when the output file is opened, the input +file is also opened and input extensions are copied to the output +file, up to the specific extension being opened. Processing of input +and output extensions then proceed. +)0 P(When extension processing is complete, the remaining extensions need to +be copied from input to output. This can be done explicitly, using the +)0 39 1 A(FunFlush\201\202)39 0 TN TL()Ec /AF f D( +call with the "copy=remaining" plist: +) 1 34 PR( FunFlush\201fun, "copy=remaining"\202;)RP( +Alternatively, this will happen automatically, if the output file +is closed )BD(before)ES( the input file: +) 5 77 PR( /* we could explicitly flush remaining extensions that need copying */ + /* FunFlush\201fun2, "copy=remaining"\202; */ + /* but if we close output before input, end flush is done automatically */ + FunClose\201fun2\202; + FunClose\201fun\202;)RP( + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 51 H(Last)WB 117 Sn( updated: December 1, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (files.html) D +/Ti (Funtools Data Files) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 42 Sn( + + +)0 2 52 H(FunFiles:)WB 120 Sn()WB 118 Sn( Funtools Data Files)EA()EH( + + +)0 2 53 H(Summary)WB 121 Sn()EH( +This document describes the data file formats \201FITS, array, raw +events\202 as well as the file types \201gzip, socket, etc.\202 supported +by Funtools. + + +)0 2 54 H(Description)WB 122 Sn()EH( +)0 P(Funtools supports FITS images and binary tables, and binary files +containing array \201homogeneous\202 data or event \201heterogeneous\202 data. +IRAF-style brackets are appended to the filename to specify various +kinds of information needed to characterize these data: +) 3 49 PR( file[ext|ind|ARRAY\201\202|EVENTS\201\202,section][filters] + or + file[ext|ind|ARRAY\201\202|EVENTS\201\202,section,filters])RP( +where: +)UL()-1 LI( )BD(file)ES( is the Funtools file name +)-1 LI( )BD(ext)ES( is the FITS extension name +)-1 LI( )BD(ind)ES( is the FITS extension number +)-1 LI( )BD(ARRAY\201\202)ES( is an array specification +)-1 LI( )BD(EVENTS\201\202)ES( is an event specification +)-1 LI( )BD(section)ES( is the image section specification +)-1 LI( )BD(filters)ES( are spatial region and table \201row\202 filters)LU( + +)0 2 55 H(Supported)WB 123 Sn()WB 43 Sn( Data Formats)EA()EH( +)0 P(Funtools programs \201and the underlying libraries\202 support the +following data file formats: +)UL()-1 LI( FITS images \201and image extensions\202 +)-1 LI( FITS binary tables +)-1 LI( binary files containing an array of homogeneous data +)-1 LI( binary files containing events, i.e. records of heterogeneous data +)-1 LI( column-based text files, which are documented )0 47 1 A(here)47 0 TN TL()Ec /AF f D( +)-1 LI( non-disk files and lists of files)LU( +Information needed to identify and characterize +the event or image data can be specified on the command line +using IRAF-style bracket notation appended to the filename: +) 5 69 PR( foo.fits # open FITS default extension + image.fits[3] # open FITS extension #3 + events.fits[EVENTS] # open EVENTS extension + array.file[ARRAY\201s1024\202] # open 1024x1024 short array + events.file[EVENTS\201x:1024,y:1024...\202] # open non-FITS event list)RP( +Note that in many Unix shells \201e.g., csh and tcsh\202, filenames must +be enclosed in quotes to protect the brackets from shell processing. + +)0 2 56 H(FITS)WB 124 Sn()WB 44 Sn( Images and Binary Tables)EA()EH( +)0 P(When )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( opens a FITS file +without a bracket specifier, the default behavior is to look for a +valid image in the primary HDU. In the absence of a primary image, +Funtools will try to open an extension named either )BD(EVENTS)ES( or +)BD(STDEVT)ES(, if one of these exists. This default behavior supports +both FITS image processing and standard X-ray event list processing +\201which, after all, is what we at SAO/HEAD do\202. + +)0 P(In order to open a FITS binary table or image extension explicitly, it +is necessary to specify either the extension name or the extension +number in brackets: +) 3 71 PR( foo.fits[1] # open extension #1: the primary HDU + foo.fits[3] # open extension #3 of a FITS file + foo.fits[GTI] # open GTI extension of a FITS file)RP( +The ext argument specifies the name of the FITS extension \201i.e. the +value of the EXTENSION header parameter in a FITS extension\202, while +the index specifies the value of the FITS EXTVER header parameter. +Following FITS conventions, extension numbers start at 1. + +)0 P(When a FITS data file is opened for reading using +)0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D(, the specified extension +is automatically located and is used to initialize the Funtools internal +data structures. + +)0 2 57 H(Non-FITS)WB 125 Sn()WB 45 Sn( Raw Event Files)EA()EH( + +In addition to FITS tables, Funtools programs and libraries can operate +on non-FITS files containing heterogeneous event records. To specify +such an event file, use: + +)UL()-1 LI( file[EVENTS\201event-spec\202] +)-1 LI( file[EVENTS\201\202])LU( +where )BD(event-spec)ES( is a string that specified the names, data +types, and optional image dimensions for each element of the event +record: +)UL()-1 LI( [name]:[n][type]:[\201lodim:\202hidim])LU( + +)0 P(Data types follow standard conventions for FITS binary tables, but include +two extra unsigned types \201'U' and 'V'\202: +)UL()-1 LI( )BD(B)ES( -- unsigned 8-bit char +)-1 LI( )BD(I)ES( -- signed 16-bit int +)-1 LI( )BD(J)ES( -- signed 32-bit int +)-1 LI( )BD(K)ES( -- signed 64-bit int +)-1 LI( )BD(E)ES( -- 32-bit float +)-1 LI( )BD(D)ES( -- 64-bit float +)-1 LI( )BD(U)ES( -- unsigned 16-bit int +)-1 LI( )BD(V)ES( -- unsigned 32-bit int)LU( +An optional integer value )BD(n)ES( can be prefixed to the type to indicate +that the element is an array of n values. For example: +) 1 37 PR( foo.fits[EVENTS\201x:I,y:I,status:4J\202])RP( +defines x and y as 16-bit ints and status as an array of 4 32-bit ints. + +)0 P(Furthermore, image dimensions can be attached to the event specification +in order to tell Funtools how to bin the events into an image. They +follow the conventions for the FITS TLMIN/TLMAX keywords. If the low +image dimension is not specified, it defaults to 1. Thus: + +)UL()-1 LI( RAWX:J:1:100 +)-1 LI( RAWX:J:100)LU( +both specify that the dimension of this column runs from 1 to 100. + +)0 P(NB: it is required that all padding be specified in the record +definition. Thus, when writing out whole C structs instead of +individual record elements, great care must be taken to include +the compiler-added padding in the event definition. + +)0 P(For example, suppose a FITS binary table has the following set of column +definitions: +) 22 54 PR( TTYPE1 = 'X ' / Label for field + TFORM1 = '1I ' / Data type for field + TLMIN1 = 1 / Min. axis value + TLMAX1 = 10 / Max. axis value + TTYPE2 = 'Y ' / Label for field + TFORM2 = '1I ' / Data type for field + TLMIN2 = 2 / Min. axis value + TLMAX2 = 11 / Max. axis value + TTYPE3 = 'PHA ' / Label for field + TFORM3 = '1I ' / Data type for field + TTYPE4 = 'PI ' / Label for field + TFORM4 = '1J ' / Data type for field + TTYPE5 = 'TIME ' / Label for field + TFORM5 = '1D ' / Data type for field + TTYPE6 = 'DX ' / Label for field + TFORM6 = '1E ' / Data type for field + TLMIN6 = 1 / Min. axis value + TLMAX6 = 10 / Max. axis value + TTYPE7 = 'DY ' / Label for field + TFORM7 = '1E ' / Data type for field + TLMIN7 = 3 / Min. axis value + TLMAX7 = 12 / Max. axis value)RP( + +An raw event file containing these same data would have the event +specification: +) 1 61 PR( EVENTS\201X:I:10,Y:I:2:11,PHA:I,PI:J,TIME:D,DX:E:10,DY:E:3:12\202)RP( + +)0 P(If no event specification string is included within the EVENTS\201\202 operator, +then the event specification is taken from the )BD(EVENTS)ES( environment +variable: +) 1 65 PR( setenv EVENTS "X:I:10,Y:I:10,PHA:I,PI:J,TIME:D,DX:E:10,DY:E:10")RP( + +)0 P(In addition to knowing the data structure, it is necessary to know the +)EM(endian)ES( ordering of the data, i.e., whether or not the data is +in )EM(bigendian)ES( format, so that we can convert to the native +format for this platform. This issue does not arise for FITS Binary +Tables because all FITS files use big-endian ordering, regardless of +platform. But for non-FITS data, big-endian data produced on a Sun +workstation but read on a Linux PC needs to be byte-swapped, since PCs +use little-endian ordering. To specify an ordering, use the +)EM(bigendian=)ES( or )EM(endian=)ES( keywords on the command-line +or the EVENTS_BIGENDIAN or EVENTS_ENDIAN environment variables. The +value of the )EM(bigendian)ES( variables should be "true" or "false", +while the value of the )EM(endian)ES( variables should be "little" or +"big". + +)0 P(For example, a PC can access data produced by a Sun using: +) 7 35 PR( hrc.nepr[EVENTS\201\202,bigendian=true] +or + hrc.nepr[EVENTS\201\202,endian=big] +or + setenv EVENTS_BIGENDIAN true +or + setenv EVENTS_ENDIAN big)RP( +If none of these are specified, the data are assumed to follow the +format for that platform and no byte-swapping is performed. + +)0 2 58 H(Non-FITS)WB 126 Sn()WB 46 Sn( Array Files)EA()EH( + +In addition to FITS images, Funtools programs and libraries can operate +on non-FITS files containing arrays of homogeneous data. To specify +an array file, use: +)UL()-1 LI( file[ARRAY\201array-spec\202] +)-1 LI( file[ARRAY\201\202])LU( + +where array-spec is of the form: +)UL()-1 LI( [type][dim1][.dim2][:skip][endian])LU( + +and where [type] is: +)UL()-1 LI( b \2018-bit unsigned char\202 +)-1 LI( s \20116-bit short int\202 +)-1 LI( u \20116-bit unsigned short int\202 +)-1 LI( i \20132-bit int\202 +)-1 LI( r,f \20132-bit float\202 +)-1 LI( d \20164-bit float\202)LU( + +)0 P(The dim1 specification is required, but dim2 is optional and defaults +to dim1. The skip specification is optional and defaults to 0. The +optional endian specification can be 'l' or 'b' and defaults to the +endian type for the current machine. + +)0 P(If no array specification is included within the ARRAY\201\202 operator, +then the array specification is taken from the )BD(ARRAY)ES( environment +variable. For example: + +) 7 76 PR( foo.arr[ARRAY\201r512\202] # bitpix=-32 dim1=512 dim2=512 + foo.arr[ARRAY\201r512.400\202] # bitpix=-32 dim1=512 dim2=400 + foo.arr[ARRAY\201r512.400]\202 # bitpix=-32 dim1=512 dim2=400 + foo.arr[ARRAY\201r512.400:2880\202] # bitpix=-32 dim1=512 dim2=400 skip=2880 + foo.arr[ARRAY\201r512l\202] # bitpix=-32 dim1=512 dim2=512 endian=little + setenv ARRAY "r512.400:2880" + foo.arr[ARRAY\201\202] # bitpix=-32 dim1=512 dim2=400 skip=2880)RP( + +)0 2 59 H(Specifying)WB 127 Sn()WB 49 Sn( Image Sections)EA()EH( + +Once a data file \201and possibly, a FITS extension\202 has been specified, +the next \201optional\202 part of a bracket specification can be used to +select image )BD(section)ES( information, i.e., to specify the x,y +limits of an image section, as well as the blocking factor to apply to +that section. This information can be added to any file specification but +only is used by Funtools image processing routines. + +)0 P(The format of the image section specification is one of the following: +)UL()-1 LI( file[xy0:xy1,block] +)-1 LI( file[x0:x1,y0:y1,block] +)-1 LI( file[x0:x1,*,block] +)-1 LI( file[*,y0:y1,block] +)-1 LI( file[*,block])LU( +where the limit values can be ints or "*" for default. A single "*" +can be used instead of val:val, as shown. Note that blocking is +applied to the section after it is extracted. + +)0 P(In addition to image sections specified by the lo and hi x,y limits, image +sections using center positions can be specified: +)UL()-1 LI( file[dim1@xcen,dim2@ycen] +)-1 LI( file[xdim2@xcen@ycen] +)-1 LI( file[dim1@xcen,dim2@ycen,block] +)-1 LI( file[dim@xcen@ycen,block])LU( +Note that the \201float\202 values for dim, dim1, dim2, xcen, ycen must be +specified or else the expression does not make sense! + +)0 P(In all cases, block is optional and defaults to 1. An 's' or 'a' can +be appended to signify "sum" or "average" blocking \201default is "sum"\202. +Section specifications are given in image coordinates by default. If you +wish to specify physical coordinates, add a 'p' as the last character +of the section specification, before the closing bracket. +For example: +) 2 21 PR()UL()-1 LI( file[-8:-7,-8:-7p] +)-1 LI( file[-8:-7,-8:-7,2p])LU()RP( +A section can be specified in any Funtools file name. If the operation +to be applied to that file is an imaging operation, then the +specification will be utilized. If the operation is purely a table +operation, then the section specification is ignored. + +)0 P(Do not be confused by: +) 2 15 PR( foo.fits[2] + foo.fits[*,2])RP( +The former specifies opening the second extension of the FITS file. +The latter specifies application of block 2 to the image section. + +)0 P(Note that the section specification must come after +any of FITS )BD(ext)ES( name or )BD(ind)ES( number, +but all sensible defaults are supported: +)UL()-1 LI( file[ext] +)-1 LI( file[ext,index] +)-1 LI( file[index] +)-1 LI( file[ext,section] +)-1 LI( file[ext,index,section] +)-1 LI( file[index,section] +)-1 LI( file[section])LU( + +)0 2 60 H(Binning)WB 128 Sn()WB 50 Sn( FITS Binary Tables and Non-FITS Event Files)EH( + +If a FITS binary table or a non-FITS raw event file is to be binned +into a 2D image \201e.g., using the +)0 10 1 A(funimage)10 0 TN TL()Ec /AF f D( +program\202, it is necessary to specify the two columns to be used for the +binning, as well as the dimensions of the image. Funtools first looks +for a specifier of the form: +) 1 74 PR( bincols=\201[xnam[:tlmin[:tlmax:[binsiz]]]],[ynam[:tlmin[:tlmax[:binsiz]]]]\202)RP( +in bracket syntax, and uses the column names thus specified. The tlmin, tlmax, +and binsiz specifiers determine the image binning dimensions using: +) 2 56 PR( dim = \201tlmax - tlmin\202/binsiz \201floating point data\202 + dim = \201tlmax - tlmin\202/binsiz + 1 \201integer data\202)RP( +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters are present in the FITS binary +table header, respectively. If only one parameter is specified, it is +assumed to be tlmax, and tlmin defaults to 1. If two parameters are +specified, they are assumed to be tlmin and tlmax. + +For example, to bin an HRC event list columns "VPOS" and "UPOS", use: +) 1 31 PR( hrc.nepr[bincols=\201VPOS,UPOS\202])RP( +or +) 1 42 PR( hrc.nepr[bincols=\201VPOS:49152,UPOS:4096\202])RP( +Note that you can optionally specify the dimensions of these columns +to cover cases where neither TLMAX keywords are defined in +the header. If either dimension is specified, then both must be specified. + +)0 P(You can set the FITS_BINCOLS or EVENTS_BINCOLS environment variable as +an alternative to adding the "bincols=" specifier to each file name +for FITS binary tables and raw event files, respectively. If no +binning keywords or environment variables are specified, or if the +specified columns are not in the binary table, the Chandra parameters +CPREF \201or PREFX\202 are searched for in the FITS binary table header. +Failing this, columns named "X" and "Y" are sought. If these are not +found, the code looks for columns containing the characters "X" and +"Y". Thus, you can bin on "DETX" and "DETX" columns without +specifying them, if these are the only column names containing the "X" +and "Y" characters. + +)0 P(Ordinarily, each event or row contributes one count to an image pixel +during the 2D binning process. Thus, if five events all have the same +\201x,y\202 position, the image pixel value for that position will have a +value of five. It is possible to specify a variable contribution +for each event by using the vcol=[colname] filter spec: +) 1 15 PR( vcol=[colname])RP( +The vcol colname is a column containing a numeric value in each event row +that will be used as the contribution of the given event to its image +pixel. For example, consider an event file that has the following content: +) 10 24 PR( x:e:4 y:e:4 v:e + ------ ------ ---- + 1 1 1.0 + 2 2 2.0 + 3 3 3.0 + 4 4 0.0 + 1 1 1.0 + 2 2 2.0 + 3 3 3.0 + 4 4 4.0)RP( +There are two events with x,y value of \2011,1\202 so ordinarily a 2D image will +have a value of 2 in the \2011,1\202 pixel. If the v column is specified as the +value column: +) 1 20 PR( foo.fits'[vcol=v]')RP( +then each pixel will contain the additive sum of the associated \201x,y\202 +column values from the v column. For example, image pixel \2011,1\202 will +contain 1. + 1. = 2, image pixel \2012,2\202 will contain \2012 + 2\202 = 4, etc. + +)0 P(An important variation on the use of a value column to specify the +contribution an event makes to an image pixel is when the value column +contains the reciprocal of the event contribution. For this case, the +column name should be prefixed with a / \201divide sign\202 thus: +) 1 21 PR( foo.fits'[vcol=/v]')RP( +Each image pixel value will then be the sum of the reciprocals of the value +column. A zero in the value column results in NaN \201not a number\202. +Thus, in the above example, image pixel \2011.1\202 will contain 1/1 + 1/1 = 2, +image pixel \2012,2\202 will contain \2011/2 + 1/2\202 = 1, etc. Image pixel \2014,4\202 +will contain \2011/0 + 1/4\202 = NaN. + +)0 P(You can set the FITS_VCOL or EVENTS_VCOL environment variable as +an alternative to adding the "vcol=" specifier to each file name +for FITS binary tables and raw event files, respectively. + +)0 P(Finally, when binning events, the data type of the resulting 2D image +must be specified. This can be done with the "bitpix=[n]" keyword in +the bracket specification. For example: +) 1 45 PR( events.fits[bincols=\201VPOS,UPOS\202,bitpix=-32])RP( +will create a floating point image binned on columns VPOS and UPOS. +If no bitpix keyword is specified, bitpix=32 is assumed. As with +bincols values, you also can use the FITS_BITPIX and EVENTS_BITPIX +environment variables to set this value for FITS binary tables and +raw event files, respectively. + +)0 P(The )BD(funimage)ES( program also allows you to create a 1D image projection +along any column of a table by using the )BD(bincols=[column])ES( +filter specification and specifying a single column. +For example, the following command projects a 1D image along +the chipx column of a table: +) 1 43 PR( funimage ev.fits'[bincols=chipx]' im.fits)RP( +See )0 10 1 A(funimage)10 0 TN TL()Ec /AF f D( for more +information about creating 1D and 2D images. + +)0 P(Finally, please note that Funtools supports most FITS standards. +We will add missing support as required by the community. In general, +however, we do not support non-standard extensions. For example, we +sense the presence of the binary table 'variable length array' +proposed extension and we pass it along when copying and filtering +files, but we do not process it. We will add support for new standards +as they become official. + +)0 2 61 H(Table)WB 129 Sn()WB 119 Sn( and Spatial Region Filters)EH( +)0 P(Note that, in addition extensions and image sections, Funtools bracket +notation can be used to specify table and spatial region filters. These +filters are always placed after the image section information. They +can be specified in the same bracket or in a separate bracket +immediately following: +)UL()-1 LI( file[ext|ind|ARRAY\201\202|EVENTS\201\202,section][filters] +)-1 LI( file[ext|ind|ARRAY\201\202|EVENTS\201\202,section,filters])LU( +where: +)UL()-1 LI( )BD(file)ES( is the Funtools file name +)-1 LI( )BD(ARRAY\201\202)ES( is an array specification +)-1 LI( )BD(EVENTS\201\202)ES( is an event list specification +)-1 LI( )BD(ext)ES( is the FITS extension name +)-1 LI( )BD(ind)ES( is the FITS extension number +)-1 LI( )BD(section)ES( is the image section to extract +)-1 LI( )BD(filters)ES( are spatial region and table \201row\202 filters to apply)LU( + +The topics of table and region filtering are covered in detail in: +)UL()-1 LI()0 52 1 A(Table Filtering)52 0 TN TL()Ec /AF f D( +)-1 LI()0 54 1 A(Spatial Region Filtering)54 0 TN TL()Ec /AF f D()LU( + +)0 2 62 H(Disk)WB 130 Sn()WB 51 Sn( Files and Other Supported File Types)EA()EH( +)0 P(The specified )BD(file)ES( usually is an ordinary disk file. In +addition, gzip'ed files are supported in Funtools: gzip'ed input files +are automatically uncompressed as they are read, and gzip'ed output +files are compressed as they are written. NB: if a FITS binary table +is written in gzip format, the number of rows in the table will be set +to -1. Such a file will work with Funtools programs but will not work +with other FITS programs such as ds9. + +)0 P(The special keywords "stdin" and "stdout" designate Unix standard +input and standard output, respectively. The string "-" \201hyphen\202 will +be taken to mean "stdin" if the file is opened for reading and +"stdout" if the file is opened for writing. + +)0 P(A file also can be an INET socket on the same or another machine using +the syntax: +) 1 14 PR( machine:port)RP( +Thus, for example: +) 1 14 PR( karapet:1428)RP( +specifies that I/O should be performed to/from port 1428 on the +machine karapet. If no machine name is specified, the default is to +use the current machine: +) 1 7 PR( :1428)RP( +This means to open port 1428 on the current machine. Socket support +allows you to generate a distributed pipe: +) 2 48 PR( on karapet: funtask1 in.fits bynars:1428 + on bynars: funtask2 :1428 out.fits)RP( +The socket mechanism thus supports simple parallel processing using +)BD(process decomposition)ES(. Note that parallel processing using +)BD(data decomposition)ES( is supported via the )BD(section)ES( specifier \201see +below\202, and the )BD(row#)ES( specifier, which is part of +)0 52 1 A(Table Filtering)52 0 TN TL()Ec /AF f D(. + +)0 P(A file also can be a pointer to shared memory using the syntax: +) 1 22 PR( shm:[id|@key][:size])RP( +A shared memory segment is specified with a )BD(shm:)ES( prefix, +followed by either the shared memory id or the shared memory key +\201where the latter is prefixed by the '@' character\202. The size \201in +bytes\202 of the shared memory segment can then be appended \201preceded by +the ':' character\202. If the size specification is absent, the code will +attempt to determine the length automatically. + +If the open mode contains the string "w+", then the memory segment will be +created if it does not exist. \201It also will be released and deleted when the +file is closed.\202 In the case where a memory segment is being created, the +length of the segment is required. + +)0 P(A file also can be Unix piped command \201i.e. a program to run\202 using the syntax: +) 1 31 PR( "pipe: command arg1 ... argn")RP( +The output from the command must be a valid FITS file. It is important +to use quotes to protect spaces so that command arguments are passed +correctly. A silly example is: +) 1 60 PR( fundisp "pipe: funtable 'foo.fits[cir 512 512 .1]' stdout")RP( +This seemed like a good idea at the time ... + +)0 2 63 H(Lists)WB 131 Sn()WB 51 Sn( of Files)EA()EH( + +)0 P(Funtools also will process a list of files as a single file using the +syntax: +) 1 31 PR( "list: file1 file2 ... filen")RP( +The files in the list are separated by whitespace. Any of the +above file types can be used. For example, if two files, foo1.fits and +foo2.fits, are part of the same observation, they can be processed as +a single file \201using their own filters\202: +) 17 77 PR( fundisp "list: foo1.fits[cir\201512,512,10\202] foo2.fits[cir\201511,511,10\202]" + X Y PHA PI TIME DX DY + -------- -------- -------- -------- --------------------- -------- -------- + 512 512 6 7 79493997.45854475 578 574 + 512 512 8 9 79494575.58943175 579 573 + 512 512 5 6 79493631.03866175 578 575 + 512 512 5 5 79493290.86521725 578 575 + 512 512 8 9 79493432.00990875 579 573 + 511 511 5 5 79488631.09462625 580 575 + 511 511 10 11 79488780.60006675 580 573 + 511 511 4 4 79494562.35474326 580 575 + 511 511 6 6 79488203.01561825 580 575 + 511 511 6 6 79488017.99730176 580 575 + 511 511 4 4 79494332.45355175 580 575 + 511 511 9 10 79492685.94014275 581 574 + 511 511 5 5 79487708.71298325 580 575 + 511 511 8 9 79493719.00160225 581 573)RP( +Again, note that it is important to avoid spaces in the filters +because the list separator also is whitespace. To protect whitespace +in a filter, enclose the file specification in quotes: +) 1 72 PR( fundisp "list: 'foo1.fits[cir 512 512 .1]' foo2.fits[cir\201511,511,.1\202]")RP( + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 64 H(Last)WB 132 Sn( updated: February 15, 2006)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (text.html) D +/Ti (Column-based Text Files) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 47 Sn( + + +)0 2 65 H(Funtext:)WB 134 Sn()WB 133 Sn( Support for Column-based Text Files)EA()EH( + + +)0 2 66 H(Summary)WB 135 Sn()EH( +)0 P(This document contains a summary of the options for processing column-based +text files. + + +)0 2 67 H(Description)WB 136 Sn()EH( + +)0 P(Funtools will automatically sense and process "standard" +column-based text files as if they were FITS binary tables without any +change in Funtools syntax. In particular, you can filter text files +using the same syntax as FITS binary tables: +) 3 54 PR( fundisp foo.txt'[cir 512 512 .1]' + fundisp -T foo.txt > foo.rdb + funtable foo.txt'[pha=1:10,cir 512 512 10]' foo.fits)RP( + +)0 P(The first example displays a filtered selection of a text file. The +second example converts a text file to an RDB file. The third example +converts a filtered selection of a text file to a FITS binary table. + +)0 P(Text files can also be used in Funtools image programs. In this case, +you must provide binning parameters \201as with raw event files\202, using +the bincols keyword specifier: + +) 1 75 PR( bincols=\201[xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]])RP( + +For example: +) 1 64 PR( funcnts foo'[bincols=\201x:1024,y:1024\202]' "ann 512 512 0 10 n=10")RP( + +)0 2 68 H(Standard)WB 137 Sn( Text Files)EH( + +)0 P(Standard text files have the following characteristics: + +)UL()-1 LI( Optional comment lines start with # +)-1 LI( Optional blank lines are considered comments +)-1 LI( An optional table header consists of the following \201in order\202: +)UL( )-1 LI( a single line of alpha-numeric column names + )-1 LI( an optional line of unit strings containing the same number of cols + )-1 LI( an optional line of dashes containing the same number of cols)LU( +)-1 LI( Data lines follow the optional header and \201for the present\202 consist of + the same number of columns as the header. +)-1 LI( Standard delimiters such as space, tab, comma, semi-colon, and bar.)LU( + +)0 P(Examples: + +) 26 46 PR( # rdb file + foo1 foo2 foo3 foos + ---- ---- ---- ---- + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + # multiple consecutive whitespace and dashes + foo1 foo2 foo3 foos + --- ---- ---- ---- + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + # comma delims and blank lines + foo1,foo2,foo3,foos + + 1,2.2,3,xxxx + 10,20.2,30,yyyy + + # bar delims with null values + foo1|foo2|foo3|foos + 1||3|xxxx + 10|20.2||yyyy + + # header-less data + 1 2.2 3 xxxx + 10 20.2 30 yyyy)RP( + +)0 P(The default set of token delimiters consists of spaces, tabs, commas, +semi-colons, and vertical bars. Several parsers are used +simultaneously to analyze a line of text in different ways. One way +of analyzing a line is to allow a combination of spaces, tabs, and +commas to be squashed into a single delimiter \201no null values between +consecutive delimiters\202. Another way is to allow tab, semi-colon, and +vertical bar delimiters to support null values, i.e. two consecutive +delimiters implies a null value \201e.g. RDB file\202. A successful parser +is one which returns a consistent number of columns for all rows, with +each column having a consistent data type. More than one parser can +be successful. For now, it is assumed that successful parsers all +return the same tokens for a given line. \201Theoretically, there are +pathological cases, which will be taken care of as needed\202. Bad parsers +are discarded on the fly. + +)0 P(If the header does not exist, then names "col1", "col2", etc. are +assigned to the columns to allow filtering. Furthermore, data types +for each column are determined by the data types found in the columns +of the first data line, and can be one of the following: string, int, +and double. Thus, all of the above examples return the following +display: +) 4 58 PR( fundisp foo'[foo1>5]' + FOO1 FOO2 FOO3 FOOS + ---------- --------------------- ---------- ------------ + 10 20.20000000 30 yyyy)RP( + +)0 2 69 H(Comments)WB 138 Sn( Convert to Header Params)EH( + +)0 P(Comments which precede data rows are converted into header parameters and +will be written out as such using funimage or funhead. Two styles of comments +are recognized: + +)0 P(1. FITS-style comments have an equal sign "=" between the keyword and +value and an optional slash "/" to signify a comment. The strict FITS +rules on column positions are not enforced. In addition, strings only +need to be quoted if they contain whitespace. For example, the following +are valid FITS-style comments: + +) 5 55 PR( # fits0 = 100 + # fits1 = /usr/local/bin + # fits2 = "/usr/local/bin /opt/local/bin" + # fits3c = /usr/local/bin /opt/local/bin /usr/bin + # fits4c = "/usr/local/bin /opt/local/bin" / path dir)RP( + +Note that the fits3c comment is not quoted and therefore its value is the +single token "/usr/local/bin" and the comment is "opt/local/bin /usr/bin". +This is different from the quoted comment in fits4c. + +)0 P(2. Free-form comments can have an optional colon separator between the +keyword and value. In the absence of quote, all tokens after the +keyword are part of the value, i.e. no comment is allowed. If a string +is quoted, then slash "/" after the string will signify a comment. +For example: + +) 9 54 PR( # com1 /usr/local/bin + # com2 "/usr/local/bin /opt/local/bin" + # com3 /usr/local/bin /opt/local/bin /usr/bin + # com4c "/usr/local/bin /opt/local/bin" / path dir + + # com11: /usr/local/bin + # com12: "/usr/local/bin /opt/local/bin" + # com13: /usr/local/bin /opt/local/bin /usr/bin + # com14c: "/usr/local/bin /opt/local/bin" / path dir)RP( + +)0 P(Note that com3 and com13 are not quoted, so the whole string is part of +the value, while comz4c and com14c are quoted and have comments following +the values. + +)0 P(Some text files have column name and data type information in the header. +You can specify the format of column information contained in the +header using the "hcolfmt=" specification. See below for a detailed +description. + +)0 2 70 H(Multiple)WB 139 Sn( Tables in a Single File)EH( + +)0 P( +Multiple tables are supported in a single file. If an RDB-style file +is sensed, then a ^L \201vertical tab\202 will signify end of +table. Otherwise, an end of table is sensed when a new header \201i.e., +all alphanumeric columns\202 is found. \201Note that this heuristic does not +work for single column tables where the column type is ASCII and the +table that follows also has only one column.\202 You also can specify +characters that signal an end of table condition using the )BD(eot=)ES( +keyword. See below for details. + +)0 P(You can access the nth table \201starting from 1\202 in a multi-table file +by enclosing the table number in brackets, as with a FITS extension: + +) 1 18 PR( fundisp foo'[2]')RP( +The above example will display the second table in the file. +\201Index values start at 1 in oder to maintain logical compatibility +with FITS files, where extension numbers also start at 1\202. + + +)0 2 71 H(TEXT\201\202)WB 140 Sn( Specifier)EH( + +)0 P(As with ARRAY\201\202 and EVENTS\201\202 specifiers for raw image arrays and raw +event lists respectively, you can use TEXT\201\202 on text files to pass +key=value options to the parsers. An empty set of keywords is +equivalent to not having TEXT\201\202 at all, that is: + +) 2 23 PR( fundisp foo + fundisp foo'[TEXT\201\202]')RP( + +are equivalent. A multi-table index number is placed before the TEXT\201\202 +specifier as the first token, when indexing into a multi-table: + + fundisp foo'[2,TEXT\201...\202]' + +)0 P(The filter specification is placed after the TEXT\201\202 specifier, separated +by a comma, or in an entirely separate bracket: + +) 2 47 PR( fundisp foo'[TEXT\201...\202,circle 512 512 .1]' + fundisp foo'[2,TEXT\201...\202][circle 512 512 .1]')RP( + +)0 2 72 H(Text\201\202)WB 141 Sn( Keyword Options)EH( + +)0 P( +The following is a list of keywords that can be used within the TEXT\201\202 +specifier \201the first three are the most important\202: + +)0 DL( +)0 P()0 DT( delims="[delims]" +)DD(Specify token delimiters for this file. Only a single parser having these +delimiters will be used to process the file. +) 2 40 PR( fundisp foo.fits'[TEXT\201delims="!"\202]' + fundisp foo.fits'[TEXT\201delims="\200t%"\202]')RP( + +)0 P()0 DT( comchars="[comchars]" +)DD( Specify comment characters. You must include "\200n" to allow blank lines. +These comment characters will be used for all standard parsers \201unless delims +are also specified\202. +) 1 42 PR( fundisp foo.fits'[TEXT\201comchars="!\200n"\202]')RP( + +)0 P()0 DT( cols="[name1:type1 ...]" +)DD( Specify names and data type of columns. This overrides header +names and/or data types in the first data row or default names and +data types for header-less tables. +) 1 70 PR( fundisp foo.fits'[TEXT\201cols="x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e"\202]')RP( +)0 P(If the column specifier is the only keyword, then the cols= is not +required \201in analogy with EVENTS\201\202\202: +) 1 63 PR( fundisp foo.fits'[TEXT\201x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e\202]')RP( +Of course, an index is allowed in this case: +) 1 65 PR( fundisp foo.fits'[2,TEXT\201x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e\202]')RP( + +)0 P()0 DT( eot="[eot delim]" +)DD( Specify end of table string specifier for multi-table files. RDB +files support ^L. The end of table specifier is a string and the whole +string must be found alone on a line to signify EOT. For example: +) 1 37 PR( fundisp foo.fits'[TEXT\201eot="END"\202]')RP( +will end the table when a line contains "END" is found. Multiple lines +are supported, so that: +) 1 43 PR( fundisp foo.fits'[TEXT\201eot="END\200nGAME"\202]')RP( +will end the table when a line contains "END" followed by a line +containing "GAME". +)0 P(In the absence of an EOT delimiter, a new table will be sensed when a new +header \201all alphanumeric columns\202 is found. + +)0 P()0 DT( null1="[datatype]" +)DD( Specify data type of a single null value in row 1. +Since column data types are determined by the first row, a null value +in that row will result in an error and a request to specify names and +data types using cols=. If you only have a one null in row 1, you don't +need to specify all names and columns. Instead, use null1="type" to +specify its data type. + +)0 P()0 DT( alen=[n] +)DD(Specify size in bytes for ASCII type columns. +FITS binary tables only support fixed length ASCII columns, so a +size value must be specified. The default is 16 bytes. + +)0 P()0 DT( nullvalues=["true"|"false"] +)DD(Specify whether to expect null values. +Give the parsers a hint as to whether null values should be allowed. The +default is to try to determine this from the data. + +)0 P()0 DT( whitespace=["true"|"false"] +)DD( Specify whether surrounding white space should be kept as part of +string tokens. By default surrounding white space is removed from +tokens. + +)0 P()0 DT( header=["true"|"false"] +)DD(Specify whether to require a header. This is needed by tables +containing all string columns \201and with no row containing dashes\202, in +order to be able to tell whether the first row is a header or part of +the data. The default is false, meaning that the first row will be +data. If a row dashes are present, the previous row is considered the +column name row. + +)0 P()0 DT( units=["true"|"false"] +)DD(Specify whether to require a units line. +Give the parsers a hint as to whether a row specifying units should be +allowed. The default is to try to determine this from the data. + +)0 P()0 DT( i2f=["true"|"false"] +)DD(Specify whether to allow int to float conversions. +If a column in row 1 contains an integer value, the data type for that +column will be set to int. If a subsequent row contains a float in +that same column, an error will be signaled. This flag specifies that, +instead of an error, the float should be silently truncated to +int. Usually, you will want an error to be signaled, so that you can +specify the data type using cols= \201or by changing the value of +the column in row 1\202. + +)0 P()0 DT( comeot=["true"|"false"|0|1|2] +)DD(Specify whether comment signifies end of table. +If comeot is 0 or false, then comments do not signify end of table and +can be interspersed with data rows. If the value is true or 1 \201the +default for standard parsers\202, then non-blank lines \201e.g. lines +beginning with '#'\202 signify end of table but blanks are allowed +between rows. If the value is 2, then all comments, including blank +lines, signify end of table. + +)0 P()0 DT( lazyeot=["true"|"false"] +)DD(Specify whether "lazy" end of table should be permitted \201default is +true for standard formats, except rdb format where explicit ^L is required +between tables\202. A lazy EOT can occur when a new table starts directly +after an old one, with no special EOT delimiter. A check for this EOT +condition is begun when a given row contains all string tokens. If, in +addition, there is a mismatch between the number of tokens in the +previous row and this row, or a mismatch between the number of string +tokens in the prev row and this row, a new table is assumed to have +been started. For example: +) 9 20 PR( ival1 sval3 + ----- ----- + 1 two + 3 four + + jval1 jval2 tval3 + ----- ----- ------ + 10 20 thirty + 40 50 sixty)RP( +Here the line "jval1 ..." contains all string tokens. In addition, +the number of tokens in this line \2013\202 differs from the number of +tokens in the previous line \2012\202. Therefore a new table is assumed +to have started. Similarly: +) 9 20 PR( ival1 ival2 sval3 + ----- ----- ----- + 1 2 three + 4 5 six + + jval1 jval2 tval3 + ----- ----- ------ + 10 20 thirty + 40 50 sixty)RP( +Again, the line "jval1 ..." contains all string tokens. The number of +string tokens in the previous row \2011\202 differs from the number of +tokens in the current row\2013\202. We therefore assume a new table as been +started. This lazy EOT test is not performed if lazyeot is explicitly +set to false. + +)0 P()0 DT( hcolfmt=[header column format] +)DD( Some text files have column name and data type information in the header. +For example, VizieR catalogs have headers containing both column names +and data types: +) 3 95 PR( #Column e_Kmag \201F6.3\202 ?\201k_msigcom\202 K total magnitude uncertainty \2014\202 [ucd=ERROR] + #Column Rflg \201A3\202 \201rd_flg\202 Source of JHK default mag \2016\202 [ucd=REFER_CODE] + #Column Xflg \201I1\202 [0,2] \201gal_contam\202 Extended source contamination \20110\202 [ucd=CODE_MISC])RP( + +while Sextractor files have headers containing column names alone: + +) 4 79 PR( # 1 X_IMAGE Object position along x [pixel] + # 2 Y_IMAGE Object position along y [pixel] + # 3 ALPHA_J2000 Right ascension of barycenter \201J2000\202 [deg] + # 4 DELTA_J2000 Declination of barycenter \201J2000\202 [deg])RP( +The hcolfmt specification allows you to describe which header lines +contain column name and data type information. It consists of a string +defining the format of the column line, using "$col" \201or "$name"\202 to +specify placement of the column name, "$fmt" to specify placement of the +data format, and "$skip" to specify tokens to ignore. You also can +specify tokens explicitly \201or, for those users familiar with how +sscanf works, you can specify scanf skip specifiers using "%*"\202. +For example, the VizieR hcolfmt above might be specified in several ways: +) 3 67 PR( Column $col \201$fmt\202 # explicit specification of "Column" string + $skip $col \201$fmt\202 # skip one token + %*s $col \201$fmt\202 # skip one string \201using scanf format\202)RP( +while the Sextractor format might be specified using: +) 2 59 PR( $skip $col # skip one token + %*d $col # skip one int \201using scanf format\202)RP( +You must ensure that the hcolfmt statement only senses actual column +definitions, with no false positives or negatives. For example, the +first Sextractor specification, "$skip $col", will consider any header +line containing two tokens to be a column name specifier, while the +second one, "%*d $col", requires an integer to be the first token. In +general, it is preferable to specify formats as explicitly as +possible. + +)0 P(Note that the VizieR-style header info is sensed automatically by the +funtools standard VizieR-like parser, using the hcolfmt "Column $col +\201$fmt\202". There is no need for explicit use of hcolfmt in this case. + +)0 P()0 DT( debug=["true"|"false"] +)DD(Display debugging information during parsing. +)LD( + +)0 2 73 H(Environment)WB 142 Sn( Variables)EH( + +)0 P( +Environment variables are defined to allow many of these TEXT\201\202 values to be +set without having to include them in TEXT\201\202 every time a file is processed: + +) 10 36 PR( keyword environment variable + ------- -------------------- + delims TEXT_DELIMS + comchars TEXT_COMCHARS + cols TEXT_COLUMNS + eot TEXT_EOT + null1 TEXT_NULL1 + alen TEXT_ALEN + bincols TEXT_BINCOLS + hcolfmt TEXT_HCOLFMT)RP( + +)0 2 74 H(Restrictions)WB 143 Sn( and Problems)EH( + +)0 P( +As with raw event files, the '+' \201copy extensions\202 specifier is not +supported for programs such as funtable. + +)0 P(String to int and int to string data conversions are allowed by the +text parsers. This is done more by force of circumstance than by +conviction: these transitions often happens with VizieR catalogs, +which we want to support fully. One consequence of allowing these +transitions is that the text parsers can get confused by columns which +contain a valid integer in the first row and then switch to a +string. Consider the following table: +) 4 20 PR( xxx yyy zzz + ---- ---- ---- + 111 aaa bbb + ccc 222 ddd)RP( +The xxx column has an integer value in row one a string in row two, +while the yyy column has the reverse. The parser will erroneously +treat the first column as having data type int: +) 5 38 PR( fundisp foo.tab + XXX YYY ZZZ + ---------- ------------ ------------ + 111 'aaa' 'bbb' + 1667457792 '222' 'ddd')RP( +while the second column is processed correctly. This situation can be avoided +in any number of ways, all of which force the data type of the first column +to be a string. For example, you can edit the file and explicitly quote the +first row of the column: +) 10 40 PR( xxx yyy zzz + ---- ---- ---- + "111" aaa bbb + ccc 222 ddd + + [sh] fundisp foo.tab + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd')RP( +You can edit the file and explicitly set the data type of the first column: +) 10 40 PR( xxx:3A yyy zzz + ------ ---- ---- + 111 aaa bbb + ccc 222 ddd + + [sh] fundisp foo.tab + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd')RP( +You also can explicitly set the column names and data types of all columns, +without editing the file: +) 5 52 PR( [sh] fundisp foo.tab'[TEXT\201xxx:3A,yyy:3A,zzz:3a\202]' + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd')RP( +The issue of data type transitions \201which to allow and which to disallow\202 +is still under discussion. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 75 H(Last)WB 144 Sn( updated: August 3, 2007)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (view.html) D +/Ti (Database View Support for Tables) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 48 Sn( + + +)0 2 76 H(Funview:)WB 146 Sn()WB 145 Sn( Database View Support for Tables)EA()EH( + + +)0 2 77 H(Summary)WB 147 Sn()EH( +)0 P(This document contains a summary of the options for utilizing +database-inspired Views of tables. + + +)0 2 78 H(Description)WB 148 Sn()EH( + +)0 2 79 H(Database)WB 149 Sn( Views)EH( +)0 P(In database parlance, a )BD(View)ES( defines a "virtual table", i.e., +a description of row and/or column selection filters \201but with no +permanent storage space allocated\202. When used in place of a table, a +View selects the specified rows and/or columns from one or more real +tables. Views enable you to see complicated data tables in a more +convenient format. They also can be used as a security mechanism, by +restricting user access to specific columns and/or rows. [See: +) 1 72 PR(http://www.cs.unibo.it/~ciaccia/COURSES/RESOURCES/SQLTutorial/sqlch5.htm)RP( +for a good discussion of SQL Views.] + +)0 P(Funtools supports an expanded notion of Views for all tabular data +\201FITS tables, raw binary tables, and ASCII column files\202. Funtools +Views allow you to pre-set values for the filter specification, the +columns to activate, and display format \201though the latter is for +fundisp only\202. Setting the filter and column activation values +provides functionality equivalent to that of a classical database +View, while the ability to set the format is similar to classical +report writing capabilities. + +)0 2 80 H(Funtools)WB 150 Sn( View Attributes)EH( +)0 P(A Funtools View is a text file containing one or more of the following +columns: +) 7 46 PR( column description + ------ ----------------------------- + view name of view + file data file name or template + filter filter specification + columns columns to activate + format fundisp format specification)RP( +All of the attribute columns are optional, including +the )BD(view)ES( name itself. This means that a View can be named or +unnamed. Unnamed Views can refer to a specific file or a template of +files \201obviously if neither the view or the file column is specified, +the input View specification will never be used\202. You can specify any +combination of filter, column, and format parameters. \201It also is +possible to apply file-specific View to other files; see the discussion +on )BD(View Lists)ES( below\202. Each column has a size limit of 1024 characters. + +)0 P(For example, consider the following View file: +) 13 70 PR( view file format columns filter + ---- ---------------------- ------ ------------ ------- + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 + x2 ${HOME}/data/snr.ev x y pi pha cir 512 512 .1 + x1 ${HOME}/data/snr.ev cir 512 512 .1 + x1a ${HOME}/data/snr.ev x y pi pha + x0 ${HOME}/data/snr.ev + xf I=%4d + xc x y pi pha + xr cir 512 512 .1 + *.ev x y pi pha + *.fit x y dx dy cir 400 400 3 + *.fits I=%3d x y dx dy cir 400 400 3)RP( +This database example is in rdb format, i.e. using tab delimiters and +permitting null values. Any valid ASCII table format is acceptable, +but if you use a format that does not permit null values, it will be +necessary to quote the null strings. + +)0 P(The first five entries \201x3, x2, x1, x1a, x0\202 are named entries defining +default values specifically for the snr.ev data file. Typically, you +would use these Views by specifying View name, and the corresponding +file, filter, column, and format values would be used. Note that the x0 +View is essentially an alias for the pathname of this file. + +)0 P(The next three entries define defaults that can be applied to any +file. You typically would use these View names in conjunction with +a specific file name \201see )BD(View Lists)ES( below\202 so that the associated +parameter\201s\202 were applied to that file. + +)0 P(The last three entry in the database define unnamed Views that +pertains to all files ending with the specified templates. In these +cases, any View that specifies a file name matching the file template +would be processed with the associated parameter attributes. + +)0 2 81 H(Invoking)WB 151 Sn( a Funtools View \201in Place of an Input File\202)EH( +)0 P(To use a Funtools View, you simply pre-pend the "v:" prefix to a View name or +a file name where an input file name usually is specified. For example: +) 1 14 PR( fundisp v:x3)RP( +specifies that the View named x3 \201with its file name and associated +parameters\202 is processed as the input file to fundisp. Using the +example database, above, this is equivalent to: +) 1 73 PR( fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha")RP( +That is, the format is used with fundisp's -f \201format\202 switch, while the +filename and extension are composed of the x3 View's filename and +region filter. + +)0 P(Similarly, executing a command such as: +) 1 19 PR( fundisp v:foo.fit)RP( +will match the unnamed View associated with the template "*.fit". +This is equivalent to executing: +) 1 46 PR( fundisp foo.fit'[cir 400 400 3]' "x y dx dy")RP( +Of course, if you omit the "v:" prefix, then no View processing takes place: +) 2 66 PR( fundisp foo.fit # process foo.fit without any View parameters + fundisp x3 # error \201assuming there is no file named x3\202)RP( + +)0 2 82 H(Basic)WB 152 Sn( View Matching Rules)EH( + +)0 P(When a "v:" prefix is recognized, Funtools searches for a View database +file in the following order: +) 5 59 PR( location description + ------------ ------------------------------------ + FUN_VIEWFILE environment variable \201any file name\202 + ./.funtools.vu hidden file, default name + $HOME/.funtools.vu hidden file, default name)RP( +The first View database file located is used to construct a new +filename, as well as an activation column specification and a format +specification. The following rules are used: + +)0 P(1. An attempt is made to match the input name \201i.e., the part of the +input View after the "v:" prefix\202 against the )BD(view)ES( column value +\201if present\202 of each row in the database. If a match is found, the +values of all non-blank columns are saved for later use. Also note +that the first match terminates the search: i.e., )BD(the order of the +database rows matters)ES(. + +)0 P(2. If no )BD(view)ES( match is made, an attempt is made to match the input +name against the )BD(file)ES( column value \201if present\202. Matching is +performed on the full pathname of both the input name and the +database file name, and on the non-directory \201root\202 part of these +files. This means that the root specification: +) 1 18 PR( fundisp v:snr.ev)RP( +will match a row in the database that has a full pathname in the file, +allowing you to use a )BD(file)ES(-matched View without having to +specify the full pathname. In this example, the "v:snr.ev" View +specification will match the first row \201v:x3\202 in the database: +) 1 67 PR( x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1)RP( +even though the row contains a fully qualified pathname as the file +value. Once again, values of all non-blank columns are saved, and the +first match terminates the search. + +)0 P(3. If neither a )BD(view)ES( or a )BD(view)ES( match has been found, +then a simple template match is attempted against the )BD(view)ES( +values. Template matching supports a simplified version of file +globbing \201not a regular expression\202, with support for a single "*" +\201all characters\202, "?" \201single character\202, or "[...]" \201range\202 specification. + +)0 P(4. If no template match was found on the )BD(view)ES( column, then a +simple template match is attempted against the )BD(file)ES( columns. + +)0 P(5. If no match is found, then the filename \201minus the "v:" prefix\202 is +returned. + +)0 2 83 H(More)WB 153 Sn( on View Matching Rules: Single vs. Multiple Matches )EH( + +The matching rules described above stop after the first match, +regardless of whether that match provides values for all three +parameters \201filter, columns, and format\202. In cases where a )BD(view)ES( +or )BD(file)ES( match does not provide all three values, it is possible +that a template match might do so. With regard to the example View +database above, the x1 View provides only a filter, while omitting +both the format and columns values. But note that the final rows in +the database could provide the values via a template match on the +filename. This sort of multiple matching is especially valuable in +order to provide "global" values to several Views. + +)0 P(Obviously, multiple matching might not be wanted in every +case. Therefore, we support both multiple matching and single matching +according to the value of the FUN_VIEWMATCH environment variable. If +the FUN_VIEWMATCH environment variable exists and if its value begins +with "s", then a single match is used and missing parameters are not +filled in with subsequent template matches on the file name. That is, +matching rules above are followed exactly as explained above. If the +value of this environment variable begins with "m" \201or does not exist\202, +then multiple matches are used to try to fill in missing parameters. +In this case, template matching always takes place and missing values are +taken from these template matches. + +)0 P(Thus, in the example above, the View specification: +) 1 14 PR( fundisp v:x1)RP( +will take the file name and filter value from the x1 View: +) 1 70 PR( x1 ${HOME}/data/snr.ev cir 512 512 .1)RP( +The column value then will be taken from the "*.ev" file template match +against the x1 file name: +) 1 52 PR( *.ev x y pi pha)RP( +Note once again that order is important: missing values are taken in the +order in which the template matches are processed. + +)0 2 84 H(View)WB 154 Sn( Lists: Applying a View to Any File)EH( + +)0 P(It is possible to apply a named View, or even several Views, to any +data file by appending a )BD(viewlist)ES( immediately after the standard "v:" +prefix. A viewlist takes the form: +) 1 15 PR( :v1,v2,...vn:)RP( +where v1, v2, etc. are named Views. The two ":" colon characters surrounding +the list are required. Thus, the syntax for applying a viewlist to a file is: +) 1 34 PR( v::view1,view2,...viewn:filename)RP( +Note that the name after the last ":" is assumed to be a file; it is +not permissible \201or sensible\202 to use a View name. + +)0 P(For example, the View specification: +) 1 19 PR( fundisp v::x2:foo)RP( +applies the x2 View to the file foo \201even if there is a View named foo\202 +and \201in using our example database\202 is equivalent to: +) 1 45 PR( ./fundisp foo'[cir 512 512 .1] "x y pi pha")RP( +The same command can be effected using a list of Views: +) 1 23 PR( fundisp v::x1,x1a:foo)RP( + +)0 P(What happens if a viewlist is used and the file also matches a +template? Consider, for example, this View specification: +) 1 23 PR( fundisp v::x2:foo.fit)RP( +Here, the x2 View will supply filter and column values, while the +template *.fit can also supply \201different\202 filter and column +values. In this case, the explicitly specified Views of the viewlist +trump the matched view values. + +)0 P(On the other hand, if a file template match can supply a View value +that is not supplied by the viewlist, then that value will be taken +from the file template match. For example: +) 1 24 PR( fundisp v::x2:foo.fits)RP( +does not explicitly supply a format value, but the file match on *.fits +can and does. You can avoid supplying missing values using file template +matching by replacing the first ":" with a "-" in a viewlist +specification: +) 1 24 PR( fundisp v:-x2:foo.fits)RP( +The use of ":+" to explicitly allow file template matching is also +supported, but is the same as the default case. Note that the nuances +of viewlist support are subject to change as our experience and +understanding grow. + +)0 2 85 H(Overriding)WB 155 Sn( Values Associated with a View)EH( + +)0 P(To override values associated with a View, simply supply the override +values in the correct place on the command line. Thus, given +the example database described above, the command: +) 1 14 PR( fundisp v:x3)RP( +specifies that the View named x3, along with its file name and +associated parameters, be processed as the input file to fundisp in +this way: +) 1 73 PR( fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha")RP( +To override one or more of these values, simply specify a new value +for the format, filter, or columns. For example, if your input View file +contains a filter, then the View will use that filter as an override +of the View filter: +) 1 31 PR( fundisp v:x3'[cir 400 400 3]')RP( +will use the columns and format of the x3 View but not the x3 filter. Further +examples are: +) 2 67 PR( fundisp v:x3 "x y dx dy" # activate a different set of columns + fundisp -f "I=%3d" v:x3 # use a different format statement)RP( + +)0 P(Note that extension names, extension index values, and other +non-filter specifications )BD(do not)ES( override the View +filter. Thus: +) 1 22 PR( fundisp v:foo.fit[3])RP( +will still use the filter associated with the .fit template \201see above\202, since +the "3" is an extension index, not a filter. + +)0 2 86 H(Environment)WB 156 Sn( Variables)EH( + +The following environment variables are used by Funtools Views: +)0 DL()0 DT()BD(FUN_VIEWNAME)ES( +)DD( The )BD(FUN_VIEWNAME)ES( environment variable specifies the +name and location of the View database file. If not present, the +files ./.funtools.vu and $HOME/.funtools.vu are searched for, in +that order. + +)0 DT()BD(FUN_VIEWMATCH)ES( +)DD( The )BD(FUN_VIEWMATCH)ES( environment variable specifies whether a +single match or multiple match algorithm is used to locate parameter +values. If the value of this environment variable begins with "s", +then a single match is used and missing parameters are not filled in +with subsequent template matches on the file name. If the value begins +with "m", then multiple matches are used to try to fill in missing +parameters. The default is to use multiple matches.)LD( + +)0 2 87 H(Restrictions)WB 157 Sn( and Problems)EH( + +Support for overriding a filter \201while not overriding extension names, +extension indexes, etc.\202 requires that we can sense the presence of a +filter in a bracket specification. It is unclear yet whether our +algorithm is perfect. + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 88 H(Last)WB 158 Sn( updated: August 3, 2007)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (filters.html) D +/Ti (Table Filtering) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 52 Sn( + + +)0 2 89 H(Funfilters:)WB 165 Sn()WB 159 Sn( Filtering Rows in a Table)EA()EH( + + +)0 2 90 H(Summary)WB 166 Sn()EH( +)0 P(This document contains a summary of the user interface for +filtering rows in binary tables. + + +)0 2 91 H(Description)WB 167 Sn()EH( +)0 P(Table filtering allows a program to select rows from an table \201e.g., +X-ray event list\202 by checking each row against one or more expressions +involving the columns in the table. When a table is filtered, only +valid rows satisfying these expressions are passed through for processing. + +)0 P(A filter expression is specified using bracket notation appended to +the filename of the data being processed: +) 1 23 PR( foo.fits[pha==1&)SY(\160)ES(==2])RP( +It is also possible to put region specification inside a file and +then pass the filename in bracket notation: +) 1 19 PR( foo.fits[@my.reg])RP( +Filters must be placed after the extension and image section +information, when such information is present. The correct order is: +)UL()-1 LI( file[fileinfo,sectioninfo][filters] +)-1 LI( file[fileinfo,sectioninfo,filters])LU( +where: +)UL()-1 LI( )BD(file)ES( is the Funtools file name +)-1 LI( )BD(fileinfo)ES( is an ARRAY, EVENT, FITS extension, or FITS index +)-1 LI( )BD(sectioninfo)ES( is the image section to extract +)-1 LI( )BD(filters)ES( are spatial region and table \201row\202 filters to apply)LU( +See )0 42 1 A(Funtools Files)42 0 TN TL()Ec /AF f D( for more information +on file and image section specifications. + +)0 2 92 H(Filter)WB 168 Sn( Expressions)EH( + +)0 P(Table filtering can be performed on columns of data in a FITS +binary table or a raw event file. Table filtering is accomplished by +means of )BD(table filter specifications)ES(. An table filter +specification consists of one or more )BD(filter expressions)ES( Filter +specifications also can contain comments and local/global processing +directives. + +)0 P(More specifically, a filter specification consist of one or more lines +containing: +) 13 75 PR( # comment until end of line + # include the following file in the table descriptor + @file + # each row expression can contain filters separated by operators + [filter_expression] BOOLOP [filter_expression2], ... + # each row expression can contain filters separated by the comma operator + [filter_expression1], [filter_expression2], ... + # the special row# keyword allows a range of rows to be processed + row#=m:n + # or a single row + row#=m + # regions are supported -- but are described elsewhere + [spatial_region_expression])RP( + +)0 P(A single filter expression consists of an arithmetic, logical, or +other operations involving one or more column values from a +table. Columns can be compared to other columns, to header values, +or to numeric constants. Standard math functions can be applied to +columns. Separate filter expressions can be combined using boolean operators. +Standard C semantics can be used when constructing expressions, with +the usual precedence and associativity rules holding sway: +) 15 55 PR( Operator Associativity + -------- ------------- + \201\202 left to right + !! \201logical not\202 right to left + ! \201bitwise not\202 - \201unary minus\202 right to left + * / left to right + + - left to right + < <= > >= left to right + == != left to right + & \201bitwise and\202 left to right + ^ \201bitwise exclusive or\202 left to right + | \201bitwise inclusive or\202 left to right + && \201logical and\202 left to right + || \201logical or\202 left to right + = right to left)RP( +For example, if energy and pha are columns in a table, +then the following are valid expressions: +) 4 24 PR( pha>1 + energy == pha + \201pha>1\202 && \201energy<=2\202 + max\201pha,energy\202>=2.5)RP( + +)0 P(Comparison values can be integers or floats. Integer comparison values can be +specified in decimal, octal \201using '0' as prefix\202, hex \201using '0x' as prefix\202 +or binary \201using '0b' as prefix\202. Thus, the following all specify the same +comparison test of a status mask: +) 4 40 PR( \201status & 15\202 == 8 # decimal + \201status & 017\202 == 010 # octal + \201status & 0xf\202 == 0x8 # hex + \201status & 0b1111\202 == 0b1000 # binary)RP( +)0 P(The special keyword row# allows you to process a range of rows. +When row# is specified, the filter code skips to the designated +row and only processes the specified number of rows. The +"*" character can be utilized as the high limit value to denote +processing of the remaining rows. Thus: +) 1 14 PR( row#=100:109)RP( +processes 10 rows, starting with row 100 \201counting from 1\202, +while: +) 1 12 PR( row#=100:*)RP( +specifies that all but the first 99 rows are to be processed. + +)0 P(Spatial region filtering allows a program to select regions of an +image or rows of a table \201e.g., X-ray events\202 using simple geometric +shapes and boolean combinations of shapes. For a complete description +of regions, see )0 54 1 A(Spatial Region Filtering)54 0 TN TL()Ec /AF f D(. + +)0 2 93 H(Separators)WB 169 Sn()WB 160 Sn( Also Are Operators)EA()EH( +)0 P(As mentioned previously, multiple filter expressions can be specified +in a filter descriptor, separated by commas or new-lines. +When such a comma or new-line separator is used, the boolean AND operator +is automatically generated in its place. Thus and expression such as: +) 1 15 PR( pha==1,pi=2:4)RP( +is equivalent to: +) 1 26 PR( \201pha==1\202 && \201pi>=2&)SY(\160)ES(<=4\202)RP( +)0 P([Note that the behavior of separators is different for filter expressions +and spatial region expressions. The former uses AND as the operator, while +the latter user OR. See +)0 60 1 A(Combining Region and Table Filters)60 0 TN TL()Ec /AF f D( +for more information about these conventions and how they are treated +when combined.] + +)0 2 94 H(Range)WB 170 Sn()WB 161 Sn( Lists)EA()EH()0 P( +)0 P(Aside from the standard C syntax, filter expressions can make use of +IRAF-style )BD(range lists)ES( which specify a range of values. The +syntax requires that the column name be followed by an '=' sign, which +is followed by one or more comma-delimited range expressions of the form: +) 4 52 PR( col = vv # col == vv in range + col = :vv # col <= vv in range + col = vv: # col >= vv in range + col = vv1:vv2 # vv1 <= col <= vv2 in range)RP( +The vv's above must be numeric constants; the right hand side of a +range list cannot contain a column name or header value. +)0 P(Note that, unlike an ordinary comma separator, the comma separator used +between two or more range expressions denotes OR. Thus, when two or +more range expressions are combined with a comma separator, the resulting +expression is a shortcut for more complicated boolean logic. For example: +) 1 18 PR( col = :3,6:8,10:)RP( +is equivalent to: +) 1 47 PR( \201col<=3\202 || \201col>=6 && col <=8\202 || \201col >=10\202)RP( +Note also that the single-valued rangelist: +) 1 11 PR( col = val)RP( +is equivalent to the C-based filter expression: +) 1 12 PR( col == val)RP( +assuming, of course, that val is a numeric constant. + +)0 2 95 H(Math)WB 171 Sn()WB 162 Sn( Operations and Functions)EA()EH()0 P( +)0 P(It is permissible to specify C math functions as part of the filter syntax. +When the filter parser recognizes a function call, it automatically +includes the math.h and links in the C math library. Thus, it is +possible to filter rows by expressions such as these: +)UL()-1 LI(\201pi+pha\202>\2012+log\201pi\202-pha\202 +)-1 LI( min\201pi,pha\202*14>x +)-1 LI( max\201pi,pha\202==\201pi+1\202 +)-1 LI( feq\201pi,pha\202 +)-1 LI( div\201pi,pha\202>0)LU( +The function feq\201a,b\202 returns true \2011\202 if the difference between a and b +\201taken as double precision values\202 is less than approximately 10E-15. +The function div\201a,b\202 divides a by b, but returns NaN \201not a number\202 +if b is 0. It is a safe way to avoid floating point errors when +dividing one column by another. + +)0 2 96 H(Include)WB 172 Sn()WB 163 Sn( Files)EA()EH()0 P( +)0 P(The special )BD(@filename)ES( directive specifies an include file +containing filter expressions. This file is processed as part of +the overall filter descriptor: +) 1 23 PR( foo.fits[pha==1,@foo])RP( + +)0 2 97 H(Header)WB 173 Sn()WB 164 Sn( Parameters)EA()EH()0 P( +)0 P(The filter syntax supports comparison between a column value and a +header parameter value of a FITS binary tables \201raw event files have no +such header\202. The header parameters can be taken from the binary +table header or the primary header. For example, assuming there is a +header value MEAN_PHA in one of these headers, you can select photons +having exactly this value using: + +)UL()-1 LI( pha==MEAN_PHA)LU( + +)0 2 98 H(Examples)WB 174 Sn()EH( +)0 P(Table filtering is more easily described by means of examples. +Consider data containing the following table structure: +)UL()-1 LI( double TIME +)-1 LI( int X +)-1 LI( int Y +)-1 LI( short PI +)-1 LI( short PHA +)-1 LI( int DX +)-1 LI( int DY)LU( + +)0 P(Tables can be filtered on these columns using IRAF/QPOE range syntax or +any valid C syntax. The following examples illustrate the possibilities: +)0 DL( +)0 P()0 DT( pha=10 +)0 DT( pha==10 +)DD( select rows whose pha value is exactly 10 + +)0 P()0 DT( pha=10:50 +)DD( select rows whose pha value is in the range of 10 to 50 + +)0 P()0 DT( pha=10:50,100 +)DD( select rows whose pha value is in the range of 10 to 50 or is +equal to 100 + +)0 P()0 DT( pha>=10 && pha<=50 +)DD( select rows whose pha value is in the range of 10 to 50 + +)0 P()0 DT( pi=1,2&&pha>3 +)DD( select rows whose pha value is 1 or 2 and whose pi value is 3 + +)0 P()0 DT( pi=1,2 || pha>3 +)DD( select rows whose pha value is 1 or 2 or whose pi value is 3 + +)0 P()0 DT( pha==pi+1 +)DD( select rows whose pha value is 1 less than the pi value + +)0 P()0 DT( \201pha==pi+1\202 && \201time>50000.0\202 +)DD( select rows whose pha value is 1 less than the pi value +and whose time value is greater than 50000 + +)0 P()0 DT(\201pi+pha\202>20 +)DD( select rows in which the sum of the pi and pha values is greater +than 20 + +)0 P()0 DT( pi%2==1 +)DD( select rows in which the pi value is odd)LD( + +)0 P(Currently, integer range list limits cannot be specified in binary +notation \201use decimal, hex, or octal instead\202. Please contact us if +this is a problem. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 99 H(Last)WB 175 Sn( updated: November 17, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (idx.html) D +/Ti (Table Filtering with Indexes) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 53 Sn( + + +)0 2 100 H(Funidx:)WB 177 Sn()WB 176 Sn( Using Indexes to Filter Rows in a Table)EA()EH( + + +)0 2 101 H(Summary)WB 178 Sn()EH( +)0 P(This document contains a summary of the user interface for +filtering rows in binary tables with indexes. + + +)0 2 102 H(Description)WB 179 Sn()EH( +)0 P(Funtools )0 52 1 A(Table Filtering)52 0 TN TL()Ec /AF f D( allows rows in a +table to be selected based on the values of one or more columns in the +row. Because the actual filter code is compiled on the fly, it is very +efficient. However, for very large files \201hundreds of Mb or larger\202, +evaluating the filter expression on each row can take a long time. Therefore, +funtools supports index files for columns, which are used automatically during +filtering to reduce dramatically the number of row evaluations performed. +The speed increase for indexed filtering can be an order of magnitude or +more, depending on the size of the file. + +)0 P(The )0 11 1 A(funindex)11 0 TN TL()Ec /AF f D( program creates an +index on one or more columns in a binary table. For example, to create an index +for the column pi in the file huge.fits, use: +) 1 23 PR( funindex huge.fits pi)RP( +This will create an index named huge_pi.idx. + +)0 P(When a filter expression is initialized for row evaluation, funtools +looks for an index file for each column in the filter expression. If +found, and if the file modification date of the index file is later +than that of the data file, then the index will be used to reduce the +number of rows that are evaluated in the filter. When +)0 54 1 A(Spatial Region Filtering)54 0 TN TL()Ec /AF f D( is part of the +expression, the columns associated with the region are checked for index +files. + +)0 P(If an index file is not available for a given column, then in general, +all rows must be checked when that column is part of a filter +expression. This is not true, however, when a non-indexed column is +part of an AND expression. In this case, only the rows that pass the +other part of the AND expression need to be checked. Thus, in some cases, +filtering speed can increase significantly even if all columns are not +indexed. + +)0 P(Also note that certain types of filter expression syntax cannot make +use of indices. For example, calling functions with column names as +arguments implies that all rows must be checked against the function +value. Once again, however, if this function is part of an AND +expression, then a significant improvement in speed still is possible +if the other part of the AND expression is indexed. + +)0 P(For example, note below the dramatic speedup in searching a 1 Gb +file using an AND filter, even when one of the columns \201pha\202 has no +index: + +) 49 69 PR( time fundisp \200 + huge.fits'[idx_activate=0,idx_debug=1,pha=2348&&cir 4000 4000 1]' \200 + "x y pha" + x y pha + ---------- ----------- ---------- + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 42.36u 13.07s 6:42.89 13.7% + + time fundisp \200 + huge.fits'[idx_activate=1,idx_debug=1,pha=2348&&cir 4000 4000 1]' \200 + "x y pha" + x y pha + ---------- ----------- ---------- + idxeq: [INDEF] + idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] + idxand\2011\202: INDEF [IDX_OR_SORT] )WR( + idxall\2011\202: [IDX_OR_SORT] + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 1.55u 0.37s 1:19.80 2.4%)RP( + +When all columns are indexed, the increase in speed can be even more dramatic: +) 50 67 PR( time fundisp \200 + huge.fits'[idx_activate=0,idx_debug=1,pi=770&&cir 4000 4000 1]' \200 + "x y pi" + x y pi + ---------- ----------- ---------- + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 42.60u 12.63s 7:28.63 12.3% + + time fundisp \200 + huge.fits'[idx_activate=1,idx_debug=1,pi=770&&cir 4000 4000 1]' \200 + "x y pi" + x y pi + ---------- ----------- ---------- + idxeq: pi start=9473025,stop=9492240 => pi[ROW 9473025:9492240] + idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] + idxor sort/merge: pi[ROW 9473025:9492240] [IDX_OR_SORT] )WR( + idxmerge\2015\202: [IDX_OR_SORT] pi[ROW] + idxall\2011\202: [IDX_OR_SORT] + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 1.67u 0.30s 0:24.76 7.9%)RP( + +)0 P(The miracle of indexed filtering \201and indeed, of any indexing\202 is the +speed of the binary search on the index, which is of order log2\201n\202 +instead of n. \201The funtools binary search method is taken from +http://www.tbray.org/ongoing/When/200x/2003/03/22/Binary, to whom +grateful acknowledgement is made.\202 This means that the larger the +file, the better the performance. Conversely, it also means that for +small files, using an index \201and the overhead involved\202 can slow +filtering down somewhat. Our tests indicate that on a file containing +a few tens of thousands of rows, indexed filtering can be 10 to 20 +percent slower than non-indexed filtering. Of course, your mileage +will vary with conditions \201disk access speed, amount of available +memory, process load, etc.\202 + +)0 P(Any problem encountered during index processing will result in +indexing being turned off, and replaced by filtering all rows. You can turn +filtering off manually by setting the idx_activate variable to 0 \201in a filter +expression\202 or the FILTER_IDX_ACTIVATE environment variable to 0 \201in the global +environment\202. Debugging output showing how the indexes are being processed can +be displayed to stderr by setting the idx_debug variable to 1 \201in a filter +expression\202 or the FILTER_IDX_DEBUG environment variable to 1 \201in the global +environment\202. + +)0 P(Currently, indexed filtering only works with FITS binary tables and raw +event files. It does not work with text files. This restriction might be +removed in a future release. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 103 H(Last)WB 180 Sn( updated: August 3, 2007)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (regions.html) D +/Ti (Spatial Region Filtering) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 54 Sn( + + +)0 2 104 H(Regions:)WB 182 Sn()WB 181 Sn( Spatial Region Filtering)EA()EH( + + +)0 2 105 H(Summary)WB 183 Sn()EH( +)0 P(This document contains a summary of the user interface for spatial +region filtering images and tables. + + +)0 2 106 H(Description)WB 184 Sn()EH( +)0 P(Spatial region filtering allows a program to select regions of an +image or rows of a table \201e.g., X-ray events\202 to process using +simple geometric shapes and boolean combinations of shapes. When an +image is filtered, only pixels found within these shapes are +processed. When a table is filtered, only rows found within these +shapes are processed. + +)0 P(Spatial region filtering for images and tables is accomplished by +means of )BD(region specifications)ES(. A region specification +consists of one or more )BD(region expressions)ES(, which are geometric +shapes,combined according to the rules of boolean algebra. Region +specifications also can contain comments and local/global processing +directives. + +)0 P(Typically, region specifications are specified using bracket notation +appended to the filename of the data being processed: +) 1 31 PR( foo.fits[circle\201512,512,100\202])RP( +It is also possible to put region specification inside a file and +then pass the filename in bracket notation: +) 1 19 PR( foo.fits[@my.reg])RP( + +)0 P(When region filters are passed in bracket notation in this manner, the +filtering is set up automatically when the file is opened and all +processing occurs through the filter. Programs also can use the filter +library API to open filters explicitly. + +)0 2 107 H(Region)WB 185 Sn( Expressions)EH( + +More specifically, region specifications consist of one or more lines +containing: +) 9 68 PR( # comment until end of line + global keyword=value keyword=value ... # set global value\201s\202 + # include the following file in the region descriptor + @file + # use the FITS image as a mask \201cannot be used with other regions\202 + @fitsimage + # each region expression contains shapes separated by operators + [region_expression1], [region_expression2], ... + [region_expression], [region_expression], ...)RP( + +)0 P(A single region expression consists of: +) 8 72 PR( # parens and commas are optional, as is the + sign + [+-]shape\201num , num , ...\202 OP1 shape num num num OP2 shape ... + +e.g.: + + \201[+-]shape\201num , num , ...\202 && shape num num || shape\201num, num\202 + # a comment can come after a region -- reserved for local properties + [+-]shape\201num , num , ...\202 # local properties go here, e.g. color=red)RP( + +)0 P(Thus, a region descriptor consists of one or more )BD(region +expressions)ES( or )BD(regions)ES(, separated by comas, new-lines, or +semi-colons. Each )BD(region)ES( consists of one or more )BD(geometric +shapes)ES( combined using standard boolean operation. Several types +of shapes are supported, including: + +) 11 57 PR( shape: arguments: + ----- ---------------------------------------- + ANNULUS xcenter ycenter inner_radius outer_radius + BOX xcenter ycenter xwidth yheight \201angle\202 + CIRCLE xcenter ycenter radius + ELLIPSE xcenter ycenter xwidth yheight \201angle\202 + FIELD none + LINE x1 y1 x2 y2 + PIE xcenter ycenter angle1 angle2 + POINT x1 y1 + POLYGON x1 y1 x2 y2 ... xn yn)RP( + +)0 P(In addition, the following regions accept )BD(accelerator)ES( syntax: + +) 13 73 PR( shape arguments + ----- ------------------------------------------ + ANNULUS xcenter ycenter radius1 radius2 ... radiusn + ANNULUS xcenter ycenter inner_radius outer_radius n=[number] + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn \201angle\202 + BOX xcenter ycenter xwlo yhlo xwhi yhhi n=[number] \201angle\202 + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn \201angle\202 + ELLIPSE xcenter ycenter xwlo yhlo xwhi yhhi n=[number] \201angle\202 + PIE xcenter ycenter angle1 angle2 \201angle3\202 \201angle4\202 \201angle5\202 ... + PIE xcenter ycenter angle1 angle2 \201n=[number]\202 + POINT x1 y1 x2 y2 ... xn yn)RP( +Note that the circle accelerators are simply aliases for the annulus +accelerators. See )0 55 1 A(region geometry)55 0 TN TL()Ec /AF f D( +for more information about accelerators. + +)0 P(Finally, the following are combinations of pie with different shapes +\201called "panda" for "Pie AND Annulus"\202 allow for easy specification of +radial sections: + +) 6 75 PR( shape: arguments: + ----- --------- + PANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + BPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad \201ang\202 # box + EPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad \201ang\202 # ellipse)RP( + +The panda and cpanda specify combinations of annulus and circle with pie, +respectively and give identical results. The bpanda combines box and pie, +while epanda combines ellipse and pie. +See )0 55 1 A(region geometry)55 0 TN TL()Ec /AF f D( +for more information about pandas. + +)0 P(The following "shapes" are ignored by funtools \201generated by ds9\202: +) 8 62 PR( shape: arguments: + ----- --------- + PROJECTION x1 y1 x2 y2 width # NB: ignored by funtools + RULER x1 y1 x2 y2 # NB: ignored by funtools + TEXT x y # NB: ignored by funtools + GRID # NB: ignored by funtools + TILE # NB: ignored by funtools + COMPASS # NB: ignored by funtools)RP( + +)0 P(All arguments to regions are real values; integer values are +automatically converted to real where necessary. All angles are in +degrees and run from the positive image x-axis to the positive image +y-axis. If a rotation angle is part of the associated WCS header, that +angle is added implicitly as well. + +)0 P(Note that 3-letter abbreviations are supported for all shapes, so that +you can specify "circle" or "cir". + +)0 P()0 2 108 H(Columns)WB 186 Sn( Used in Region Filtering)EH( +)0 P(By default, the x,y values in a region expression refer to the two +"image binning" columns, i.e. the columns that would be used to +bin the data into an image. For images, these are just the 2 dimensions +of the image. For tables, these usually default to x and y but +can be changed as required. For example, in Funtools, new binning +columns are specified using a bincols=\201col1,col2\202 statement within +the bracket string on the command line. +)0 P(Alternate columns for region filtering can be specified by the syntax: +) 1 25 PR( \201col1,col2\202=region\201...\202)RP( +e.g.: +) 3 34 PR( \201X,Y\202=annulus\201x,y,ri,ro\202 + \201PHA,PI\202=circle\201x,y,r\202 + \201DX,DY\202=ellipse\201x,y,a,b[,angle]\202)RP( + +)0 P()0 2 109 H(Region)WB 187 Sn( Algebra)EH( + +\201See also )0 56 1 A(Region Algebra)56 0 TN TL()Ec /AF f D( for more complete +information.\202 + +)0 P(Region shapes can be combined together using Boolean operators: +) 6 72 PR( Symbol Operation Use + -------- --------- ----------------------------------- + ! not Exclude this shape from this region + & or && and Include only the overlap of these shapes + | or || inclusive or Include all of both shapes + ^ exclusive or Include both shapes except their overlap)RP( +Note that the !region syntax must be combined with another region in order +that we be able to assign a region id properly. That is, +) 1 21 PR( !circle\201512,512,10\202)RP( +is not a legal region because there is no valid region id to work with. +To get the full field without a circle, combine the above with field\201\202, +as in: +) 1 32 PR( field\201\202 && !circle\201512,512,10\202)RP( + +)0 2 110 H()WB 188 Sn( Region Separators Also Are Operators)EH( + +)0 P(As mentioned previously, multiple region expressions can be specified +in a region descriptor, separated by commas, new-lines, or +semi-colons. When such a separator is used, the boolean OR operator +is automatically generated in its place but, unlike explicit use of +the OR operator, the region ID is incremented \201starting from 1\202. + +)0 P(For example, the two shapes specified in this example are given the +same region value: +) 1 50 PR( foo.fits[circle\201512,512,10\202||circle\201400,400,20\202])RP( +On the other hand, the two shapes defined in the following example are +given different region values: +) 1 49 PR( foo.fits[circle\201512,512,10\202,circle\201400,400,20\202])RP( + +)0 P(Of course these two examples will both mask the same table rows or +pixels. However, in programs that distinguish region id's \201such as +)0 5 1 A(funcnts)5 0 TN TL()Ec /AF f D( \202, they will act +differently. The explicit OR operator will result in one region +expression consisting of two shapes having the same region id and +funcnts will report a single region. The comma operator will cause +funcnts to report two region expressions, each with one shape, in +its output. + +)0 P(In general, commas are used to separate region expressions entered +in bracket notation on the command line: +) 2 57 PR( # regions are added to the filename in bracket notation + foo.fits[circle\201512,512,100\202,circle\201400,400,20\202])RP( +New-lines are used to separate region +expressions in a file: +) 4 58 PR( # regions usually are separated by new-lines in a file + # use @filename to include this file on the command line + circle\201512,512,100\202 + circle\201400,400,20\202)RP( +Semi-colons are provided for backward compatibility with the original +IRAF/PROS implementation and can be used in either case. + +)0 P(If a pixel is covered by two different regions expressions, it is +given the mask value of the )BD(first)ES( region that contains that +pixel. That is, successive regions )BD(do not)ES( overwrite previous +regions in the mask, as was the case with the original PROS regions. +In this way, an individual pixel is covered by one and only one +region. This means that one must sometimes be careful about the order +in which regions are defined. If region N is fully contained within +region M, then N should be defined )BD(before)ES( M, or else it will be +"covered up" by the latter. + +)0 2 111 H(Region)WB 189 Sn( Exclusion)EH( +)0 P(Shapes also can be globally excluded from all the region specifiers in +a region descriptor by using a minus sign before a region: + +) 4 73 PR( operator arguments: + -------- ----------- + - Globally exclude the region expression following '-' sign + from ALL regions specified in this file)RP( +The global exclude region can be used by itself; in such a case, field\201\202 is +implied. + +)0 P(A global exclude differs from the local exclude \201i.e. a shape prefixed +by the logical not "!" symbol\202 in that global excludes are logically +performed last, so that no region will contain pixels from a globally +excluded shape. A local exclude is used in a boolean expression with +an include shape, and only excludes pixels from that include shape. +Global excludes cannot be used in boolean expressions. + +)0 2 112 H(Include)WB 190 Sn( Files)EH( + +)0 P(The )BD(@filename)ES( directive specifies an include file +containing region expressions. This file is processed as part of +the overall region descriptor: +) 1 35 PR( foo.fits[circle\201512,512,10\202,@foo])RP( +A filter include file simply includes text without changing the state +of the filter. It therefore can be used in expression. That is, if the +file foo1 contains "pi==1" and foo2 contains "pha==2" then +the following expressions are equivalent: +) 3 57 PR( "[@foo1&&@foo2]" is equivalent to "[pi==1&&pha==2]" + "[pha==1||@foo2]" is equivalent to "[pi==1||pha==2]" + "[@foo1,@foo2]" is equivalent to "[pi==1,pha==2]")RP( +Be careful that you specify evaluation order properly using +parenthesis, especially if the include file contains multiple +filter statements. For example, consider a file containing two +regions such as: +) 2 19 PR( circle 512 512 10 + circle 520 520 10)RP( +If you want to include only events \201or pixels\202 that are in these regions +and have a pi value of 4, then the correct syntax is: +) 1 17 PR( pi==4&&\201@foo\202)RP( +since this is equivalent to: +) 1 53 PR( pi==4 && \201circle 512 512 10 || circle 520 520 10\202)RP( +If you leave out the parenthesis, you are filtering this statement: +) 1 52 PR( pi==4 && circle 512 512 10 || circle 520 520 10\202)RP( +which is equivalent to: +) 1 54 PR( \201pi==4 && circle 512 512 10\202 || circle 520 520 10\202)RP( +The latter syntax only applies the pi test to the first region. + +)0 P(For image-style filtering, the )BD(@filename)ES( can specify an 8-bit +or 16-bit FITS image. In this case, the pixel values in the mask image +are used as the region mask. The valid pixels in the mask must have +positive values. Zero values are excluded from the mask and negative +values are not allowed. Moreover, the region id value is taken as +the image pixel value and the total number of regions is taken to be +the highest pixel value. The dimensions of the image mask must be less +than or equal to the image dimensions of the data. The mask will be +replicated as needed to match the size of the image. \201Thus, best +results are obtained when the data dimensions are an even multiple of +the mask dimensions.\202 + +)0 P(An image mask can be used in any image filtering operation, regardless +of whether the data is of type image or table. For example, the +)0 5 1 A(funcnts)5 0 TN TL()Ec /AF f D( \202 +program performs image filtering on images or tables, and so +FITS image masks are valid input for either type of data in this +program.. An image mask cannot be used in a program such as +)0 7 1 A(fundisp)7 0 TN TL()Ec /AF f D( \202 +when the input data is a table, because fundisp displays +rows of a table and processes these rows using event-style filtering. + +)0 2 113 H(Global)WB 191 Sn( and Local Properties of Regions)EH( + +)0 P(The ds9 image display program describes a host of properties such as +color, font, fix/free state, etc. Such properties can be specified +globally \201for all regions\202 or locally \201for an individual region\202. +The )BD(global)ES( keyword specifies properties and qualifiers for all +regions, while local properties are specified in comments on the same +line as the region: +) 4 30 PR( global color=red + circle\20110,10,2\202 + circle\20120,20,3\202 # color=blue + circle\20130,30,4\202)RP( +The first and third circles will be red, which the second circle will +be blue. Note that funtools currently ignores region properties, as +they are used in display only. + +)0 2 114 H()WB 192 Sn( Coordinate Systems)EH( + +For each region, it is important to specify the coordinate system +used to interpret the region, i.e., to set the context in which position and +size values are interpreted. For this purpose, the following keywords +are recognized: + +) 12 68 PR( name description + ---- ------------------------------------------ + PHYSICAL pixel coords of original file using LTM/LTV + IMAGE pixel coords of current file + FK4, B1950 sky coordinate systems + FK5, J2000 sky coordinate systems + GALACTIC sky coordinate systems + ECLIPTIC sky coordinate systems + ICRS currently same as J2000 + LINEAR linear wcs as defined in file + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV)RP( + +)0 P()0 2 115 H(Specifying)WB 193 Sn( Positions, Sizes, and Angles)EH( + +The arguments to region shapes can be floats or integers describing +positions and sizes. They can be specified as pure numbers or using +explicit formatting directives: + +) 21 57 PR( position arguments description + ------------------ ------------------------------ + [num] context-dependent \201see below\202 + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + [num]:[num]:[num] hms for 'odd' position arguments + [num]:[num]:[num] dms for 'even' position arguments + [num]h[num]m[num]s explicit hms + [num]d[num]m[num]s explicit dms + + size arguments description + -------------- ----------- + [num] context-dependent \201see below\202 + [num]" arc seconds + [num]' arc minutes + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels)RP( + + +When a "pure number" \201i.e. one without a format directive such as 'd' +for 'degrees'\202 is specified, its interpretation depends on the context +defined by the 'coordsys' keyword. In general, the rule is: + +)0 P()BD(All pure numbers have implied units corresponding to the current +coordinate system.)ES( + +)0 P(If no such system is explicitly specified, the default system is +implicitly assumed to be PHYSICAL. + +)0 P(In practice this means that for IMAGE and PHYSICAL systems, pure +numbers are pixels. Otherwise, for all systems other than linear, +pure numbers are degrees. For LINEAR systems, pure numbers are in the +units of the linear system. This rule covers both positions and +sizes. + +)0 P(The input values to each shape can be specified in several coordinate +systems including: + +) 12 68 PR( name description + ---- ---------------------------- + IMAGE pixel coords of current file + LINEAR linear wcs as defined in file + FK4, B1950 various sky coordinate systems + FK5, J2000 + GALACTIC + ECLIPTIC + ICRS + PHYSICAL pixel coords of original file using LTM/LTV + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV)RP( + +)0 P(If no coordinate system is specified, PHYSICAL is assumed. PHYSICAL or +a World Coordinate System such as J2000 is preferred and most general. +The coordinate system specifier should appear at the beginning of the +region description, on a separate line \201in a file\202, or followed by a +new-line or semicolon; e.g., + +) 2 26 PR( global coordsys physical + circle 6500 9320 200)RP( + +The use of celestial input units automatically implies WORLD +coordinates of the reference image. Thus, if the world coordinate +system of the reference image is J2000, then + +) 1 27 PR( circle 10:10:0 20:22:0 3')RP( + +is equivalent to: + +) 1 35 PR( circle 10:10:0 20:22:0 3' # j2000)RP()RP( +Note that by using units as described above, you may mix coordinate +systems within a region specifier; e.g., + +) 1 32 PR( circle 6500 9320 3' # physical)RP( + +)0 P(Note that, for regions which accept a rotation angle: + +) 2 29 PR(ellipse \201x, y, r1, r2, angle\202 +box\201x, y, w, h, angle\202)RP( + +the angle is relative to the specified coordinate system. In +particular, if the region is specified in WCS coordinates, the angle +is related to the WCS system, not x/y image coordinate axis. For WCS +systems with no rotation, this obviously is not an issue. However, +some images do define an implicit rotation \201e.g., by using a non-zero +CROTA value in the WCS parameters\202 and for these images, the angle +will be relative to the WCS axes. In such case, a region specification +such as: + +) 1 54 PR(fk4;ellipse\20122:59:43.985, +58:45:26.92,320", 160", 30\202)RP( + +will not, in general, be the same region specified as: + +) 1 38 PR(physical;ellipse\201465, 578, 40, 20, 30\202)RP( + +even when positions and sizes match. The angle is relative to WCS axes +in the first case, and relative to physical x,y axes in the second. + + +)0 P(More detailed descriptions are available for: +)0 55 1 A(Region Geometry)55 0 TN TL()Ec /AF f D(, +)0 56 1 A(Region Algebra)56 0 TN TL()Ec /AF f D(, +)0 57 1 A(Region Coordinates)57 0 TN TL()Ec /AF f D(, and +)0 58 1 A(Region Boundaries)58 0 TN TL()Ec /AF f D(. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 116 H(Last)WB 194 Sn( updated: November 17, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (reggeometry.html) D +/Ti (Region Geometry) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 55 Sn( + + +)0 2 117 H(RegGeometry:)WB 196 Sn()WB 195 Sn( Geometric Shapes in Spatial Region Filtering)EA()EH( + + +)0 2 118 H(Summary)WB 197 Sn()EH( +)0 P(This document describes the geometry of regions available for spatial +filtering in IRAF/PROS analysis. + + +)0 2 119 H(Geometric)WB 198 Sn( shapes)EH( +)0 P(Several geometric shapes are used to describe regions. The valid +shapes are: + +) 11 57 PR( shape: arguments: + ----- ---------------------------------------- + ANNULUS xcenter ycenter inner_radius outer_radius + BOX xcenter ycenter xwidth yheight \201angle\202 + CIRCLE xcenter ycenter radius + ELLIPSE xcenter ycenter xwidth yheight \201angle\202 + FIELD none + LINE x1 y1 x2 y2 + PIE xcenter ycenter angle1 angle2 + POINT x1 y1 + POLYGON x1 y1 x2 y2 ... xn yn)RP( + + +All arguments are real values; integer values are automatically +converted to real where necessary. All angles are in degrees and +specify angles that run counter-clockwise from the positive y-axis. + +)0 P(Shapes can be specified using "command" syntax: +) 1 23 PR( [shape] arg1 arg2 ...)RP( +or using "routine" syntax: +) 1 26 PR( [shape]\201arg1, arg2, ...\202)RP( +or by any combination of the these. \201Of course, the parentheses must +balance and there cannot be more commas than necessary.\202 The shape +keywords are case-insensitive. Furthermore, any shape can be +specified by a three-character unique abbreviation. For example, one +can specify three circular regions as: + +) 1 64 PR( "foo.fits[CIRCLE 512 512 50;CIR\201128 128, 10\202;cir\201650,650,20\202]")RP( + +\201Quotes generally are required to protect the region descriptor +from being processed by the Unix shell.\202 + +)2 1 1 HR( +)0 P(The )BD(annulus)ES( shape specifies annuli, centered at xcenter, +ycenter, with inner and outer radii \201r1, r2\202. For example, +) 1 20 PR( ANNULUS 25 25 5 10)RP( +specifies an annulus centered at 25.0 25.0 with an inner radius of 5.0 and +an outer radius of 10. Assuming \201as will be done for all examples in this +document, unless otherwise noted\202 this shape is used in a mask of size 40x40, +it will look like this: +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:....................111111111........... + 33:...................11111111111.......... + 32:.................111111111111111........ + 31:.................111111111111111........ + 30:................11111111111111111....... + 29:...............1111111.....1111111...... + 28:...............111111.......111111...... + 27:...............11111.........11111...... + 26:...............11111.........11111...... + 25:...............11111.........11111...... + 24:...............11111.........11111...... + 23:...............11111.........11111...... + 22:...............111111.......111111...... + 21:...............1111111.....1111111...... + 20:................11111111111111111....... + 19:.................111111111111111........ + 18:.................111111111111111........ + 17:...................11111111111.......... + 16:....................111111111........... + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................)WR( + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( + +)2 1 1 HR( +)0 P(The )BD(box)ES( shape specifies an orthogonally oriented box, +centered at xcenter, ycenter, of size xwidth, yheight. It requires four +arguments and accepts an optional fifth argument to specify a rotation angle. +When the rotation angle is specified \201in degrees\202, the box is rotated by +an angle that runs counter-clockwise from the positive y-axis. + +)0 P(The )BD(box)ES( shape specifies a rotated box, centered at +xcenter, ycenter, of size xwidth, yheight. The box is rotated by an angle +specified in degrees that runs counter-clockwise from the positive y-axis. +If the angle argument is omitted, it defaults to 0. + +)2 1 1 HR( +)0 P(The )BD(circle)ES( shape specifies a circle, centered at xcenter, +ycenter, of radius r. It requires three arguments. + +)2 1 1 HR( +)0 P(The )BD(ellipse)ES( shape specifies an ellipse, centered at +xcenter, ycenter, with y-axis width a and the y-axis length b defined such +that: +) 1 27 PR( x**2/a**2 + y**2/b**2 = 1)RP( +Note that a can be less than, equal to, or greater than b. The ellipse +is rotated the specified number of degrees. The rotation is done according +to astronomical convention, counter-clockwise from the positive y-axis. +An ellipse defined by: +) 1 23 PR( ELLIPSE 20 20 5 10 45)RP( +will look like this: +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:............111111...................... + 26:............11111111.................... + 25:............111111111................... + 24:............11111111111................. + 23:............111111111111................ + 22:............111111111111................ + 21:.............111111111111............... + 20:.............1111111111111.............. + 19:..............111111111111.............. + 18:...............111111111111............. + 17:...............111111111111............. + 16:................11111111111............. + 15:..................111111111............. + 14:...................11111111............. + 13:.....................111111............. + 12:........................................)WR( + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( + + +)2 1 1 HR( +)0 P(The )BD(field)ES( shape specifies the entire field as a +region. It is not usually specified explicitly, but is used implicitly in the +case where no regions are specified, that is, in cases where either a null +string or some abbreviation of the string "none" is input. +)BD(Field)ES( takes no arguments. + +)2 1 1 HR( +)0 P(The )BD(pie)ES( shape specifies an angular wedge of the entire field, +centered at xcenter, ycenter. The wedge runs between the two specified angles. +The angles are given in degrees, running counter-clockwise from the positive +x-axis. For example, +) 1 18 PR( PIE 20 20 90 180)RP( +defines a region from 90 degrees to 180 degrees, i.e., quadrant 2 of the +Cartesian plane. The display of such a region looks like this: + +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:11111111111111111111.................... + 39:11111111111111111111.................... + 38:11111111111111111111.................... + 37:11111111111111111111.................... + 36:11111111111111111111.................... + 35:11111111111111111111.................... + 34:11111111111111111111.................... + 33:11111111111111111111.................... + 32:11111111111111111111.................... + 31:11111111111111111111.................... + 30:11111111111111111111.................... + 29:11111111111111111111.................... + 28:11111111111111111111.................... + 27:11111111111111111111.................... + 26:11111111111111111111.................... + 25:11111111111111111111.................... + 24:11111111111111111111.................... + 23:11111111111111111111.................... + 22:11111111111111111111.................... + 21:11111111111111111111.................... + 20:........................................ + 19:........................................ + 18:........................................ + 17:........................................ + 16:........................................ + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................)WR( + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( +The pie slice specified is always a counter-clockwise sweep between +the angles, starting at the first angle and ending at the second. Thus: +) 1 17 PR( PIE 10 15 30 60)RP( +describes a 30 degree sweep from 2 o'clock to 1 o'clock, while: +) 1 17 PR( PIE 10 15 60 30)RP( +describes a 330 degree counter-clockwise sweep from 1 o'clock to 2 o'clock +passing through 12 o'clock \2010 degrees\202. Note in both of these examples that +the center of the slice can be anywhere on the plane. The second mask looks +like this: + +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:111111111111111111111111................ + 39:11111111111111111111111................. + 38:11111111111111111111111................. + 37:1111111111111111111111.................. + 36:1111111111111111111111.................. + 35:111111111111111111111................... + 34:11111111111111111111.................... + 33:11111111111111111111.................... + 32:1111111111111111111....................1 + 31:1111111111111111111..................111 + 30:111111111111111111.................11111 + 29:111111111111111111................111111 + 28:11111111111111111...............11111111 + 27:1111111111111111..............1111111111 + 26:1111111111111111.............11111111111 + 25:111111111111111............1111111111111 + 24:111111111111111..........111111111111111 + 23:11111111111111.........11111111111111111 + 22:11111111111111........111111111111111111 + 21:1111111111111.......11111111111111111111 + 20:111111111111......1111111111111111111111 + 19:111111111111....111111111111111111111111 + 18:11111111111....1111111111111111111111111 + 17:11111111111..111111111111111111111111111 + 16:1111111111.11111111111111111111111111111 + 15:1111111111111111111111111111111111111111 + 14:1111111111111111111111111111111111111111 + 13:1111111111111111111111111111111111111111 + 12:1111111111111111111111111111111111111111)WR( + 11:1111111111111111111111111111111111111111 + 10:1111111111111111111111111111111111111111 + 9:1111111111111111111111111111111111111111 + 8:1111111111111111111111111111111111111111 + 7:1111111111111111111111111111111111111111 + 6:1111111111111111111111111111111111111111 + 5:1111111111111111111111111111111111111111 + 4:1111111111111111111111111111111111111111 + 3:1111111111111111111111111111111111111111 + 2:1111111111111111111111111111111111111111 + 1:1111111111111111111111111111111111111111)RP( +The pie slice goes to the edge of the field. To limit its scope, pie +usually is is combined with other shapes, such as circles and annuli, +using boolean operations. \201See below and in "help regalgebra"\202. + +)0 P(Pie Performance Notes: +)0 P(Pie region processing time is proportional to the size of the image, +and not the size of the region. This is because the pie shape is the +only infinite length shape, and we essentially must check all y rows +for inclusion \201unlike other regions, where the y limits can be +calculated beforehand\202. Thus, pie can run very slowly on large images. +In particular, it will run MUCH more slowly than the panda shape in +image-based region operations \201such as funcnts\202. We recommend use of +panda over pie where ever possible. + +)0 P(If you must use pie, always try to put it last in a boolean && +expression. The reason for this is that the filter code is optimized +to exit as soon as the result is know. Since pie is the slowest +region, it is better to avoid executing it if another region can decide +the result. Consider, for example, the difference in time required to +process a Chandra ACIS file when a pie and circle are combined in +two different orders: + +) 5 74 PR( time ./funcnts nacis.fits "circle 4096 4096 100 && pie 4096 4096 10 78" +2.87u 0.38s 0:35.08 9.2% + + time ./funcnts nacis.fits "pie 4096 4096 10 78 && circle 4096 4096 100 " +89.73u 0.36s 1:03.50 141.8%)RP( + +)0 P(Black-magic performance note: + +)0 P(Panda region processing uses a )BD(quick test)ES( pie region instead of +the normal pie region when combining its annulus and pie shapes. This +)BD(qtpie)ES( shape differs from the normal pie in that it utilizes the +y limits from the previous region with which it is combined. In a +panda shape, which is a series of annuli combined with pies, the +processing time is thus reduced to that of the annuli. + +)0 P(You can use the qtpie shape instead of pie in cases where you are +combining pie with another shape using the && operator. This will +cause the pie limits to be set using limits from the other shape, and +will speed up the processing considerably. For example, the above +execution of funcnts can be improved considerably using this technique: + +) 2 75 PR( time ./funcnts nacis.fits "circle 4096 4096 100 && qtpie 4096 4096 10 78" +4.66u 0.33s 0:05.87 85.0%)RP( + +)0 P(We emphasize that this is a quasi-documented feature and might change in +the future. The qtpie shape is not recognized by ds9 or other programs. + +)2 1 1 HR( +)0 P(The )BD(line)ES( shape allows single pixels in a line between \201x1,y1\202 and +\201x2,y2\202 to be included or excluded. For example: +) 44 49 PR( LINE \2015,6, 24,25\202 +) 43 49 PR(displays as: +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:.......................1................ + 24:......................1................. + 23:.....................1.................. + 22:....................1................... + 21:...................1.................... + 20:..................1..................... + 19:.................1...................... + 18:................1....................... + 17:...............1........................ + 16:..............1......................... + 15:.............1.......................... + 14:............1...........................)WR( + 13:...........1............................)WR( + 12:..........1.............................)WR( + 11:.........1.............................. + 10:........1............................... + 9:.......1................................ + 8:......1................................. + 7:.....1.................................. + 6:....1................................... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( + +)2 1 1 HR( +)0 P(The )BD(point)ES( shape allows single pixels to be included or +excluded. Although the \201x,y\202 values are real numbers, they are truncated +to integer and the corresponding pixel is included or excluded, as specified. + +)0 P(Several points can be put in one region declaration; unlike the +original IRAF implementation, each now is given a different region mask value. +This makes it easier, for example, for funcnts to determine the number of +photons in the individual pixels. For example, +) 1 37 PR( POINT \2015,6, 10,11, 20,20, 35,30\202)RP( +will give the different region mask values to all four points, as shown below: + +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:..................................4..... + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:........................................ + 24:........................................ + 23:........................................ + 22:........................................ + 21:........................................ + 20:...................3.................... + 19:........................................ + 18:........................................ + 17:........................................ + 16:........................................ + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................)WR( + 11:.........2.............................. + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:....1................................... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( + +)2 1 1 HR( +)0 P(The )BD(polygon)ES( shape specifies a polygon with vertices +\201x1, y1\202 ... \201xn, yn\202. The polygon is closed automatically: one should +not specify the last vertex to be the same as the first. Any number of +vertices are allowed. For example, the following polygon defines a +right triangle as shown below: +) 1 33 PR( POLYGON \20110,10, 10,30, 30,30\202)RP( + +looks like this: + +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:..........11111111111111111111.......... + 29:..........1111111111111111111........... + 28:..........111111111111111111............ + 27:..........11111111111111111............. + 26:..........1111111111111111.............. + 25:..........111111111111111............... + 24:..........11111111111111................ + 23:..........1111111111111................. + 22:..........111111111111.................. + 21:..........11111111111................... + 20:..........1111111111.................... + 19:..........111111111..................... + 18:..........11111111...................... + 17:..........1111111....................... + 16:..........111111........................ + 15:..........11111......................... + 14:..........1111.......................... + 13:..........111........................... + 12:..........11............................)WR( + 11:..........1............................. + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( +Note that polygons can get twisted upon themselves if edge lines +cross. Thus: +) 1 37 PR( POL \20110,10, 20,20, 20,10, 10,20\202)RP( +will produce an area which is two triangles, like butterfly wings, as shown +below: + +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:........................................ + 24:........................................ + 23:........................................ + 22:........................................ + 21:........................................ + 20:........................................ + 19:..........1........1.................... + 18:..........11......11.................... + 17:..........111....111.................... + 16:..........1111..1111.................... + 15:..........1111111111.................... + 14:..........1111..1111.................... + 13:..........111....111.................... + 12:..........11......11....................)WR( + 11:..........1........1.................... + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( + +)2 1 1 HR( +)0 P(The following are combinations of pie with different shapes +\201called "panda" for "Pie AND Annulus"\202 allow for easy specification of +radial sections: +) 6 76 PR( shape: arguments: + ----- --------- + PANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + BPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad \201ang\202 # box + EPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad \201ang\202 # ellipse)RP( + +The )BD(panda)ES( \201)BD(P)ES(ies )BD(AND)ES( )BD(A)ES(nnuli\202 shape can be +used to create combinations of pie and annuli markers. It is analogous +to a Cartesian product on those shapes, i.e., the result is several +shapes generated by performing a boolean AND between pies and +annuli. Thus, the panda and cpanda specify combinations of annulus and +circle with pie, respectively and give identical results. The bpanda +combines box and pie, while epanda combines ellipse and pie. + +)0 P(Consider the example shown below: +) 1 31 PR( PANDA\20120,20, 0,360,3, 0,15,4\202)RP( +Here, 3 pie slices centered at 20, 20 are combined with 4 annuli, also +centered at 20, 20. The result is a mask with 12 regions \201displayed in +base 16 to save characters\202: +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:..............44444444444............... + 33:............444444444444444............. + 32:...........88444444444444444............ + 31:.........888844443333344444444.......... + 30:........88888833333333333444444......... + 29:........88888733333333333344444......... + 28:.......8888877733333333333344444........ + 27:......888887777332222233333344444....... + 26:......888877777622222222333334444....... + 25:.....88887777766622222222333334444...... + 24:.....88887777666622222222233334444...... + 23:.....88887777666651111222233334444...... + 22:.....88877776666551111122223333444...... + 21:.....88877776666555111122223333444...... + 20:.....888777766665559999aaaabbbbccc...... + 19:.....888777766665559999aaaabbbbccc...... + 18:.....888777766665599999aaaabbbbccc...... + 17:.....88887777666659999aaaabbbbcccc...... + 16:.....888877776666aaaaaaaaabbbbcccc...... + 15:.....888877777666aaaaaaaabbbbbcccc...... + 14:......8888777776aaaaaaaabbbbbcccc....... + 13:......888887777bbaaaaabbbbbbccccc....... + 12:.......88888777bbbbbbbbbbbbccccc........)WR( + 11:........888887bbbbbbbbbbbbccccc......... + 10:........888888bbbbbbbbbbbcccccc......... + 9:.........8888ccccbbbbbcccccccc.......... + 8:...........88ccccccccccccccc............ + 7:............ccccccccccccccc............. + 6:..............ccccccccccc............... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( + +)2 1 1 HR( +)0 P(Several regions with different mask values can be combined in the +same mask. This supports comparing data from the different regions. +\201For information on how to combine different shapes into a single +region, see "help regalgebra".\202 For example, consider the following +set of regions: +) 3 24 PR( ANNULUS 25 25 5 10 + ELLIPSE 20 20 5 10 315 + BOX 15 15 5 10)RP( +The resulting mask will look as follows: + +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:....................111111111........... + 33:...................11111111111.......... + 32:.................111111111111111........ + 31:.................111111111111111........ + 30:................11111111111111111....... + 29:...............1111111.....1111111...... + 28:...............111111.......111111...... + 27:...............11111.222222..11111...... + 26:...............111112222222..11111...... + 25:...............111112222222..11111...... + 24:...............111112222222..11111...... + 23:...............111112222222..11111...... + 22:...............111111222222.111111...... + 21:..............211111112222.1111111...... + 20:............322211111111111111111....... + 19:............32222111111111111111........ + 18:............22222111111111111111........ + 17:............222222211111111111.......... + 16:............22222222111111111........... + 15:............222222222................... + 14:............22222222.................... + 13:............222222...................... + 12:............33333.......................)WR( + 11:............33333....................... + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................)RP( +Note that when a pixel is in 2 or more regions, it is arbitrarily +assigned to a one of the regions in question \201often based on how a +give C compiler optimizes boolean expressions\202. + +)0 P()0 2 120 H(Region)WB 199 Sn( accelerators)EH( + +)0 P(Two types of \200fBaccelerators)ES(, to simplify region specification, +are provided as natural extensions to the ways shapes are described. +These are: extended lists of parameters, specifying multiple regions, +valid for annulus, box, circle, ellipse, pie, and points; and +)BD(n=)ES(, valid for annulus, box, circle, ellipse, and pie \201not +point\202. In both cases, one specification is used to define several +different regions, that is, to define shapes with different mask +values in the region mask. + +)0 P(The following regions accept )BD(accelerator)ES( syntax: +) 13 73 PR( shape arguments + ----- ------------------------------------------ + ANNULUS xcenter ycenter radius1 radius2 ... radiusn + ANNULUS xcenter ycenter inner_radius outer_radius n=[number] + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn \201angle\202 + BOX xcenter ycenter xwlo yhlo xwhi yhhi n=[number] \201angle\202 + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn \201angle\202 + ELLIPSE xcenter ycenter xwlo yhlo xwhi yhhi n=[number] \201angle\202 + PIE xcenter ycenter angle1 angle2 \201angle3\202 \201angle4\202 \201angle5\202 ... + PIE xcenter ycenter angle1 angle2 \201n=[number]\202 + POINT x1 y1 x2 y2 ... xn yn)RP( +Note that the circle accelerators are simply aliases for the annulus +accelerators. + +)0 P(For example, several annuli at the same center can be specified in one +region expression by specifying more than two radii. If )BD(N)ES( +radii are specified, then )BD(N)ES(-1 annuli result, with the outer +radius of each preceding annulus being the inner radius of the +succeeding annulus. Each annulus is considered a separate region, and +is given a separate mask value. For example, +) 1 30 PR( ANNULUS 20 20 0 2 5 10 15 20)RP( +specifies five different annuli centered at 20 20, and is equivalent to: +) 5 25 PR( ANNULUS 20.0 20.0 0 2 + ANNULUS 20.0 20.0 2 5 + ANNULUS 20.0 20.0 5 10 + ANNULUS 20.0 20.0 10 15 + ANNULUS 20.0 20.0 15 20)RP( +The mask is shown below: + +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:.............5555555555555.............. + 38:...........55555555555555555............ + 37:.........555555555555555555555.......... + 36:........55555555555555555555555......... + 35:......555555555555555555555555555....... + 34:.....55555555544444444444555555555...... + 33:....5555555544444444444444455555555..... + 32:....5555555444444444444444445555555..... + 31:...555555444444444444444444444555555.... + 30:..55555544444444444444444444444555555... + 29:..55555544444443333333334444444555555... + 28:.5555554444444333333333334444444555555.. + 27:.5555544444433333333333333344444455555.. + 26:555555444444333333333333333444444555555. + 25:555554444443333333333333333344444455555. + 24:555554444433333332222233333334444455555. + 23:555554444433333322222223333334444455555. + 22:555554444433333222222222333334444455555. + 21:555554444433333222111222333334444455555. + 20:555554444433333222111222333334444455555. + 19:555554444433333222111222333334444455555. + 18:555554444433333222222222333334444455555. + 17:555554444433333322222223333334444455555. + 16:555554444433333332222233333334444455555. + 15:555554444443333333333333333344444455555. + 14:555555444444333333333333333444444555555. + 13:.5555544444433333333333333344444455555.. + 12:.5555554444444333333333334444444555555..)WR( + 11:..55555544444443333333334444444555555... + 10:..55555544444444444444444444444555555... + 9:...555555444444444444444444444555555.... + 8:....5555555444444444444444445555555..... + 7:....5555555544444444444444455555555..... + 6:.....55555555544444444444555555555...... + 5:......555555555555555555555555555....... + 4:........55555555555555555555555......... + 3:.........555555555555555555555.......... + 2:...........55555555555555555............ + 1:.............5555555555555..............)RP( + +)0 P(For boxes and ellipses, if an odd number of arguments is specified, +then the last argument is assumed to be an angle. Otherwise, the +angle is assumed to be zero. For example: +) 1 38 PR( ellipse 20 20 3 5 6 10 9 15 12 20 45)RP( +specifies an 3 ellipses at a 45 degree angle: +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........33333333........................ + 35:......333333333333...................... + 34:.....3333333333333333................... + 33:....333333333333333333.................. + 32:....33333332222233333333................ + 31:...3333332222222222333333............... + 30:...33333222222222222233333.............. + 29:...333332222222222222223333............. + 28:...3333222222211112222223333............ + 27:...33332222211111111222223333........... + 26:...333322222111111111122223333.......... + 25:...3333222211111111111122223333......... + 24:....3332222111111..1111122223333........ + 23:....333322211111.....11112222333........ + 22:....33332222111.......11112223333....... + 21:.....33322221111.......11122223333...... + 20:.....33332221111.......11112223333...... + 19:.....33332222111.......11112222333...... + 18:......33332221111.......11122223333..... + 17:.......33322221111.....111112223333..... + 16:.......3333222211111..1111112222333..... + 15:........3333222211111111111122223333.... + 14:.........333322221111111111222223333.... + 13:..........33332222211111111222223333.... + 12:...........3333222222111122222223333....)WR( + 11:............333322222222222222233333.... + 10:.............33333222222222222233333.... + 9:..............3333332222222222333333.... + 8:...............33333333222223333333..... + 7:.................333333333333333333..... + 6:..................3333333333333333...... + 5:.....................333333333333....... + 4:.......................33333333......... + 3:........................................ + 2:........................................ + 1:........................................)RP( +Note in the above example that the lower limit is not part of the +region for boxes, circles, and ellipses. This makes circles and annuli +equivalent, i.e.: +) 2 26 PR( circle 20 20 5 10 15 20 + annulus 20 20 5 10 15 20)RP( +both give the following region mask: +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:.............3333333333333.............. + 38:...........33333333333333333............ + 37:.........333333333333333333333.......... + 36:........33333333333333333333333......... + 35:......333333333333333333333333333....... + 34:.....33333333322222222222333333333...... + 33:....3333333322222222222222233333333..... + 32:....3333333222222222222222223333333..... + 31:...333333222222222222222222222333333.... + 30:..33333322222222222222222222222333333... + 29:..33333322222221111111112222222333333... + 28:.3333332222222111111111112222222333333.. + 27:.3333322222211111111111111122222233333.. + 26:333333222222111111111111111222222333333. + 25:333332222221111111111111111122222233333. + 24:33333222221111111.....11111112222233333. + 23:3333322222111111.......1111112222233333. + 22:333332222211111.........111112222233333. + 21:333332222211111.........111112222233333. + 20:333332222211111.........111112222233333. + 19:333332222211111.........111112222233333. + 18:333332222211111.........111112222233333. + 17:3333322222111111.......1111112222233333. + 16:33333222221111111.....11111112222233333. + 15:333332222221111111111111111122222233333. + 14:333333222222111111111111111222222333333. + 13:.3333322222211111111111111122222233333.. + 12:.3333332222222111111111112222222333333..)WR( + 11:..33333322222221111111112222222333333... + 10:..33333322222222222222222222222333333... + 9:...333333222222222222222222222333333.... + 8:....3333333222222222222222223333333..... + 7:....3333333322222222222222233333333..... + 6:.....33333333322222222222333333333...... + 5:......333333333333333333333333333....... + 4:........33333333333333333333333......... + 3:.........333333333333333333333.......... + 2:...........33333333333333333............ + 1:.............3333333333333..............)RP( + +)0 P(As a final example, specifying several angles in one pie slice +expression is equivalent to specifying several separate slices with +the same center. As with the annulus, if )BD(N)ES( angles are +specified, then )BD(N)ES(-1 slices result, with the ending angle of +each preceding slice being the starting angle of the succeeding slice. +Each slice is considered a separate region, and is given a separate +mask value. For example, +) 1 26 PR( PIE 12 12 315 45 115 270)RP( +specifies three regions as shown below: +) 42 51 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:2222222222222222222222222222222222222222 + 39:2222222222222222222222222222222222222221 + 38:2222222222222222222222222222222222222211 + 37:2222222222222222222222222222222222222111 + 36:2222222222222222222222222222222222221111 + 35:3222222222222222222222222222222222211111 + 34:3222222222222222222222222222222222111111 + 33:3322222222222222222222222222222221111111 + 32:3322222222222222222222222222222211111111 + 31:3332222222222222222222222222222111111111 + 30:3332222222222222222222222222221111111111 + 29:3333222222222222222222222222211111111111 + 28:3333222222222222222222222222111111111111 + 27:3333322222222222222222222221111111111111 + 26:3333322222222222222222222211111111111111 + 25:3333322222222222222222222111111111111111 + 24:3333332222222222222222221111111111111111 + 23:3333332222222222222222211111111111111111 + 22:3333333222222222222222111111111111111111 + 21:3333333222222222222221111111111111111111 + 20:3333333322222222222211111111111111111111 + 19:3333333322222222222111111111111111111111 + 18:3333333332222222221111111111111111111111 + 17:3333333332222222211111111111111111111111 + 16:3333333333222222111111111111111111111111 + 15:3333333333222221111111111111111111111111 + 14:3333333333322211111111111111111111111111 + 13:3333333333322111111111111111111111111111 + 12:33333333333.1111111111111111111111111111)WR( + 11:3333333333331111111111111111111111111111 + 10:333333333333.111111111111111111111111111 + 9:333333333333..11111111111111111111111111 + 8:333333333333...1111111111111111111111111 + 7:333333333333....111111111111111111111111 + 6:333333333333.....11111111111111111111111 + 5:333333333333......1111111111111111111111 + 4:333333333333.......111111111111111111111 + 3:333333333333........11111111111111111111 + 2:333333333333.........1111111111111111111 + 1:333333333333..........111111111111111111)RP( + +)0 P(The annulus, box, circle, ellipse, and pie shapes also accept an +)BD(n=[int])ES( syntax for specifying multiple regions. The +)BD(n=[int])ES(syntax interprets the previous \201shape-dependent\202 +arguments as lower and upper limits for the region and creates n +shapes with evenly spaced boundaries. For example, if )BD(n=[int])ES( +is specified in an annulus, the two immediately preceding radii +\201)BD(rn)ES( and )BD(rm)ES(\202 are divided into )BD(int)ES( annuli, such +that the inner radius of the first is )BD(rn)ES( and the outer radius +of the last is )BD(rm)ES(. For example, +) 1 24 PR( ANNULUS 20 20 5 20 n=3)RP( +is equivalent to: +) 1 26 PR( ANNULUS 20 20 5 10 15 20)RP( +If this syntax is used with an ellipse or box, then the two preceding +pairs of values are taken to be lower and upper limits for a set of +ellipses or boxes. A circle uses the two preceding arguments for upper +and lower radii. For pie, the two preceding angles are divided into n +wedges such that the starting angle of the first is the lower bound +and the ending angle of the last is the upper bound. In all cases, +the )BD(n=[int])ES( syntax allows any single alphabetic character +before the "=", i.e, i=3, z=3, etc. are all equivalent. + +)0 P(Also note that for boxes and ellipses, the optional angle argument is +always specified after the )BD(n=[int])ES( syntax. For example: +) 45 57 PR( ellipse 20 20 4 6 16 24 n=3 45 +) 44 57 PR(specifies 3 elliptical regions at an angle of 45 degrees: + +) 42 45 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........33333333........................ + 39:.....33333333333333..................... + 38:....33333333333333333................... + 37:...33333333333333333333................. + 36:..33333333333333333333333............... + 35:.3333333333222223333333333.............. + 34:3333333322222222222233333333............ + 33:33333332222222222222223333333........... + 32:333333222222222222222222333333.......... + 31:3333322222222222222222222333333......... + 30:33333222222222111122222222333333........ + 29:333332222222111111112222222333333....... + 28:3333222222211111111111222222333333...... + 27:3333222222111111111111112222233333...... + 26:33332222221111111111111112222233333..... + 25:33332222211111111.111111112222233333.... + 24:333322222111111......111111222223333.... + 23:333322222111111.......111112222233333... + 22:33333222221111.........11111222223333... + 21:333332222211111.........11112222233333.. + 20:.33332222211111.........11111222223333.. + 19:.33333222221111.........111112222233333. + 18:..33332222211111.........11112222233333. + 17:..333332222211111.......111111222233333. + 16:...333322222111111......111111222223333. + 15:...333332222211111111.111111112222233333)WR( + 14:....333332222211111111111111122222233333)WR( + 13:.....33333222221111111111111122222233333 + 12:.....33333322222211111111111222222233333)WR( + 11:......3333332222222111111112222222333333 + 10:.......333333222222221111222222222333333 + 9:........33333322222222222222222222333333 + 8:.........333333222222222222222222333333. + 7:..........33333332222222222222223333333. + 6:...........3333333322222222222233333333. + 5:.............3333333333222223333333333.. + 4:..............33333333333333333333333... + 3:................33333333333333333333.... + 2:..................33333333333333333..... + 1:....................33333333333333......)RP( + +)0 P(Both the variable argument syntax and the )BD(n=[int])ES( syntax must +occur alone in a region descriptor \201aside from the optional angle for +boxes and ellipses\202. They cannot be combined. Thus, it is not valid +to precede or follow an )BD(n=[int])ES( accelerator with more angles or +radii, as in this example: +) 3 49 PR( # INVALID -- one too many angles before a=5 ... + # and no angles are allowed after a=5 + PIE 12 12 10 25 50 a=5 85 135)RP( +Instead, use three separate specifications, such as: +) 3 21 PR( PIE 12 12 10 25 + PIE 12 12 25 50 a=5 + PIE 12 12 85 135)RP( +The original \201IRAF\202 implementation of region filtering permitted this +looser syntax, but we found it caused more confusion than it was worth +and therefore removed it. + +)0 P(NB: Accelerators may be combined with other shapes in a boolean +expression in any order. \201This is a change starting with funtools +v1.1.1. Prior to this release, the accelerator shape had to be +specified last\202. The actual region mask id values returned depend on the +order in which the shapes are specified, although the total number of +pixels or rows that pass the filter will be consistent. For this +reason, use of accelerators in boolean expressions is discouraged in +programs such as funcnts, where region mask id values are used +to count events or image pixels. + +)0 P([All region masks displayed in this document were generated using the +)BD(fundisp)ES( routine and the undocumented "mask=all" argument \201with +spaced removed using sed \202: +) 2 72 PR( fundisp "funtools/funtest/test40.fits[ANNULUS 25 25 5 10]" mask=all |\200 + sed 's/ //g')RP( +Note that you must supply an image of the appropriate size -- in this case, +a FITS image of dimension 40x40 is used.] + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 121 H(Last)WB 200 Sn( updated: March 12, 2007)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (regalgebra.html) D +/Ti (Region Algebra) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 56 Sn( + + +)0 2 122 H(RegAlgebra:)WB 202 Sn()WB 201 Sn( Boolean Algebra on Spatial Regions)EA()EH( + + +)0 2 123 H(Summary)WB 203 Sn()EH( +)0 P(This document describes the boolean arithmetic defined for +region expressions. + + +)0 2 124 H(Description)WB 204 Sn()EH( +)0 P(When defining a region, several shapes can be combined using boolean +operations. The boolean operators are \201in order of precedence\202: +) 6 53 PR( Symbol Operator Associativity + ------ -------- ------------- + ! not right to left + & and left to right + ^ exclusive or left to right + | inclusive or left to right)RP( +For example, to create a mask consisting of a large circle with a +smaller box removed, one can use the )BD(and)ES( and )BD(not)ES( +operators: +) 1 36 PR( CIRCLE\20111,11,15\202 & !BOX\20111,11,3,6\202)RP( + +and the resulting mask is: +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:1111111111111111111111.................. + 2:1111111111111111111111.................. + 3:11111111111111111111111................. + 4:111111111111111111111111................ + 5:111111111111111111111111................ + 6:1111111111111111111111111............... + 7:1111111111111111111111111............... + 8:1111111111111111111111111............... + 9:111111111...1111111111111............... + 10:111111111...1111111111111............... + 11:111111111...1111111111111............... + 12:111111111...1111111111111............... + 13:111111111...1111111111111............... + 14:111111111...1111111111111............... + 15:1111111111111111111111111............... + 16:1111111111111111111111111............... + 17:111111111111111111111111................ + 18:111111111111111111111111................ + 19:11111111111111111111111................. + 20:1111111111111111111111.................. + 21:1111111111111111111111.................. + 22:111111111111111111111................... + 23:..11111111111111111..................... + 24:...111111111111111...................... + 25:.....11111111111........................ + 26:........................................ + 27:........................................ + 28:........................................ + 29:........................................)WR( + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................)RP( +A three-quarter circle can be defined as: +) 1 40 PR( CIRCLE\20120,20,10\202 & !PIE\20120,20,270,360\202)RP( + +and looks as follows: +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:........................................ + 2:........................................ + 3:........................................ + 4:........................................ + 5:........................................ + 6:........................................ + 7:........................................ + 8:........................................ + 9:........................................ + 10:........................................ + 11:...............111111111................ + 12:..............11111111111............... + 13:............111111111111111............. + 14:............111111111111111............. + 15:...........11111111111111111............ + 16:..........1111111111111111111........... + 17:..........1111111111111111111........... + 18:..........1111111111111111111........... + 19:..........1111111111111111111........... + 20:..........1111111111111111111........... + 21:..........1111111111.................... + 22:..........1111111111.................... + 23:..........1111111111.................... + 24:..........1111111111.................... + 25:...........111111111.................... + 26:............11111111.................... + 27:............11111111.................... + 28:..............111111.................... + 29:...............11111....................)WR( + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................)RP( +Two non-intersecting ellipses can be made into the same region: +) 1 40 PR( ELL\20120,20,10,20,90\202 | ELL\2011,1,20,10,0\202)RP( + +and looks as follows: +) 42 49 PR( 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:11111111111111111111.................... + 2:11111111111111111111.................... + 3:11111111111111111111.................... + 4:11111111111111111111.................... + 5:1111111111111111111..................... + 6:111111111111111111...................... + 7:1111111111111111........................ + 8:111111111111111......................... + 9:111111111111............................ + 10:111111111............................... + 11:...........11111111111111111............ + 12:........111111111111111111111111........ + 13:.....11111111111111111111111111111...... + 14:....11111111111111111111111111111111.... + 15:..11111111111111111111111111111111111... + 16:.1111111111111111111111111111111111111.. + 17:111111111111111111111111111111111111111. + 18:111111111111111111111111111111111111111. + 19:111111111111111111111111111111111111111. + 20:111111111111111111111111111111111111111. + 21:111111111111111111111111111111111111111. + 22:111111111111111111111111111111111111111. + 23:111111111111111111111111111111111111111. + 24:.1111111111111111111111111111111111111.. + 25:..11111111111111111111111111111111111... + 26:...11111111111111111111111111111111..... + 27:.....11111111111111111111111111111...... + 28:.......111111111111111111111111......... + 29:...........11111111111111111............)WR( + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................)RP( +You can use several boolean operations in a single region expression, +to create arbitrarily complex regions. With the important exception +below, you can apply the operators in any order, using parentheses if +necessary to override the natural precedences of the operators. + +)0 P(NB: Using a panda shape is always much more efficient than explicitly +specifying "pie & annulus", due to the ability of panda to place a +limit on the number of pixels checked in the pie shape. If you are +going to specify the intersection of pie and annulus, use panda +instead. + +)0 P(As described in "help regreometry", the )BD(PIE)ES( slice goes to the +edge of the field. To limit its scope, )BD(PIE)ES( usually is is +combined with other shapes, such as circles and annuli, using boolean +operations. In this context, it is worth noting that that there is a +difference between )BD(-PIE)ES( and )BD(&!PIE)ES(. The former is a +global exclude of all pixels in the )BD(PIE)ES( slice, while the latter +is a local excludes of pixels affecting only the region\201s\202 with which +the )BD(PIE)ES( is combined. For example, the following region uses +)BD(&!PIE)ES( as a local exclude of a single circle. Two other circles +are also defined and are unaffected by the local exclude: +) 21 56 PR( CIRCLE\2011,8,1\202 + CIRCLE\2018,8,7\202&!PIE\2018,8,60,120\202&!PIE\2018,8,240,300\202 + CIRCLE\20115,8,2\202 + + 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 + - - - - - - - - - - - - - - - + 15: . . . . . . . . . . . . . . . + 14: . . . . 2 2 2 2 2 2 2 . . . . + 13: . . . 2 2 2 2 2 2 2 2 2 . . . + 12: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 11: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 10: . . . . 2 2 2 2 2 2 2 . . . . + 9: . . . . . . 2 2 2 . . . . 3 3 + 8: 1 . . . . . . . . . . . . 3 3 + 7: . . . . . . 2 2 2 . . . . 3 3 + 6: . . . . 2 2 2 2 2 2 2 . . . . + 5: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 4: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 3: . . . 2 2 2 2 2 2 2 2 2 . . . + 2: . . . . 2 2 2 2 2 2 2 . . . . + 1: . . . . . . . . . . . . . . .)RP( +Note that the two other regions are not affected by the )BD(&!PIE)ES(, +which only affects the circle with which it is combined. + +)0 P(On the other hand, a )BD(-PIE)ES( is an global exclude that does +affect other regions with which it overlaps: +) 23 39 PR( CIRCLE\2011,8,1\202 + CIRCLE\2018,8,7\202 + -PIE\2018,8,60,120\202 + -PIE\2018,8,240,300\202 + CIRCLE\20115,8,2\202 + + 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 + - - - - - - - - - - - - - - - + 15: . . . . . . . . . . . . . . . + 14: . . . . 2 2 2 2 2 2 2 . . . . + 13: . . . 2 2 2 2 2 2 2 2 2 . . . + 12: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 11: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 10: . . . . 2 2 2 2 2 2 2 . . . . + 9: . . . . . . 2 2 2 . . . . . . + 8: . . . . . . . . . . . . . . . + 7: . . . . . . 2 2 2 . . . . . . + 6: . . . . 2 2 2 2 2 2 2 . . . . + 5: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 4: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 3: . . . 2 2 2 2 2 2 2 2 2 . . . + 2: . . . . 2 2 2 2 2 2 2 . . . . + 1: . . . . . . . . . . . . . . .)RP( +The two smaller circles are entirely contained within the two exclude +)BD(PIE)ES( slices and therefore are excluded from the region. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 125 H(Last)WB 205 Sn( updated: November 17, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (regcoords.html) D +/Ti (Spatial Region Coordinates) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 57 Sn( + + +)0 2 126 H(RegCoords:)WB 207 Sn()WB 206 Sn( Spatial Region Coordinates)EA()EH( + + +)0 2 127 H(Summary)WB 208 Sn()EH( +)0 P(This document describes the specification of coordinate systems, and the +interpretation of coordinate values, for spatial region filtering. + + +)0 2 128 H(Pixel)WB 209 Sn( coordinate systems)EH( +)0 P(The default coordinate system for regions is PHYSICAL, which means +that region position and size values are taken from the original +data. \201Note that this is a change from the original IRAF/PROS +implementation, in which the IMAGE coordinate system was the default.\202 +PHYSICAL coordinates always refer to pixel positions on the original +image \201using IRAF LTM and LTV keywords\202. With PHYSICAL coordinates, +if a set of coordinates specifies the position of an object in an +original FITS file, the same coordinates will specify the same object +in any FITS derived from the original. Physical coordinates are +invariant with blocking of FITS files or taking sections of images, +even when a blocked section is written to a new file. + +)0 P(Thus, although a value in pixels refers, by default, to the PHYSICAL +coordinate system, you may specify that position values refer to the +image coordinate system using the )BD(global)ES( or )BD(local)ES( +properties commands: + +) 2 23 PR( global coordsys image + circle 512 512 100)RP( + +The )BD(global)ES( command changes the coordinate system for all +regions that follow, while the )BD(local)ES( command changes the +coordinate system only for the region immediately following: +) 3 22 PR( local coordsys image + circle 512 512 100 + circle 1024 1024 200)RP( +This changes the coordinate system only for the region that follows. +In the above example, the second region uses the global coordinate +system \201PHYSICAL by default\202. + +)0 P()0 2 129 H(World)WB 210 Sn( Coordinate Systems)EH( + +If World Coordinate System information is contained in the data file +being filtered, it also is possible to define regions using a sky +coordinate system. Supported systems include: + +) 10 67 PR( name description + ---- ----------- + PHYSICAL pixel coords of original file using LTM/LTV + IMAGE pixel coords of current file + FK4, B1950 sky coordinate systems + FK5, J2000 sky coordinate systems + GALACTIC sky coordinate systems + ECLIPTIC sky coordinate systems + ICRS currently same as J2000 + LINEAR linear wcs as defined in file)RP( + +In addition, two mosaic coordinate systems have been defined that +utilize the \201evolving\202 IRAF mosaic keywords: + +) 4 68 PR( name description + ---- ----------- + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV)RP( +Again, to use one of these coordinate systems, the )BD(global)ES( or +)BD(local)ES( properties commands are used: + +) 1 26 PR( global coordsys galactic)RP( + +)0 2 130 H(WCS)WB 211 Sn( Positions and Sizes)EH( + +In addition to pixels, positional values in a WCS-enabled region can +be specified using sexagesimal or degrees format: + +) 11 57 PR( position arguments description + ------------------ ----------- + [num] context-dependent \201see below\202 + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + [num]:[num]:[num] hms for 'odd' position arguments + [num]:[num]:[num] dms for 'even' position arguments + [num]h[num]m[num]s explicit hms + [num]d[num]m[num]s explicit dms)RP( + +If ':' is used as sexagesimal separator, the value is considered to be +specifying hours/minutes/seconds if it is the first argument of a +positional pair, and degrees/minutes/seconds for the second argument +of a pair \201except for galactic coordinates, which always use degrees\202: + +) 7 78 PR( argument description + ----------- ----------- + 10:20:30.0 10 hours, 20 minutes, 30 seconds for 1st positional argument + 10 degrees, 20 minutes, 30 seconds for 2nd positional argument + 10h20m30.0 10 hours, 20 minutes, 30 seconds + 10d20m30.0 10 degrees, 20 minutes, 30 seconds + 10.20d 10.2 degrees)RP( + +Similarly, the units of size values are defined by the formating +character\201s\202 attached to a number: + +) 9 53 PR( size arguments description + -------------- ----------- + [num] context-dependent \201see below\202 + [num]" arc seconds + [num]' arc minutes + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels)RP( + +For example: +) 8 34 PR( argument description + ----------- ----------- + 10 ten pixels + 10' ten minutes of arc + 10" ten seconds of arc + 10d ten degrees + 10p ten pixels + 0.5r half of a radian)RP( + +)0 P(An example of using sky coordinate systems follows: + +) 4 65 PR( global coordsys B1950 + -box 175.54d 20.01156d 10' 10' + local coordsys J2000 + pie 179.57d 22.4d 0 360 n=4 && annulus 179.57d 22.4d 3' 24' n=5)RP( + +At the FK4 1950 coordinates 175.54d RA, 20.01156d DEC exclude a 10 +minute by 10 minute box. Then at the FK5 2000 coordinates 179.57d RA +22.4d DEC draw a radial profile regions pattern with 4 quadrants and 5 +annuli ranging from 3 minutes to 24 minutes in diameter. In this +example, the default coordinate system is overridden by the commands +in the regions spec. + +)0 2 131 H(NB:)WB 212 Sn( The Meaning of Pure Numbers Are Context Sensitive)EH( + +)0 P(When a "pure number" \201i.e. one without a format directive such as 'd' +for 'degrees'\202 is specified as a position or size, its interpretation +depends on the context defined by the 'coordsys' keyword. In general, +the rule is: + +)0 P()BD(All pure numbers have implied units corresponding to the current +coordinate system.)ES( + +)0 P(If no coordinate system is explicitly specified, the default system is +implicitly assumed to be PHYSICAL. In practice this means that for +IMAGE and PHYSICAL systems, pure numbers are pixels. Otherwise, +for all systems other than LINEAR, pure numbers are degrees. For +LINEAR systems, pure numbers are in the units of the linear system. +This rule covers both positions and sizes. + +)0 P(As a corollary, when a sky-formatted number is used with the IMAGE +or PHYSICAL coordinate system \201which includes the default case of no +coordsys being specified\202, the formatted number is assumed to be in +the units of the WCS contained in the current file. If no sky WCS is +specified, an error results. + +)0 P(Examples: + +) 2 43 PR( circle\201512,512,10\202 + ellipse 202.44382d 47.181656d 0.01d 0.02d)RP( + +)0 P(In the absence of a specified coordinate system, the circle uses the +default PHYSICAL units of pixels, while the ellipse explicitly uses degrees, +presumably to go with the WCS in the current file. + +) 5 43 PR( global coordsys=fk5 + global color=green font="system 10 normal" + circle 202.44382 47.181656 0.01 + circle 202.44382 47.181656 10p + ellipse\201512p,512p,10p,15p,20\202)RP( + + +)0 P(Here, the circles use the FK5 units of degrees \201except for the +explicit use of pixels in the second radius\202, while the ellipse +explicitly specifies pixels. The ellipse angle is in degrees. + +)0 P(Note that Chandra data format appears to use "coordsys=physical" +implicitly. Therefore, for most Chandra applications, valid regions +can be generated safely by asking ds9 to save/display regions in +pixels using the PHYSICAL coordsys. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 132 H(Last)WB 213 Sn( updated: November 17, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (regbounds.html) D +/Ti (Spatial Region Boundaries) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 58 Sn( + + +)0 2 133 H(RegBounds:)WB 215 Sn()WB 214 Sn( Region Boundaries)EA()EH( + + +)0 2 134 H(Summary)WB 216 Sn()EH( +Describes how spatial region boundaries are handled. + + +)0 2 135 H(Description)WB 217 Sn()EH( +)0 P(The golden rule for spatial region filtering was first enunciated by +Leon VanSpeybroeck in 1986: + +)0 P()BD(Each photon will be counted once, and no photon will be counted +more than once)ES(. + +)0 P(This means that we must be careful about boundary +conditions. For example, if a circle is contained in an annulus such +that the inner radius of the annulus is the same as the radius of the +circle, then photons on that boundary must always be assigned to one +or the other region. That is, the number of photons in both regions +must equal the sum of the number of photons in each region taken +separately. + +With this in mind, the rules for determining whether a boundary image +pixel or table row are assigned to a region are defined below. + +)0 2 136 H(Image)WB 218 Sn( boundaries : radially-symmetric shapes \201circle, annuli, ellipse\202)EH( + +For image filtering, pixels whose center is inside the boundary are +included. This also applies non-radially-symmetric shapes. When a +pixel center is exactly on the boundary, the pixel assignment rule is: + +)UL()-1 LI( the outer boundary of a symmetric shape does not include such pixels +)-1 LI( the inner boundary of a symmetric shape \201annulus\202 includes such pixels)LU( + +In this way, an annulus with radius from 0 to 1, centered exactly on a +pixel, includes the pixel on which it is centered, but none of its +neighbors. + +These rules ensure that when defining concentric shapes, no pixels are +omitted between concentric regions and no pixels are claimed by two +regions. When applied to small symmetric shapes, the shape is less +likely to be skewed, as would happen with non-radially-symmetric +rules. These rules differ from the rules for box-like shapes, which +are more likely to be positioned adjacent to one another. + +)0 2 137 H(Image)WB 219 Sn( Boundaries: non-radially symmetric shapes \201polygons, boxes\202)EH( + +For image filtering, pixels whose center is inside the boundary are +included. This also applies radially-symmetric shapes. When a pixel +center is exactly on the boundary of a non-radially symmetric region, +the pixel is included in the right or upper region, but not the left +or lower region. This ensures that geometrically adjoining regions +touch but don't overlap. + +)0 2 138 H(Row)WB 220 Sn( Boundaries are Analytic)EH( + +When filtering table rows, the boundary rules are the same as for +images, except that the calculation is not done on the center of a +pixel, \201since table rows, especially X-ray events rows, often have +discrete, floating point positions\202 but are calculated exactly. That +is, an row is inside the boundary without regard to its integerized +pixel value. For rows that are exactly on a region boundary, the +above rules are applied to ensure that all rows are counted once and +no row is counted more than once. + +)0 P(Because row boundaries are calculated differently from image boundaries, +certain programs will give different results when filtering the same +region file. In particular, fundisp/funtable \201which utilize analytic +row filtering\202 perform differently from funcnts \201which performs image +filtering, even on tables\202. + +)0 2 139 H(Image)WB 221 Sn( Boundaries vs. Row Boundaries: Practical Considerations)EH( + +)0 P(You will sometimes notice a discrepancy between running funcnts on an +binary table file and running fundisp on the same file with the same filter. +For example, consider the following: +) 2 49 PR( fundisp test1.fits"[box\2014219,3887,6,6,0\202]" | wc + 8893 320148 3752846)RP( +Since fundisp has a 2-line header, there are actually 8891 photons +that pass the filter. But then run funtable and select only the +rows that pass this filter, placing them in a new file: +) 1 58 PR( ./funtable test1.fits"[box\2014219,3887,6,6,0\202]" test2.fits)RP( +Now run funcnts using the original filter on the derived file: +) 12 61 PR( ./funcnts test2.fits "physical; box\2014219,3887,6,6,0\202" + + [... lot of processed output ...] + + # the following source and background components were used: + source region\201s\202 + ---------------- + physical; box\2014219,3887,6,6,0\202 + + reg counts pixels + ---- ------------ --------- + 1 7847.000 36)RP( +There are 1044 rows \201events\202 that pass the row filter in fundisp \201or +funtable\202 but fail to make it through funcnts. Why? + +)0 P(The reason can be traced to how analytic row filtering \201fundisp, funtable\202 +differs from integerized pixel filtering\201funcnts, funimage\202. Consider the +region: +) 1 22 PR( box\2014219,3887,6,6,0\202)RP( +Analytically \201i.e., using row filtering\202, positions will pass this +filter successfully if: +) 2 19 PR( 4216 <= x <= 4222 + 3884 <= y <= 3890)RP( +For example, photons with position values of x=4216.4 or y=3884.08 will pass. + +)0 P(Integerized image filtering is different in that the pixels that will +pass this filter have centers at: +) 2 40 PR( x = 4217, 4218, 4219, 4220, 4221, 4222 + y = 3885, 3886, 3887, 3888, 3889, 3890)RP( +Note that there are 6 pixels in each direction, as specified by the region. +That means that positions will pass the filter successfully if: +) 2 24 PR( 4217 <= \201int\202x <= 4222 + 3885 <= \201int\202y <= 3890)RP( +Photons with position values of x=4216.4 or y=3884.08 will NOT pass. + +)0 P(Note that the position values are integerized, in effect, binned into +image values. This means that x=4222.4 will pass this filter, but not +the analytic filter above. We do this to maintain the design goal that +either all counts in a pixel are included in an integerized filter, or +else none are included. + +)0 P([It could be argued that the correct photon limits for floating point +row data really should be: +) 2 23 PR( 4216.5 <= x <= 4222.5 + 3884.5 <= y <= 3890.5)RP( +since each pixel extends for .5 on either side of the center. We chose +to the maintain integerized algorithm for all image-style filtering so +that funcnts would give the exact same results regardless of whether +a table or a derived non-blocked binned image is used.] + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 140 H(Last)WB 222 Sn( updated: November 16, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (regdiff.html) D +/Ti (Differences Between Funtools and IRAF Regions) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 59 Sn( + + +)0 2 141 H(RegDiff:Differences)WB 224 Sn()WB 223 Sn( Between Funtools and IRAF Regions)EA()EH( + + +)0 2 142 H(Summary)WB 225 Sn()EH( +Describes the differences between Funtools/ds9 regions and the old IRAF/PROS +regions. + + +)0 2 143 H(Description)WB 226 Sn()EH( +)0 P(We have tried to make Funtools regions compatible with their +predecessor, IRAF/PROS regions. For simple regions and simple boolean +algebra between regions, there should be no difference between the two +implementations. The following is a list of differences and +incompatibilities between the two: + +)UL( +)0 P()-1 LI(If a pixel is covered by two different regions expressions, +Funtools assigns the mask value of the )BD(first)ES( region that +contains that pixel. That is, successive regions )BD(do not)ES( +overwrite previous regions in the mask, as was the case with the +original PROS regions. This means that one must define overlapping +regions in the reverse order in which they were defined in PROS. If +region N is fully contained within region M, then N should be defined +)BD(before)ES( M, or else it will be "covered up" by the latter. This +change is necessitated by the use of optimized filter compilation, i.e., +Funtools only tests individual regions until a proper match is made. + +)0 P()-1 LI( The )BD(PANDA)ES( region has replaced the old PROS syntax in which +a )BD(PIE)ES( accelerator was combined with an )BD(ANNULUS)ES( accelerator +using )BD(AND)ES(. That is, +) 1 48 PR( ANNULUS\20120,20,0,15,n=4\202 & PIE\20120,20,0,360,n=3\202)RP( +has been replaced by: +) 1 29 PR( PANDA\20120,20,0,360,3,0,15,4\202)RP( +The PROS syntax was inconsistent with the meaning of the )BD(AND)ES( operator. + +)0 P()-1 LI( The meaning of pure numbers \201i.e., without format specifiers\202 in +regions has been clarified, as has the syntax for specifying coordinate +systems. See the general discussion on +)0 54 1 A(Spatial Region Filtering)54 0 TN TL()Ec /AF f D( +for more information. +)LU( + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 144 H(Last)WB 227 Sn( updated: November 16, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (combo.html) D +/Ti (Combining Region and Table Filters) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 60 Sn( + + +)0 2 145 H(FunCombine:)WB 230 Sn()WB 228 Sn( Combining Region and Table Filters)EA()EH( + + +)0 2 146 H(Summary)WB 231 Sn()EH( +)0 P(This document discusses the conventions for combining region and table +filters, especially with regards to the comma operator. + + + +)0 2 147 H(Comma)WB 232 Sn()WB 229 Sn( Conventions)EA()EH( +)0 P(Filter specifications consist of a series of boolean expressions, +separated by commas. These expressions can be table filters, +spatial region filters, or combinations thereof. Unfortunately, +common usage requires that the comma operator must act differently +in different situations. Therefore, while its use is intuitive in +most cases, commas can be a source of confusion. + +)0 P(According to long-standing usage in IRAF, when a comma separates two +table filters, it takes on the meaning of a boolean )BD(and)ES(. Thus: +) 1 24 PR( foo.fits[pha==1,pi==2])RP( +is equivalent to: +) 1 27 PR( foo.fits[pha==1 && pi==2])RP( + +When a comma separates two spatial region filters, however, it has +traditionally taken on the meaning of a boolean )BD(or)ES(. Thus: +) 1 46 PR( foo.fits[circle\20110,10,3\202,ellipse\20120,20,8,5\202])RP( +is equivalent to: +) 1 49 PR( foo.fits[circle\20110,10,3\202 || ellipse\20120,20,8,5\202])RP( +\201except that in the former case, each region is given a unique id +in programs such as funcnts\202. + +)0 P(Region and table filters can be combined: +) 1 34 PR( foo.fits[circle\20110,10,3\202,pi=1:5])RP( +or even: +) 1 61 PR( foo.fits[pha==1&&circle\20110,10,3\202,pi==2&&ellipse\20120,20,8,5\202])RP( +In these cases, it is not obvious whether the command should utilize an +)BD(or)ES( or )BD(and)ES( operator. We therefore arbitrarily chose to +implement the following rule: +)UL()-1 LI( if both expressions contain a region, the operator used is )BD(or)ES(. +)-1 LI( if one \201or both\202 expression\201s\202 does not contain a region, the operator +used is )BD(and)ES(.)LU( +This rule handles the cases of pure regions and pure column filters properly. +It unambiguously assigns the boolean )BD(and)ES( to all mixed cases. Thus: +) 1 34 PR( foo.fits[circle\20110,10,3\202,pi=1:5])RP( +and +) 1 34 PR( foo.fits[pi=1:5,circle\20110,10,3\202])RP( +both are equivalent to: +) 1 37 PR( foo.fits[circle\20110,10,3\202 && pi=1:5])RP( + +)0 P([NB: This arbitrary rule )BD(replaces the previous arbitrary rule)ES( +\201pre-funtools 1.2.3\202 which stated: +)UL()-1 LI( if the 2nd expression contains a region, the operator used is )BD(or)ES(. +)-1 LI( if the 2nd expression does not contain a region, the operator +used is )BD(and)ES(.)LU( +In that scenario, the )BD(or)ES( operator was implied by: +) 1 21 PR( pha==4,circle 5 5 1)RP( +while the )BD(and)ES( operator was implied by +) 1 21 PR( circle 5 5 1,pha==4)RP( +Experience showed that this non-commutative treatment of the comma +operator was confusing and led to unexpected results.] + +)0 P(The comma rule must be considered provisional: comments and complaints +are welcome to help clarify the matter. Better still, we recommend +that the comma operator be avoided in such cases in favor of an +explicit boolean operator. + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 148 H(Last)WB 233 Sn( updated: November 16, 2005)EH( + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (env.html) D +/Ti (Funtools Environment Variables) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 61 Sn( + + +)0 2 149 H(FunEnv:)WB 235 Sn()WB 234 Sn( Funtools Environment Variables)EA()EH( + + +)0 2 150 H(Summary)WB 236 Sn()EH( +Describes the environment variables which can be used to tailor the overall +Funtools environment. + + +)0 2 151 H(Description)WB 237 Sn()EH( +)0 P(The following environment variables are supported by Funtools: +)0 DL()0 P()0 DT()BD(FITS_EXTNAME)ES( +)DD( The )BD(FITS_EXTNAME)ES( environment variable specifies the +default FITS extension name when )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( is called on a file lacking +a primary image. Thus, +) 1 29 PR( setenv FITS_EXTNAME "NEWEV")RP( +will allow you to call )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( on files without specifying NEWEV in +the +)0 42 1 A(Funtools bracket specification)42 0 TN TL()Ec /AF f D(. +If no FITS_EXTNAME variable is defined and the extension name also is +not passed in the bracket specification, then the default will be to +look for standard X-ray event table extension names "EVENTS" or +"STDEVT" \201we are, after all, and X-ray astronomy group at heart!\202. + +)0 P()0 DT()BD(FITS_EXTNUM)ES( +)DD( The )BD(FITS_EXTNUM)ES( environment variable specifies the +default FITS extension number when )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( is called on a file lacking +a primary image. Thus, +) 1 22 PR( setenv FITS_EXTNUM 7)RP( +will allow you to call )0 25 1 A(FunOpen\201\202)25 0 TN TL()Ec /AF f D( on files to open the seventh +extension without specifying the number in the +)0 42 1 A(Funtools bracket specification)42 0 TN TL()Ec /AF f D(. + +)0 P()0 DT()BD(FITS_BINCOLS)ES( and )BD(EVENTS_BINCOLS)ES( +)DD( These environment variable specifies the default binning key for +FITS binary tables and raw event files, respectively. They can be +over-ridden using the )BD(bincols=[naxis1,naxis2])ES( keyword in a +)0 42 1 A(Funtools bracket specification)42 0 TN TL()Ec /AF f D(. +The value of each environment variable +is a pair of comma-delimited columns, enclosed in parentheses, to use +for binning. For example, if you want to bin on detx and dety by +default, then use: +) 1 35 PR( setenv FITS_BINCOLS "\201detx,dety\202")RP( +in preference to adding a bincols specification to each filename: +) 1 31 PR( foo.fits[bincols=\201detx,dety\202])RP( + +)0 P()0 DT()BD(FITS_BITPIX)ES( and )BD(EVENTS_BITPIX)ES( +)DD( These environment variable specifies the default bitpix value for +binning FITS binary tables and raw event files, respectively. They can +be over-ridden using the )BD(bitpix=[value])ES( keyword in a +)0 42 1 A(Funtools bracket specification)42 0 TN TL()Ec /AF f D(. The value +of each environment variable is one of the standard FITS bitpix values +\2018,16,32,-32,-64\202. For example, if you want binning routines to +create a floating array, then use: +) 1 24 PR( setenv FITS_BITPIX -32)RP( +in preference to adding a bitpix specification to each filename: +) 1 22 PR( foo.fits[bitpix=-32])RP( + +)0 P()0 DT()BD(ARRAY)ES( +)DD( The )BD(ARRAY)ES( environment variable specifies the default +definition of an array file for Funtools. +It is used if there is no array specification passed in the +)BD(ARRAY\201\202)ES( directive in a +)0 46 1 A(Non-FITS Array specification)46 0 TN TL()Ec /AF f D(. +The value of the environment variable is a valid array specification such as: +) 2 25 PR( setenv ARRAY "s100.150" + foo.arr[ARRAY\201\202])RP( +This can be defined in preference to adding the specification to each filename: +) 1 26 PR( foo.arr[ARRAY\201s100.150\202])RP( + +)0 P()0 DT()BD(EVENTS)ES( +)DD( The )BD(EVENTS)ES( environment variable specifies the default +definition of an raw event file for Funtools. +It is used if there is no EVENTS specification passed in the +)BD(EVENTS\201\202)ES( directive in a +)0 45 1 A(Non-FITS EVENTS specification)45 0 TN TL()Ec /AF f D(. +The value of the environment variable is a valid EVENTS specification such as: +) 2 73 PR( setenv EVENTS "x:J:1024,y:J:1024,pi:I,pha:I,time:D,dx:E:1024,dx:E:1024" + foo.ev[EVENTS\201\202])RP( +This can be defined in preference to adding the specification to each filename: +) 1 73 PR( foo.ev[EVENTS\201x:J:1024,y:J:1024,pi:I,pha:I,time:D,dx:E:1024,dx:E:1024\202])RP()LD( + +The following filter-related environment variables are supported by Funtools: +)0 DL( +)0 P()0 DT()BD(FILTER_PTYPE)ES( +)DD( The )BD(FILTER_PTYPE)ES( environment variable specifies how to +build a filter. There are three possible methods: +)0 DL()0 DT(process or p +)DD(The filter is compiled and linked against the funtools library \201which +must therefore be accessible in the original install directory\202 to produce +a slave program. This program is fed events or image data and returns +filter results. + +)0 DT(dynamic or d \201gcc only\202 +)DD(The filter is compiled and linked against the funtools library \201which +must therefore be accessible in the original install directory\202 to produce +a dynamic shared object, which is loaded into the funtools program and +executed as a subroutine. \201Extensive testing has shown that, contrary to +expectations, this method is no faster than using a slave process.\202 + +)0 DT(contained or c +)DD(The filter and all supporting region code is compiled and linked +without reference to the funtools library to produce a slave program +\201which is fed events or image data and returns filter results\202. This method +is slower than the other two, because of the time it takes to compile the +region filtering code. It is used by stand-alone programs such as ds9, +which do not have access to the funtools library.)LD( + +By default, )BD(dynamic)ES( is generally used for gcc compilers and +)BD(process)ES( for other compilers. However the filter building algorithm +will check for required external files and will use )BD(contained)ES( is +these are missing. + +)0 P()0 DT()BD(FUN_MAXROW)ES( +)DD( The )BD(FUN_MAXROW)ES( environment variable is used by core +row-processing Funtools programs \201funtable, fundisp, funcnts, funhist, +funmerge, and funcalc\202 to set the maximum number of rows read at once +\201i.e. it sets the third argument to the FunTableRowGet\201\202 call\202. The +default is 8192. Note that this variable is a convention only: it will +not be a part of a non-core Funtools program unless code is explicitly +added, since each call to FunTableRowGet\201\202 specifies its own maximum +number of rows to read. NB: if you make this value very large, you +probably will need to increase )BD(FUN_MAXBUFSIZE)ES( \201see below\202 as well. + +)0 P()0 DT()BD(FUN_MAXBUFSIZE)ES( +)DD( The )BD(FUN_MAXBUFSIZE)ES( environment variable is used to limit the +max buffer size that will be allocated to hold table row data. This +buffer size is calculated to be the row size of the table multiplied +by the maximum number of rows read at once \201see above\202. Since the +row size is unlimited \201and we have examples of it being larger than 5 +Mb\202, it is possible that the total buffer size will exceed the machine +capabilities. We therefore set a default value of 5Mb for the max buffer +size, and adjust maxrow so that the total size calculated is less than +this max buffer size. \201If the row size is greater than this max buffer +size, then maxrow is set to 1.\202 This environment variable will change +the max buffer size allowed. + +)0 P()0 DT()BD(FILTER_CC)ES( +)DD( The )BD(FILTER_CC)ES( environment variable specifies the compiler to +use for compiling a filter specification. You also can use the )BD(CC)ES( +environment variable. If neither has been set, then gcc will be used +if available. Otherwise cc is used if available. + +)0 P()0 DT()BD(FILTER_EXTRA)ES( +)DD( The )BD(FILTER_EXTRA)ES( environment variable specifies extra options +to add to a filter compile command line. In principle, you can add libraries, +include files, and compiler switches. This variable should be used with care. + +)0 P()0 DT()BD(FILTER_TMPDIR)ES( +)DD( The )BD(FILTER_TMPDIR)ES( environment variable specifies the temporary +directory for filter compilation intermediate files. You also can use +the )BD(TMPDIR)ES( and )BD(TMP)ES( variables. By default, /tmp is used +as the temporary directory. + +)0 P()0 DT()BD(FILTER_KEEP)ES( +)DD( The )BD(FILTER_KEEP)ES( environment variable specifies whether the +intermediate filter files \201i.e. C source file and compile log file\202 +should be saved after a filter is built. The default is "false", so that +these intermediate files are deleted. This variable is useful for debugging, +but care should be taken to reset its value to false when debugging is +complete. +)LD( + + + + + +)0 P()0 0 1 A(Go to Funtools Help Index)0 0 TN TL()Ec /AF f D( + +)0 5 152 H(Last)WB 238 Sn( updated: November 16, 2005)EH( + + +)WB NL +/Cb Db D /Ct [16#00 16#00 16#00] D /Cl [16#00 16#00 16#00] D /CL -1 D Ct Sc +DS +/Ba f D /BO 0 D Bs +/UR (changelog.html) D +/Ti (Funtools ChangeLog) D +/Au () D +/Df f D +/ME [()] D + +0 BO R +()1 Sl()WB 62 Sn( +)0 2 153 H(Funtools)WB 239 Sn( ChangeLog)EH( + +)0 P(This ChangeLog covers both the Funtools library and the suite of +applications. It will be updated as we continue to develop and improve +Funtools. The up-to-date version can be found +)R5 2 A(here)EA(. +[The changelog for the initial development of Funtools, covering the +beta releases, can be found +)R6 2 A(here)EA(.] + +)0 2 154 H()WB 240 Sn( Patch Release 1.4.5 \201internal ds9 release\202)EH( +)UL()0 P()-1 LI( Removed permission checking from Find\201\202 on cygwin systems. This was broken +by Windows 7. +)0 P()-1 LI( Removed addition of -no-cpp-precomp flag from gcc 4.2 and beyond.)LU( + +)0 2 155 H()WB 241 Sn( Patch Release 1.4.4 \201internal ds9 release\202)EH( +)UL()0 P()-1 LI( Fixed -J funcone, which was not properly outputting all rows. +)0 P()-1 LI( Filter: when an image is flipped, the rotation angle must be reversed. +)0 P()-1 LI( Filter: add support for windows-based ipc communication when using tcc +compiler.)LU( + +)0 2 156 H()WB 242 Sn( Patch Release 1.4.3 \201internal ds9 release\202)EH( +)UL()0 P()-1 LI( Filter: improve checks for existence of compiler, even if CC is set. +)0 P()-1 LI( Change launch.h to xlaunch.h to avoid conflict with OS X. +)0 P()-1 LI( handle flipped images in filtering)LU( + +)0 2 157 H()WB 243 Sn( Patch Release 1.4.2 \201internal ds9 release\202)EH( +)UL()0 P( +)-1 LI( Port to gcc 4.2. +)0 P()-1 LI( Fix 1-byte filters on intel machines \201missing SW1 no-op\202. +)0 P()-1 LI( Remove ambiguity from filt.l \201and calc.l\202 using [A-z] in a +case-insensitive lexer. +)0 P()-1 LI( In funsky, the default unit for RA was changed from hours to degrees. +)0 P()-1 LI( Fixed bug in funtable in which TCRVL header values were output as strings. +)0 P()-1 LI( Added support for running funtools filters in Rosetta \201i.e. running PPC +executables on an Intel Mac\202 by sensing and handling data swap requirements. +Only works with FILTER_PTYPE set to 'c' \201can't link against wrong architecture +libraries\202. +)0 P()-1 LI( Fixed bug in FITS library to allow "-" in extension names. +)0 P()-1 LI( Code and documentation now agree that the copy extension specifier \201'+'\202 +comes after the extension name.)LU( + +)0 2 158 H()WB 244 Sn( Patch Release 1.4.1 \201internal ds9 release\202)EH( +)UL()0 P()-1 LI( Modified internal Launch\201\202 routine to use posix_spawn\201\202, if necessary. +This is required for OS X 10.5 \201leopard\202, which frowns upon use of fork\201\202 +and exec\201\202. Also modified zprocess routines to use Launch\201\202.)LU( + +)0 2 159 H()WB 245 Sn( Public Release 1.4.0 \20115 August 2007\202)EH( + +)UL()-1 LI( Public release of production-quality code, incorporating changes +and improvements from previous beta releases, including: +)UL()-1 LI( Support for access to ASCII text column files. +)-1 LI( Support for fast indexed access of binary tables. +)-1 LI( Support for database views of tables, i.e. pre-set values for the filter + specification, the columns to activate, and display format. +)-1 LI( New programs include funcone \201cone search\202, funindex \201create index files\202, + and funcen \201calculate centroids within regions\202.)LU( +)LU( + +)0 2 160 H()WB 246 Sn( Release 1.3.0b[n] \201mainly internal SAO beta releases\202)EH( +)UL( +)0 P()-1 LI( Added -F[c] switch to change the column delimiter to the +specified character. + +)0 P()-1 LI( Extended fundisp's format switch \201-f\202 so that it can now handle complex +formats such as 'x=sometext%3d- y=othertest%3d.ext'. + +)0 P()-1 LI( Added support for creating and processing 1D FITS images. + +)0 P()-1 LI( Added vcol=colname and vcol=/colname to filter specifications to +support use of a third value column when binning 2D images. + +)0 P()-1 LI( Added switches to funcone to write out data rows are not within +any cone \201-J, -X\202 and centers which have no close data rows \201-L\202. + +)0 P()-1 LI( In funjoin, added ability to specify a numeric tolerance for when joining +two files. + +)0 P()-1 LI( shared memory support in gio now can create a shared segment if w+ is +specified as the open mode. + +)0 P()-1 LI( Changed reggeometry man page so that examples correctly show angles +going counter-clockwise from the x-axis instead of from the y-axis. + +)0 P()-1 LI( Added checks to funmerge to ensure that all files have the same columns. + +)0 P()-1 LI( Fixed bug in text support that prevented header-less files from being +processed properly. + +)0 P()-1 LI( Added support for 64-bit images \201bitpix=64\202 and table columns \201TFORM=K\202. + +)0 P()-1 LI( Filter code was not applying bscale/bzero to columns. + +)0 P( +)-1 LI( Fixed funimage bug that caused a .5/block error in WCS CRPIX values +generated from binary tables. + +)0 P()-1 LI( Added feq\201a,b\202 and div\201a,b\202 macros to funcalc. + +)0 P()-1 LI( Added support for single-line #define to funcalc. + +)0 P()-1 LI( Updated wcs library to 3.6.6 + +)0 P()-1 LI( Fix bug in funcen in which ra,dec was not being calculated correctly +if physical and image coords did not match up. + +)0 P()-1 LI( The filter syntax "col1 = col2" now explicitly generates an error +\201you really want to do "col1 == col2"\202. + +)0 P()-1 LI( Added -o switch to include offset from the nominal target position. + +)0 P()-1 LI( Fundisp now displays multi-dimensional vector columns properly. + +)0 P()-1 LI( Documented support for lists of files processed as a single file +using "list: file1 ... filen" syntax. + +)0 P()-1 LI( Fixed bugs in support for pipe file type \201i.e. ability to pass +commands as a filename using "pipe: cmd arg1 ... argn" syntax\202. + +)0 P()-1 LI( Fixed bug in funhist processing of image-based pixel histograms +\201i.e using "xy" for columns\202 where a region was specified. All pixels +outside the region were erroneously being added to the bin containing +the 0 value. + +)0 P()-1 LI( Disabled multi-file processing in funds9, which was breaking support +for pathnames containing spaces and is not used by ds9 anyway. + +)0 P()-1 LI( Added support for Views of tables, i.e. pre-set values for the +filter specification, the columns to activate, and display format +\201though the latter is for fundisp only\202. + +)0 P()-1 LI( Added -l switch to funimage to read x, y, val columns from a list. + +)0 P()-1 LI( Removed useless and meaningless section syntax foo'[*]' because it +breaks pointer de-referencing on string columns \201i.e. foo'[*xxx=='a']'\202. +Use foo'[*,*]' instead, as documented. + +)0 P()-1 LI( String variables were not always being terminated properly in the +filter code because FITS 'A' data is not necessarily null-terminated. + +)0 P()-1 LI( Added funtools version number to all usage\201\202 displays. + +)0 P()-1 LI( Added explanation of switch arguments to many usage\201\202 displays. + +)0 P()-1 LI( The filter keyword row# now supports single row selection as well +as range selection, i.e., "row#=100" along with previous "row#=100:200". + +)0 P()-1 LI( fundisp now outputs "0x" before hex values. + +)0 P()-1 LI( Fixed bug in filter parser which processed rangelists incorrectly +if spaces were put into the rangelist \201i.e. "pha= 1 : 3" instead of +pha=1:3\202. + +)0 P()-1 LI( Fixed a bug in funindex which created a wrongly named index file +if more than one "." was in the input file name. + +)0 P()-1 LI( Added support to funcone to take ra, dec, radius from a list +\201i.e. columns in a FITS file or a text file\202. + +)0 P()-1 LI( Fixed a bug in FunColumnActivate so that if some columns are +explicitly activated while others are de-activated, only the +explicitly activated columns are activated \201code was activating all +columns in this case\202. + +)0 P()-1 LI( Fixed a bug in funindex which prevented indexing tables containing +a column named N. + +)0 P()-1 LI( fundisp now encloses ASCII column values in single quotes \201unless +-T is specified to output RDB format\202. + +)0 P()-1 LI( If a filter specification only involves indexed columns, then the +compiled filter is not used. + +)0 P()-1 LI( Funmerge can now be given a list of files to merge using @list syntax. +Also removed the restriction on how many files can be merged \201was limited to +the max number of open files\202. + +)0 P()-1 LI( Added ability to edit \201add, delete, modify\202 header parameters in funhead +by specifying an output file \201editing acts as a filter\202 and an edit command +file \201which can be stdin\202. + +)0 P()-1 LI( Funtools now contains preliminary code to support \201fast\202 indexed access +of binary tables. See idx.html or "man funidx" for more details. + +)0 P()-1 LI( Funtools now contains preliminary code supporting access to ASCII +column files. See text.html or "man funtext" for more details. + +)0 P()-1 LI( Fixed bug in funcalc in which columns used in an expression were +always being replaced by new columns, with all associated parameters +\201e.g. WCS\202 were being deleted. Now this only happens if the column +explicitly changes its data type. + +)0 P()-1 LI( Fixed bug in funcalc in which the raw data and user data became out +of sync for one row after every 8192 \201FUN_MAXROW\202 rows. + +)0 P()-1 LI( Fixed bug in gio in which gseek returned 0 instead of the current byte +offset for disk files. + +)0 P()-1 LI( Added funcone program to perform cone search on RA, Dec columns in +a FITS binary table. + +)0 P()-1 LI( Fixed bug in polygon, pie and rotated box region filtering for +tables \201nearby rows exactly in line between two non-vertical or +non-horizontal vertices were being accepted incorrectly\202. + +)0 P()-1 LI( Fixed pie and panda regions so that the angles now start from +positive x axis == 0 degrees and run counter-clockwise, as +documented. They were going from positive y. NB: a similar change +was made to ds9 release 4.0b3. You must be using ds9 4.0b3 or later +in order to have the correct behavior when generating regions in ds9 +and using them in funtools. + +)0 P()-1 LI( Added -p [prog] switch to funcalc to save the generated program. +instead of executing \201and deleting\202 it. + +)0 P()-1 LI( Upgraded zlib to 1.2.3. +)LU( + +)0 2 161 H()WB 247 Sn( Patch Release 1.2.4 \201internal SAO and beta release only\202)EH( +)UL( +)0 P()-1 LI( In funcalc, added support for user-specified arguments via the +-a [argstr] switch. These arguments are accessed in the compiled program +using the supplied ARGC and ARGV\201n\202 macros. + +)0 P()-1 LI( Added -n \201no header display\202 to fundisp to skip outputting header. + +)0 P()-1 LI( Added checks for various types of blank filters. + +)0 P( +)-1 LI( Added macros NROW \201current row number\202 and WRITE_ROW \201write current +row to disk\202 to funcalc. + +)0 P()-1 LI( funcalc no longer requires that at least one data column be +specified in the compiled expression. + +)0 P()-1 LI( Added FUN_NROWS to FunInfoGet\201\202 to return the total number of rows in +an input table \201i.e. value of NAXIS2\202. + +)0 P( +)-1 LI( The compiled funcalc program now includes stdlib.h and unistd.h. + +)0 P()-1 LI( The util/NaN.h header file is now modified at configure time to +contain endian status for the target architecture. References to +specific platforms have been removed. + +)0 P()-1 LI( Added -m switch to funtable to output multiple files, one for +each input region \201and a separate file for events that pass the +filters but are not in any region\202. + +)0 P()-1 LI( Added ability to add new parameters \201FunParamPutx\202 after writing +data if space is previously reserved in the form of a blank parameter +whose value is the name of the param to be updated. \201Also requires the +append argument of FunParamPutx be set to 2\202. + +)0 P( +)-1 LI( Added ability to build shared libraries. With --enable-shared=yes, +shared library is built but not used. With --enable-shared=link, +shared library is linked against \201requires proper installation and/or +use of LD_LIBRARY_PATH\202. + +)0 P()-1 LI( Added -v [column] support to funcnts so that counts in a table +can be accumulated using values from a specified column \201instead of +the default case where an integral count is accumulated for each event +in a region\202. + +)0 P()-1 LI( Added funcen program to calculate centroids within regions +\201binary tables only\202. Also added support for a funcen-based centroid +tool to funtools.ds9. + +)0 P()-1 LI( Fixed bug which prevented successful filtering of columns containing +arrays. + +)0 P()-1 LI( Added filter check to ensure that a column is not incorrectly used +as an array. + +)0 P()-1 LI( Fundisp now displays column arrays indexed from 0, not 1. + +)0 P()-1 LI( Added -i [interval] support to funcnts so that multiple intervals +can be processed in a single pass through the data. For example, +specifying -i "pha=1:5;pha=6:10;pha=11:15" will generate results in +each of 3 pha bands. + +)0 P()-1 LI( Fixed calculation of LTV quantities when binning floating point +column data \201value was off by 0.5\202. + +)0 P()-1 LI( Added support for 'D' in floating point header values. + +)0 P()-1 LI( Added -a switch to funimage and funtable to append output image or +table to an existing FITS file \201as an IMAGE or BINTABLE extension\202. + +)0 P()-1 LI( Added support for column scaling \201TSCAL and TZERO\202 on input columns. +Note that the default column type is changed to accommodate scaling \201e.g. +a column of type 'I' is changed to 'J', 'J' is changed to 'D'\202 so that +the scaled values can be handled properly by programs such as fundisp +\201which utilize default types\202. + +)0 P( +)-1 LI( Added support to FunColumnSelect\201\202 for handling structs of arrays +\201i.e. where returned columns are contiguous\202 instead of the default array +of structs \201returned row are contiguous\202. This is done by specifying +"org=structofarrays" in the plist and passing a single struct containing +the arrays. + +)0 P()-1 LI( When writing an rdb/starbase file, fundisp now outputs the full +column name, regardless of the width of the column \201which ordinarily +is truncated to match\202. + +)0 P()-1 LI( Fixed support for large files by changing all file positions variables +from "long" declarations to "off_t. + +)0 P()-1 LI( Fixed bug in funcalc incorrectly processed multiple array +references \201e.g. cur->foo[0]=cur->x;cur->foo[1]=cur->y;\202 within a single +line of code. + +)0 P()-1 LI( Added FILTER_CFLAGS environment variable for all filtering. Also added +--with-filter-cc and --with-filter-cflags options on configure to allow +specification of a default C compiler and associated CFLAGS for filtering. +All of this is necessary in order to support 64-bit libraries under Solaris. + +)0 P()-1 LI( Added the funtbl script to extract a table from Funtools ASCII output. + +)0 P()-1 LI( Added code to funimage to update IRAF DATASEC keyword. + +)0 P()-1 LI( Added checks to ensure that image dimensions are positive. + +)0 P()-1 LI( Fixed a bug in funimage where int data was being scaled using BSCALE and +BZERO but these keywords also were being retained in the output image header. +Now the data are not scaled unless the output data type is float \201in which +case the scaling parameters are removed\202. + +)0 P()-1 LI( Fixed a bug in funmerge which prevented merging of files unless one of +the -f, -w, or -x switches were used. + +)0 P()-1 LI( Fixed a bug in funtable and fundisp which caused the special '$n' column +to be output incorrectly. + +)0 P()-1 LI( Fixed sort option in funtable, which previously worked only if the +record size was an even divisor of 8192 \201and returned garbage otherwise\202. + +)0 P()-1 LI( Fixed bug in filters involving FITS data type 'X' \201bitfield\202. + +)0 P()-1 LI( Fixed bug in funcnts in which the output angles and radii were +being displayed incorrectly when multiple panda shapes were specified. + +)0 P()-1 LI( Fixed bug in pandas and pies using n= syntax when first angle +specified was greater than second. The resulting mask was of +the correct shape but contained only a single region. + +)0 P()-1 LI( Table row access routines will now decrease maxrows if memory cannot be +allocated for maxrows*sizeof\201row\202, i.e. if the size of a row is so large that +space for maxrows cannot be allocated. + +)0 P()-1 LI( The FUN_MAXBUFSIZE environment variable was added to limit the +max buffer size that will be allocated to hold table row data. The +default is 5Mb. + +)0 P()-1 LI( Generated PostScript and PDF versions of the help pages. + +)0 P()-1 LI( Moved OPTIONS section before \201often-lengthy\202 DESCRIPTION section in +man pages. + +)0 P()-1 LI( All memory allocation now does error checking on the result +\201except wcs library, which is external code\202. + +)0 P()-1 LI( Removed some compiler warnings that surfaced when using gcc -O2. + +)0 P()-1 LI( Updated wcs library to 3.5.5. + +)0 P()-1 LI( Upgraded zlib to 1.2.1. +)LU( + +)0 2 162 H()WB 248 Sn( Patch Release 1.2.3 \20112 January 2004\202)EH( +)UL( +)0 P()-1 LI( Generated man pages from the html pages. These are installed +automatically at build time. + +)0 P()-1 LI( Changed instances of sprintf\201\202 to snprintf\201\202 to protect +against buffer overflow. + +)0 P()-1 LI( Fixed a number of compiler warnings in non-ANSI compilers. + +)0 P()-1 LI( Increased SZ_LINE parameter value from 1024 to 4096. +)LU( + +)0 2 163 H()WB 249 Sn( Patch Release 1.2.3b1 \20119 August 2003\202)EH( +)UL( +)0 P()-1 LI( The rule for using comma to separate a table filter expression +and a region expression has been changed. The rule now states: +)UL()-1 LI( if both expressions contain a region, the operator used is )BD(or)ES(. +)-1 LI( if one \201or both\202 expression\201s\202 does not contain a region, the operator +used is )BD(and)ES(.)LU( +This rule handles the cases of pure regions and pure column filters properly. +It unambiguously assigns the boolean )BD(and)ES( to all mixed cases. Thus: +) 1 34 PR( foo.fits[circle\20110,10,3\202,pi=1:5])RP( +and +) 1 34 PR( foo.fits[pi=1:5,circle\20110,10,3\202])RP( +both are equivalent to: +) 1 37 PR( foo.fits[circle\20110,10,3\202 && pi=1:5])RP( + +)0 P()-1 LI( When include files are used in filters, they now have implied +parentheses surrounding them. Thus, if a region file foo.reg contains two +regions \201e.g. circle 1 2 3 and circle 4 5 6\202, the syntax: +) 1 21 PR( pha=4:5&&@foo.reg)RP( +is equivalent to: +) 1 42 PR( pha=4:5 && \201circle 1 2 3 || cir 4 5 6\202)RP( +instead of: +) 1 40 PR( pha=4:5 && circle 1 2 3 || cir 4 5 6)RP( +and the pha filter is applied to both regions. + +)0 P()-1 LI( Filters and comments now can be terminated with the string +literal "\200n" as well as ";" and the new-line character. This means +that a region can have comments embedded in it: +) 1 72 PR( funcnts foo.fits "circle 512 512 10 # color=red\200n circle 512 512 20")RP( + +)0 P()-1 LI( Added capability to update the value of an existing parameter +after writing the table or image \201assuming the output image is a +disk file or is being redirected into a file\202. + +)0 P()-1 LI( Improved handling of parentheses in filter expressions. + +)0 P( +)-1 LI( Fixed a bug in image \201not event\202 regions in which circles and +annuli with radius of 1 pixel were not being processed. No counts and +no area would be found in such regions. + +)0 P()-1 LI( Fixed a bug in funcnts in which the radii column values for out of sync +if multiple annuli were specified \201instead of a single varargs or accel +annulus\202. + +)0 P()-1 LI( By default, fundisp will display integer image data as floats +if the BSCALE and BZERO header parameters are present. + +)0 P()-1 LI( Added -L switch to funhead to output starbase list format. + +)0 P()-1 LI( Changed the name of the routine _FunColumnSelect to +FunColumnSelectArr, in order to emphasize that it is not +a private routine. + +)0 P()-1 LI( Funcalc now checks to ensure that a column was specified as part of +the expression. + +)0 P()-1 LI( Funcalc local variables in the compiled program now use a "__" prefix +to avoid conflicts with user-defined variables. + +)0 P()-1 LI( Unofficial unsigned short \201bitpix=-16\202 image data now is scaled +correctly using BSCALE and BZERO header parameters. + +)0 P()-1 LI( Ported to Intel icc and gcc 3.3 compilers. + +)0 P()-1 LI( Updated wcs library to 3.5.1. + +)0 P()-1 LI( Changed license from public domain to GNU GPL. +)LU( + +)0 2 164 H()WB 250 Sn( Patch Release 1.2.2 \20118 May 2003\202)EH( +)UL( +)0 P( +)-1 LI( Fixed funcalc so that it now actually compiles an expression and +runs it, instead of getting a "filter compilation error". Oops! + +)0 P()-1 LI( Fixed bug in FunOpen in which the bracket specification was being +removed from the filename if a disk file was opened for "w" or "a". + +)0 P()-1 LI( Fixed bug in FunFlush which prevented two successive calls to +FunImagePut from writing the second extension header properly. + +)0 P()-1 LI( All filter routines now use gerror\201stderr, ...\202 call instead of +fprintf\201stderr, ...\202 so that output to stderr can be turned off \201via +setgerror\201level\202 or GERROR environment variable\202. + +)0 P()-1 LI( All standard Funtools programs check for GERROR environment +variable before setting gerror flag. + +)0 P()-1 LI( Some error messages about invalid region arguments were not being +printed. + +)0 P()-1 LI( FITS parameters/headers now conform more closely to FITS standard: +)UL()-1 LI( Blank keywords are treated in the same way as COMMENTS and HISTORY cards +)-1 LI( XTENSION keywords are now exactly 8 characters long +)-1 LI( 'E' is output instead of 'e' in floating point param values +)-1 LI( PCOUNT and GCOUNT are output correctly for image extensions +)-1 LI( EXTEND=T is output in primary header +)-1 LI( COMMENTS and HISTORY start in column 9)LU( +)LU( + +)0 2 165 H()WB 251 Sn( Patch Release 1.2.1 \20124 April 2003\202)EH( +)UL( +)0 P()-1 LI( Varargs ellipse and box annular regions were being +processed incorrectly when the following conditions all were met: +)UL()-1 LI( the region was specified in physical or wcs coordinates +)-1 LI( the data file contained LTM/LTV keywords, i.e., it +was blocked with respect to the original data file +)-1 LI( the program being run was an image program \201e.g. funcnts, funimage\202)LU( +Varargs ellipse and boxes are regions of the form: +) 2 43 PR( ellipse x y a1 b1 a2 b2 ... an bn [angle] + box x y l1 w1 l2 w2 ... ln wn [angle])RP( +where at least 2 sets of axis \201length\202 values were specified to form +an annulus \201i.e. simple ellipses and boxes worked properly\202. With all +of the above conditions met, a region in physical coordinates saw its +second length argument converted incorrectly from physical coordinates +to image coordinates. In simple terms, this means that funcnts did not +process elliptical or box regions in physical coords on blocked images +properly. Note that blocking on the command line \201e.g. foo.fits[*,*,2]\202 +did work when no LTM/LTV keywords existed in the file. + +)0 P()-1 LI( The fundisp -f switch now supports specification of +column-specific display formats as well as a more convenient way to +specify datatype-specific display formats. Both use keyword=value +specifiers. For columns, use: +) 1 58 PR( fundisp -f "colname1=format1 colname2=format2 ..." ...)RP( +e.g. +) 1 40 PR( fundisp -f "time=%13.2f pha=%3d" ...)RP( +You also can specify display formats for individual datatypes using the FITS +binary table TFORM variables as the keywords: +) 1 65 PR( fundisp -f "D=double_format E=float_format J=int_format etc.")RP( +e.g. +) 1 35 PR( fundisp -f "D=%13.2f I=%3d" ...)RP( +The old position-dependent syntax is deprecated. + +)0 P()-1 LI( Fundisp will now print out a single 16-bit \201or 32-bit\202 unsigned +int for a column whose data format is 16X \201or 32X\202, instead of +printing 2 \201or 4\202 unsigned chars. + +)0 P()-1 LI( Fixed bug in which fundisp was not able to display bitfield data for +raw event lists. + +)0 P()-1 LI( Previously, when binning columns used implicitly in a region +and explicitly in a filter could suffer from a case sensitivity problem. +This has been fixed. + +)0 P()-1 LI( Fixed internal mask=all switch on fundisp. + +)0 P()-1 LI( Filter include files now simply include text without changing the state +of the filter. They therefore can be used in expression. That is, if foo1 +contains "pi==1" and foo2 contains "pha==2" then the following expressions +are equivalent: +) 3 57 PR( "[@foo1&&@foo2]" is equivalent to "[pi==1&&pha==2]" + "[pha==1||@foo2]" is equivalent to "[pi==1||pha==2]" + "[@foo1,@foo2]" is equivalent to "[pi==1,pha==2]")RP( + +)0 P()-1 LI( Fixed bug in filter specification which caused a SEGV if a varargs-style +region was enclosed in parens. + +)0 P()-1 LI( Updated wcs library to 3.3.2. +)LU( + +)0 2 166 H()WB 252 Sn( Public Release 1.2.0 \20124 March 2003\202)EH( +)UL( +)0 P()-1 LI( BSCALE and BZERO are now always applied to int pixel data, instead of +only being applied if the desired output is floating point. +)LU( + +)0 2 167 H()WB 253 Sn( Beta Release 1.2.b3 \2014 February 2003\202)EH( +)UL( +)0 P()-1 LI( In FunColumnSelect, added the ability to specify an offset into +an array in the type specification, using the extended syntax: +) 1 48 PR( [@][n]<type>[[poff]][:[tlmin[:tlmax[:binsiz]]]])RP( +The [poff] string specifies the offset. For example, a type specification +such as "@I[2]" specifies the third \201i.e., starting from 0\202 element in +the array pointed to by the pointer value. A value of "@2I[4]" specifies +the fifth and sixth values in the array. + +)0 P()-1 LI( Added a non-varargs version of FunColumnSelect called _FunColumnSelect: +) 3 76 PR(int _FunColumnSelect\201Fun fun, int size, char *plist, + char **names, char **types, char **modes, int *offsets, + int nargs\202;)RP( + +)0 P()-1 LI( Added support for sorting binary tables by column name using: +funtable -s "col1 col2 ... coln" ... + +)0 P()-1 LI( Added the FUN_RAW macro which, when applied to the "name" parameter +of FunParamGets\201\202, returns the 80-character raw FITS card instead of +only the value. + +)0 P()-1 LI( Added support for comparing column values with binary masks of the +form 0b[01]+, e.g.: +) 1 23 PR( \201status&0b111\202==0b001)RP( +Previously, such masks had to be specified in decimal, octal, or hex. + +)0 P()-1 LI( Completed support for type 'L' \201logical\202 in fundisp and in filtering of +binary tables. + +)0 P()-1 LI( Fixed bug in funhist that was improperly setting the number of bins +when the data was of type float. + +)0 P()-1 LI( Fixed bug in filter/Makefile where the filter OBJPATH #define was +being passed to the wrong module. +)LU( + +)0 2 168 H()WB 254 Sn( Beta Release 1.2.b2 \2017 October 2002\202)EH( +)UL( +)0 P()-1 LI( Updated wcs library to 3.1.3. + +)0 P()-1 LI( Added support for reading gzip'ed files via stdin. +)LU( + +)0 2 169 H()WB 255 Sn( Beta Release 1.2.b1 \20124 September 2002\202)EH( +)UL( +)0 P()-1 LI( Added the following accelerators to region filtering: +) 8 73 PR( shape: arguments: + ----- --------- + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn \201angle\202 + BOX xcenter ycenter xwlo yhin xwout yhhi n=[number] \201angle\202 + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn \201angle\202 + ELLIPSE xcenter ycenter xwlo yhin xwout yhhi n=[number] \201angle\202)RP( + +)0 P()-1 LI( Added the following new pandas \201Pie AND Annulus\202 to region filtering: +) 5 77 PR( shape: arguments: + ----- --------- + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # same as panda + BPANDA xcen ycen ang1 ang2 nang ixlo iylo ixhi iyhi nrad \201ang\202 # box + EPANDA xcen ycen ang1 ang2 nang ixlo iylo ixhi iyhi nrad \201ang\202 # ellipse)RP( + +)0 P()-1 LI( Added support for filtering images using simple FITS image masks, +i.e. 8-bit or 16-bit FITS images where the value of a pixel is the +region id number for that pixel \201and therefore must be greater than +0\202. The image section being filtered must either be the same size as the +mask dimensions or else be an even multiple of the mask. This works with +image-style filtering, i.e., funcnts can utilize a mask on both +images and binary tables. + +)0 P()-1 LI( Added '$n' to fundisp column specification to allow display of +ordinal value of each row passing the filter. + +)0 P()-1 LI( Added code to support region filtering on image sections. + +)0 P()-1 LI( Fixed bugs which prevented filtering more than one ASCII region file. + +)0 P()-1 LI( Fixed bug occasionally causing filter slave processes to become zombies. + +)0 P()-1 LI( Fixed bugs in event filtering: annulus with inner radius of 0 +\201i.e., a circle\202 was rejecting events with coordinates xcen, ycen. +Also, pie with angles of 0 and 360 was rejecting some events. +Image filtering \201e.g. funcnts\202 did not have these problems. + +)0 P()-1 LI( Filters now accept global exclude regions without an include region. +In such a case, the field region is implied. That is, "-circle\201x,y,r\202" +is equivalent to "field; -circle\201x,y,r\202", etc. + +)0 P()-1 LI( Fixed panda so that it can be used as a global exclude. + +)0 P()-1 LI( Allow empty ds9 region file \201comments and globals only\202 to be +a valid filter. Totally ignore zero length region or include file. + +)0 P( +)-1 LI( Fixed funcnts bug that was displaying 0 value as inner radius of +a circle, instead of just one radius value. +)LU( + +)0 2 170 H()WB 256 Sn( Public Release 1.1.0 \20122 April 2002\202)EH( + +)0 P(New features include: +)UL()0 P()-1 LI( Funtools programs now accept gzip'ed files as valid input. + +)0 P()-1 LI( Improved security via replacement of system\201\202 function. + +)0 P()-1 LI( fundisp, funcnts, funhist can output starbase/rdb format \201tabs between columns, form-feeds between tables\202. + +)0 P()-1 LI( Improved support for Windows platform, as well as new support for Mac OSX.)LU( + +)0 2 171 H()WB 257 Sn( Pre-Release 1.1.0e \20110 April 2002\202)EH( +)UL( + +)0 P()-1 LI( Added enough support to skip over variable length arrays in BINTABLES. +We will add full support if this non-standard construct becomes more widely +used. + +)0 P()-1 LI( Fixed bug in underlying fitsy _gread\201\202 routine that was returning +an arbitrary bytes-read value if the input fd was invalid. +)LU( + +)0 2 172 H()WB 258 Sn( Pre-Release 1.1.0e \20119 March 2002\202)EH( +)UL( + +)0 P()-1 LI( Added additional check for Windows/PC to filter/Nan.h. + +)0 P()-1 LI( Upgraded zlib library to 1.1.4 \201fix double free security hole\202. +)LU( + + +)0 2 173 H()WB 259 Sn( Pre-Release 1.1.0e \20127 February 2002\202)EH( +)UL( + +)0 P()-1 LI( Changed filter/process.[ch] to filter/zprocess.[ch] to avoid name +collision with Cygwin include file. + +)0 P()-1 LI( Added -a switch to funhead to display all headers in a FITS file. +)LU( + +)0 2 174 H()WB 260 Sn( Pre-Release 1.1.0e \20111 February 2002\202)EH( +)UL( + +)0 P()-1 LI( Fixed filter parser so that it ignores ds9 "ruler" and "text" markers +only up to the first \200n or ; \201was ignoring to last \200n\202. + +)0 P()-1 LI( The NBLOCK parameter in fitsy/headdata.c was too large for Mac OS X +\201max size of a declared char buf seems to be about .5 Mb\202. +)LU( + +)0 2 175 H()WB 261 Sn( Beta Release 1.0.1b5 \20131 January 2002\202)EH( +)UL( + +)0 P()-1 LI( Fixed bug introduced in calculated IRAF LTM values in 1.0.1b3. + +)0 P()-1 LI( Fixed bug in filter parser giving wrong answers when two range +lists were combined with and explicit boolean operator: +) 1 34 PR( $ fundisp $S"[x=512&&y=511,512]")RP( +incorrectly acted like: +) 1 39 PR( fundisp $S"[\201x=512&&y=511\202||\201y=512\202]")RP( +instead of: +) 1 37 PR( fundisp $S"[x=512&&\201y=511||y=512\202]")RP( +In general, we recommend use of explicit parentheses. + +)0 P()-1 LI( Fixed filter/NaN.h to recognize Compaq Alpha again \201broken by their last change to cc\202. + +)0 P()-1 LI( Removed redundant varargs definitions that conflicted with Alpha compiler definitions. + +)0 P()-1 LI( Added blank line to inc.sed to work around Apple Mac OS X bug in which the +"i" \201insert\202 command was treating final \200\200 as continuation \200 in the text. + +)0 P( +)-1 LI( Added include of mkrtemp.h to mkrtemp.c to get conditional compilation +for Mac OSX. + +)0 P()-1 LI( Added support for --with-zlib to fitsy so that ds9 could use its own +copy of zlib \201and not build the copy in fitsy\202. + +)0 P()-1 LI( Removed config.cache and Makefile files from distribution tar file. +)LU( + +)0 2 176 H()WB 262 Sn( Beta Release 1.0.1b4 \20126 January 2002\202)EH( +)UL( + +)0 P()-1 LI( Make explicit that column filters are not permitted in an image +expression \201such as the funcnts region arguments\202. + +)0 P()-1 LI( Fix bug in region parser in which a region \201without parens\202, +followed immediately by an operator: +) 1 24 PR( circle 512 512 .5&)SY(\160)ES(==1)RP( +was not processing the final argument of the region correctly. + +)0 P()-1 LI( Ignore new "tile" directive in filters \201used by ds9\202. +)LU( + +)0 2 177 H()WB 263 Sn( Beta Release 1.0.1b3 \2014 January 2002\202)EH( +)UL( + +)0 P()-1 LI( Made modifications to Makefile.in to make releases easier. + +)0 P()-1 LI( Added instructions Makefile.in so that funtools.h will always +have correct #defines for FUN_VERSION, FUN_MAJOR_VERSION, +FUN_MINOR_VERSION, and FUN_PATCH_LEVEL. + +)0 P()-1 LI( Allow #include statements in funcalc program files. + +)0 P()-1 LI( funimage now updates all 4 CDX_Y values by the block factor. + +)0 P()-1 LI( Minor changes to make funtools work under darwin \201Mac OS X\202. +)LU( + +)0 2 178 H()WB 264 Sn( Beta Release 1.0.1b2 \20114 November 2001\202)EH( +)UL( + +)0 P()-1 LI( Fixed FunOpen\201\202 bug \201introduced in b1\202 in which filenames without +extensions SEGV'ed on open. Yikes! + +)0 P()-1 LI( Funmerge now extends the tlmin/tlmax values of the output +binning columns so that merged events from widely separated files are +valid in the output table. + +)0 P()-1 LI( In funhist, added -w switch to specify bin width \201lo:hi:width\202 +instead of number of bins \201lo:hi:num\202. Added support for this new +width option in funtools.ds9. + +)0 P()-1 LI( If a tdbin value was set using bincols=\201name:tlmin:tlmax:tdbin, ...\202, +the WCS parameters were not being updated properly. + +)0 P()-1 LI( Cleaned up build support for zlib. +)LU( + +)0 2 179 H()WB 265 Sn( Beta Release 1.0.1b1 \2016 November 2001\202)EH( +)UL( + +)0 P()-1 LI( Added support for gzip'ed files to the underlying fitsy/gio +library. This means that all funtools programs now accept gzip'ed +files as valid input: +) 1 41 PR( funcnts foo.fits.gz "circle 504 512 10")RP( +It is no longer necessary to run gunzip and pipe the results to +stdin of a funtools program. + +)0 P()-1 LI( Funtools tasks are now placed in a sub-menu in the DS9 Analysis +menu, instead of at the top level. + +)0 P()-1 LI( Fixed a bug in funcnts in which the bottom-most pixel of a small +circle or annulus region could be missed when the region is only one +pixel wide for that value of y. + +)0 P()-1 LI( Added -n switch to funhist so that table histograms could be +normalized by the width of the bin \201val/\201hi_edge-lo_edge\202\202. + +)0 P()-1 LI( Added -T switch to fundisp, funcnts, funhist to output in +starbase/rdb format \201uses tabs instead of spaces between columns, +form-feeds between tables, etc.\202 + +)0 P()-1 LI( Fixed a bug in which the field\201\202 region was not being properly +processed in combination with an image section. This could affect +funcnts processing of image data where an image section was specified +\201though it usually resulted in a funcnts error\202. + +)0 P()-1 LI( Fixed bug in display of binary table header for vector columns. + +)0 P()-1 LI( Filters now recognize hex constants \201starting with 0x\202 and long +constants \201ending with L\202. + +)0 P()-1 LI(Filenames containing a ':' are now only treated as sockets if they +actually are in the form of a valid ip:port. + +)0 P()-1 LI(Replaced funtools.ds9 with a new version that calls a new funds9 +script, instead of calling funcnts or funhist directly. The new script +supports gzip'ed files and bracket specifications on filenames at the +same time, which the direct call could not. Also the new script has +better error reporting. + +)0 P()-1 LI( Replaced system\201\202 call used to compile filter and funcalc +expression with a special launch\201\202 call, which performs execvp\201\202 +directly without going through sh. \201launch\201\202 works under DOS and has +fewer security problems.\202 + +)0 P()-1 LI( Fixed image filter code in which the field\201\202 region was being ignored +if it was combined with one or more exclude regions \201and no other include +regions\202, resulting in no valid pixels. + +)0 P()-1 LI( Changed use of getdtable\201\202 to FD_SETSIZE in calls to select\201\202. + +)0 P()-1 LI( Added code to guard against FITS binary tables without proper TFORMx +parameters. + +)0 P()-1 LI( Added support to FunParamGets so that it returns the raw FITS card +if the specified input name is NULL and the input n value is positive. + +)0 P()-1 LI( Fixed bug in underlying fitsy code that set the comment in a +header parameter. +)LU( + + +)0 2 180 H()WB 266 Sn( Public Release 1.0.0 \20131 July 2001\202)EH( +)UL( +)0 P()-1 LI( "a new day with no mistakes ... yet")LU( + +)2 1 1 HR()0 0 1 A(Index to the Funtools Help Pages)0 0 TN TL()Ec /AF f D( +)0 5 181 H(Last)WB 267 Sn( updated: 22 April 2002)EH( +)WB NL +/TE t D NP TU PM 0 eq and{/Pn () D showpage}if end restore diff --git a/funtools/doc/help.html b/funtools/doc/help.html new file mode 100644 index 0000000..e32f9bf --- /dev/null +++ b/funtools/doc/help.html @@ -0,0 +1,192 @@ +<!-- =defdoc funtools funtools n --> +<HTML> +<HEAD> +<TITLE>The Funtools Help Facility</TITLE> +</HEAD> +<BODY> + +<!-- =section funtools NAME --> +<H2><A NAME="funtools">Funtools: FITS Users Need Tools</A></H2> + +<!-- =section funtools SYNOPSIS --> +<H2>Summary</H2> +This document is the Table of Contents for Funtools. + +<!-- =section funtools DESCRIPTION --> +<H2>Description</H2> +Funtools, is a "minimal buy-in" FITS library and utility package developed +at the the High Energy Astrophysics Division of SAO. The Funtools +library provides simplified access to a wide array of file types: +standard astronomical FITS images and binary tables, raw arrays and +binary event lists, and even tables of ASCII column data. A +sophisticated region filtering library (compatible with ds9) filters +images and tables using boolean operations between geometric shapes, +support world coordinates, etc. Funtools also supports advanced +capabilities such as optimized data searching using index files. + +The main goal of the Funtools project has been to develop a minimal buy-in +FITS library for researchers who are occasional (but serious) coders. In +this case, "minimal buy-in" means "easy to learn, easy to use, and easy to +re-learn next month". We have tried to achieve this goal by emphasizing two +essential capabilities. The first is the ability to develop FITS programs +without knowing much about FITS, i.e., without having to deal with the +arcane rules for generating a properly formatted FITS file. The second is +to support the use of already-familiar C/Unix facilities, especially C +structs and Unix stdio. Taken together, these two capabilities should allow +researchers to leverage their existing programming expertise while +minimizing the need to learn new and complex coding rules. + +<!-- =section funtools DESCRIPTION --> +<P> +Choose from the following topics: + +<P> +<UL> +<LI><A HREF="./programs.html">Funtools User Programs</A> +<UL> +<LI><A HREF="./programs.html#funcalc">funcalc: Funtools calculator (for binary tables)</A> +<!-- =text [funcalc(1)] --> +<LI><A HREF="./programs.html#funcen">funcen: find centroid (for binary tables)</A> +<!-- =text [funcen(1)] --> +<LI><A HREF="./programs.html#funcnts">funcnts: count photons in specified regions</A> +<!-- =text [funcnts(1)] --> +<LI><A HREF="./programs.html#funcone">funcone: cone search on RA, Dec columns</A> +<!-- =text [funcone(1)] --> +<LI><A HREF="./programs.html#fundisp">fundisp: display data in a Funtools data file</A> +<!-- =text [fundisp(1)] --> +<LI><A HREF="./programs.html#funhead">funhead: display a header in a Funtools file</A> +<!-- =text [funhead(1)] --> +<LI><A HREF="./programs.html#funhist">funhist: create a 1D histogram of a column</A> +<!-- =text [funhist(1)] --> +<LI><A HREF="./programs.html#funimage">funimage: create a FITS image from a Funtools data file</A> +<!-- =text [funimage(1)] --> +<LI><A HREF="./programs.html#funindex">funindex: create an index on a column in a binary table</A> +<!-- =text [funindex(1)] --> +<LI><A HREF="./programs.html#funjoin">funjoin: join two or more FITS binary tables on specified columns</A> +<!-- =text [funjoin(1)] --> +<LI><A HREF="./programs.html#funmerge">funmerge: merge one or more Funtools table files</A> +<!-- =text [funmerge(1)] --> +<LI><A HREF="./programs.html#funsky">funsky: convert between image and sky coordinates, using WCS info from a FITS header</A> +<!-- =text [funsky(1)] --> +<LI><A HREF="./programs.html#funtable">funtable: copy selected rows from a Funtools file to a FITS binary table</A> +<!-- =text [funtable(1)] --> +<LI><A HREF="./programs.html#funtbl">funtbl: extract a table from +Funtools ASCII output</A> +<!-- =text [funtbl(1)] --> +<LI><A HREF="./ds9.html">funtools and ds9 image display</A> +<!-- =text [funds9(n)] --> +</UL> + +<LI><A HREF="./library.html">Funtools Programming</A> +<UL> +<LI><A HREF="./library.html#summary">Funtools Programming Summary</A> +<!-- =text [funlib(3)] --> +<LI><A HREF="./library.html#tutorial">Funtools Programming Tutorial</A> +<!-- =text [funlib(3)] --> +<LI><A HREF="./library.html#order">A Short Digression on Subroutine Order</A> +<!-- =text [funlib(3)] --> +<LI><A HREF="./library.html#compiling">Compiling and Linking</A> +<!-- =text [funlib(3)] --> +<LI><A HREF="./library.html#refhandle">The Funtools Reference Handle</A> +<!-- =text [funlib(3)] --> +<LI><A HREF="./library.html#reference">The Funtools Programming Reference Manual</A> +<UL> +<LI> <A HREF="./library.html#funopen">FunOpen: open a Funtools file</A> +<!-- =text [funopen(3)] --> +<LI><A HREF="./library.html#funimageget">FunImageGet: retrieve image data</A> +<!-- =text [funimageget(3)] --> +<LI><A HREF="./library.html#funimageput">FunImagePut: output image data</A> +<!-- =text [funimageput(3)] --> +<LI><A HREF="./library.html#funimagerowget">FunImageRowGet: retrieve image data by row</A> +<!-- =text [funimagerowget(3)] --> +<LI><A HREF="./library.html#funimagerowput">FunImageRowPut: output image data by row</A> +<!-- =text [funimagerowput(3)] --> +<LI><A HREF="./library.html#funtablerowget">FunTableRowGet: retrieve rows from a table</A> +<!-- =text [funtablerowget(3)] --> +<LI><A HREF="./library.html#funtablerowput">FunTableRowPut: output rows to a table</A> +<!-- =text [funtablerowput(3)] --> +<LI><A HREF="./library.html#funcolumnselect">FunColumnSelect: select columns in a table for access</A> +<!-- =text [funcolumnselect(3)] --> +<LI><A HREF="./library.html#funcolumnactivate">FunColumnActivate: activate columns in a table for read/write</A> +<!-- =text [funcolumnactivate(3)] --> +<LI><A HREF="./library.html#funcolumnlookup">FunColumnLookup: lookup info about the columns in a table</A> +<!-- =text [funcolumnlookup(3)] --> +<LI><A HREF="./library.html#funinfoget">FunInfoGet: get info about an image or table</A> +<!-- =text [funinfoget(3)] --> +<LI><A HREF="./library.html#funinfoput">FunInfoPut: put info about an image or table</A> +<!-- =text [funinfoput(3)] --> +<LI><A HREF="./library.html#funparamget">FunParamGet: get header param</A> +<!-- =text [funparamget(3)] --> +<LI><A HREF="./library.html#funparamput">FunParamPut: put header param</A> +<!-- =text [funparamput(3)] --> +<LI><A HREF="./library.html#funflush">FunFlush: flush I/O in a Funtools file</A> +<!-- =text [funflush(3)] --> +<LI><A HREF="./library.html#funclose">FunClose: close a Funtools file</A> +<!-- =text [funclose(3)] --> +</UL> + +<LI><A HREF="./library.html#examples">Funtools Programming Examples</A> +<!-- =text [funlib(3)] --> +<UL> +<LI><A HREF="./evmerge.c">evmerge: merge new columns with existing columns</A> +<LI><A HREF="./evcol.c">evcols: add column and rows to binary tables</A> +<LI><A HREF="./imblank.c">imblank: blank out image values below a threshold</A> +</UL> +</UL> + +<LI><A HREF="./files.html">Funtools Data Files</A> +<!-- =text [funfiles(n)] --> +<UL> +<LI><A HREF="./files.html#formats">Supported Data Formats</A> +<UL> +<LI><A HREF="./files.html#fits">FITS File and Extensions</A> +<LI><A HREF="./files.html#events">Non-FITS Raw Event Files</A> +<LI><A HREF="./files.html#arrays">Non-FITS Array Files</A> +<LI><A HREF="./text.html">Column-based Text (ASCII) Files</A> +<LI><A HREF="./view.html">Database Views of Tables</A> +</UL> +<LI><A HREF="./files.html#sections">Image Sections and Blocking</A> +<LI><A HREF="./files.html#binning">Binning FITS Binary Tables and Non-FITS Event Files</H2> +<LI><A HREF="./files.html#types">Disk Files and Other Supported File Types</A> +</UL> + +<LI>Funtools Data Filtering +<UL> +<LI><A HREF="./filters.html">Table Filtering</A> +<!-- =text [funfilters(n)] --> +<LI><A HREF="./idx.html">Fast Table Filtering using Indexes</A> +<!-- =text [funidx(n)] --> +<LI><A HREF="./regions.html">Spatial Region Filtering</A> +<!-- =text [funregions(n)] --> +<UL> +<LI><A HREF="./reggeometry.html">Region Geometry</A> +<!-- =text [reggeometry(n)] --> +<LI><A HREF="./regalgebra.html">Region Algebra</A> +<!-- =text [regalgebra(n)] --> +<LI><A HREF="./regcoords.html">Region Coordinates</A> +<!-- =text [regcoords(n)] --> +<LI><A HREF="./regbounds.html">Region Boundaries</A> +<!-- =text [regbounds(n)] --> +<LI><A HREF="./regdiff.html">Differences Between Funtools and IRAF Regions</A> +<!-- =text [regdiff(n)] --> +</UL> +<LI><A HREF="./combo.html">Combining Table and Region Filters</A> +<!-- =text [funcombine(n)] --> +</UL> + +<LI> Miscellaneous +<UL> +<LI><A HREF="./env.html">Funtools Environment Variables</A> +<!-- =text [funenv(n)] --> +<LI><A HREF="./changelog.html">Funtools ChangeLog</A> +</UL> + +</UL> + +<!-- =stop --> + +<H5>Last updated: January 6, 2006</H5> + +</BODY> +</HTML> + diff --git a/funtools/doc/html2man b/funtools/doc/html2man new file mode 100755 index 0000000..4e60799 --- /dev/null +++ b/funtools/doc/html2man @@ -0,0 +1,258 @@ +#!/usr/bin/perl +# +# See COPYRIGHT +# +# Script to generate a pod file from an html source (the same one as for text files too) +# and later this pod file it passed through pod2man +# +# Use: +# html2man [ <man-dir> [<version-dir>] ] <file.html +# +# <Man-dir> is the directory where the man pages will be created +# (current directory by default). If a file name is given instead of +# directory then the directory of that file is used. +# <Version-dir> is the directory containing the ttf2pt1 files version.h +# and CHANGES.html which are used to generate the release name and date +# for the man page (by default looks in current directory and then in up to +# 5 ancestor directories). +# If the version files can not be found then the release defaults to +# "current" and the date defaults to today. +# +# Special formatting in the html file is: +# All controls are hidden within HTML comments that must occupy a whole separate line +# Such a line looks like: +# <!-- =<html2man_directive> <arguments> --> +# <!-- ==<pod_directive> <arguments> --> +# Any sort of directive must be followed by a space. The pod directives are +# automatically surrounded by empty lines in the output file. +# The html2man directives are: +# +# <!-- =defdoc <docid> <file> <section> --> +# Define a man page. Multiple man pages can be defined in the same HTML +# file. <Docid> is a short name by which this man page will be referred in the +# other directives. <File> is the name of the man page, and <section> is the +# section of the manual (do not confuse with sections within a man page). +# +# <!-- =section <docid> <page_section_name> --> +# All the text following this directive is copied (with translation) +# into the specified section of the specified man page. The sections +# may appear in arbitrary order, they will be rearranged to the standard +# order before output. Only standard section names are permitted (see @stdsect +# below). The pod directives which occur outside of man sections are ignored, +# just like the common text. The translation of HTML tags is: +# +# <br> - to paragraph break +# <b> - to B<> +# <i> - to I<> +# <tt> - to C<> +# <a href> - to F<> +# <ul>, <li>, </ul> - to =over 2, =item *, =back +# , &, <, > - to their symbols, appropriately encoded +# +# The rest of HTML tags is removed +# +# If the same section is started more than once, the text from the +# second appearance will be added to the first, etc. +# +# <!-- =stop --> +# Stop copying text to the man page. +# +# <!-- =cont --> +# Continue copying text to the man page, same section as before. +# +# <!-- =text <text> --> +# Insert this <text> into the man page (works only when copying is enabled). +# Characters <, >, & are converted as usual. + +@mons = qw(January February March April May June July August September October November December); + +$dir = $ARGV[0]; +$maindir = $ARGV[1]; + +if($dir eq "") { + $dir = "."; +} elsif( ! -d $dir ) { + if( ! ($dir =~ s|\/[^/]*$||) ) { + $dir = "."; + } +} +if($maindir eq "") { + $maindir = "."; + for($i=0; $i<5; $i++) { + if(-f "$maindir/version.h") { + last; + } + $maindir = "../$maindir"; + } +} + +if( open(VERFILE, "<$maindir/version.h") ) { + while(<VERFILE>) { + if( /^\s*\#define\s+TTF2PT1_VERSION\s+\"(.*)\"/ ) { + $release = "version $1"; + } + } + close(VERFILE); + if( $release =~ /SNAP-([0-9][0-9])([0-9][0-9])([0-9][0-9])/ ) { + $date = sprintf("%s %d, 20%02d", $mons[$2-1], $3, $1); + } elsif( open(CFILE, "<$maindir/CHANGES.html") ) { + while(<CFILE>) { + if( /\<H4\>/) { + last; + } + } + $_ = <CFILE>; + chomp; + if( $_ =~ s/^.*?-- // ) { + $date = $_; + } + close(CFILE); + } +} + +if($release eq "") { + if( open(VERFILE, "<../Makefile") ) { + while(<VERFILE>) { + if( /^VERSION\s+=\s+(.*)/ ) { + $release = "version $1"; + } + } + close(VERFILE); + } +} + +if($release eq "") { + $release = "current"; +} +if($date eq "") { + @lt = localtime(time); + $date = sprintf("%s %d, %d", $mons[$lt[4]], $lt[3], 1900+$lt[5]); +} + +#printf(STDERR "date=%s release=%s\n", $date, $release); + +$writemode = 0; + +while(<STDIN>) { + if( s/^\<\!\-\- \=(\S+)\s+//) { + $cmd = $1; + s/\s*\-\-\>\s*$//; + #printf(STDERR "cmd=%s args=%s\n", $cmd, $_); + if($cmd =~ /^=/) { + if($writemode) { + $text{$tosect} .= "\n\n$cmd $_\n\n"; + } + } elsif($cmd eq "defdoc") { + @sl = split; + push(@allids, $sl[0]); + $file{$sl[0]} = $sl[1]; + $mansect{$sl[0]} = $sl[2]; + } elsif($cmd eq "section") { + # tosect includes the file id + $tosect = $_; + $text{$tosect} .= "\n\n"; + $writemode = 1; + } elsif($cmd eq "stop") { + $writemode = 0; + $text{$tosect} .= "\n"; + } elsif($cmd eq "cont") { + $writemode = 1; + } elsif($cmd eq "text") { + if($writemode) { + s/\<\;/</gi; + s/\>\;/>/gi; + s/\&\;/\&/gi; + $text{$tosect} .= "$_\n"; + } + } + } elsif($writemode) { +# s/^\s+//; + + s/\{/\&lbr;/g; + s/\}/\&rbr;/g; + + s/\<br\>/\n\n/gi; + #s/\<blockquote\>/\n\n=over 4\n\n/gi; + #s/\<\/blockquote\>/\n\n=back\n\n/gi; + s/\<ul\>/\n\n=over 4\n\n/gi; + s/\<\/ul\>/\n\n=back\n\n/gi; + s/\<li\>\s*/\n\n=item \*\n\n/gi; + + s/\<dl\>/\n\n=over 4\n\n/gi; + s/\<\/dl\>/\n\n=back\n\n/gi; + s/\<dt\>\s*/\n\n=item \*\n\n/gi; + s/\<dd\>\s*/\n\n/gi; + + s/\<i\>(.*?)\<\/i\>/I\{\1\}/gi; + s/\<em\>(.*?)\<\/em\>/I\{\1\}/gi; + s/\<b\>(.*?)\<\/b\>/B\{\1\}/gi; + s/\<tt\>(.*?)\<\/tt\>/C\{\1\}/gi; + s/\<a href\=\.*?\>(.*?)\<\/a\>/F\{\1\}/gi; + s/\<h2\>summary\<\/h2\>//gi; + s/\<h2\>description\<\/h2\>//gi; + s/\<h2\>examples\<\/h2\>//gi; + s/\<h2\>options\<\/h2\>//gi; + s/\<h2\>(.*?)\<\/h2\>/B\{\1\}/gi; + s/\<.*?\>//g; + s/\{/\</g; + s/\}/\>/g; + + s/\ \;/S< >/gi; + s/\&\;/\&/gi; +# s/\<\;/E<lt>/gi; +# s/\>\;/E<gt>/gi; + s/\<\;/\</gi; + s/\>\;/\>/gi; + #s/\|/E<verbar>/g; + #s/\//E<sol>/g; + s/\&lbr\;/\{/g; + s/\&rbr\;/\}/g; + + #printf(STDERR "section=%s add=%s", $tosect, $_); + $text{$tosect} .= $_; + } +} + +@stdsect = ( + "NAME", + "SYNOPSIS", + "OPTIONS", + "DESCRIPTION", + "RETURN VALUE", + "ERRORS", + "EXAMPLES", + "ENVIRONMENT", + "FILES", + "SEE ALSO", + "NOTES", + "CAVEATS", + "DIAGNOSTICS", + "BUGS", + "RESTRICTIONS", + "AUTHOR", + "HISTORY" ); + +#printf(STDERR "allids= @allids\n"); +for $id (@allids) { + print(STDERR "creating man page $id $file{$id} $mansect{$id}\n\n"); + die "Unable to create pod file $dir/$file{$id}.pod" + unless open(PODF, ">./pod/$file{$id}.pod"); + print(PODF "=pod\n\n"); + for $sect (@stdsect) { + $sid = "$id $sect"; + #printf(STDERR "trying %s\n", $sid); + if(defined $text{$sid}) { + #printf(STDERR " section %s\n", $sid); + print(PODF "=head1 $sect\n\n$text{$sid}\n\n"); + } + } + print(PODF "=cut\n"); + close(PODF); + die "Unable to generate the man page $dir/$file{$id}.1" + if system("pod2man --section=\"$mansect{$id}\" --release=\"$release\" " + . "--center=\"SAORD Documentation\" --date=\"$date\" " + . "--name=\"$file{$id}\" " + . "./pod/$file{$id}.pod > $dir/man$mansect{$id}/$file{$id}.$mansect{$id}"); + + unlink("$dir/$file{$id}.pod"); +} diff --git a/funtools/doc/idx.html b/funtools/doc/idx.html new file mode 100644 index 0000000..a08b10b --- /dev/null +++ b/funtools/doc/idx.html @@ -0,0 +1,216 @@ +<!-- =defdoc funidx funidx n --> +<HTML> +<HEAD> +<TITLE>Table Filtering with Indexes</TITLE> +</HEAD> +<BODY> + +<!-- =section funidx NAME --> +<H2><A NAME="funidx">Funidx: Using Indexes to Filter Rows in a Table</A></H2> + +<!-- =section funidx SYNOPSIS --> +<H2>Summary</H2> +<P> +This document contains a summary of the user interface for +filtering rows in binary tables with indexes. + +<!-- =section funidx DESCRIPTION --> +<H2>Description</H2> +<P> +Funtools <A HREF="./filters.html">Table Filtering</A> allows rows in a +table to be selected based on the values of one or more columns in the +row. Because the actual filter code is compiled on the fly, it is very +efficient. However, for very large files (hundreds of Mb or larger), +evaluating the filter expression on each row can take a long time. Therefore, +funtools supports index files for columns, which are used automatically during +filtering to reduce dramatically the number of row evaluations performed. +The speed increase for indexed filtering can be an order of magnitude or +more, depending on the size of the file. + +<P> +The <A HREF="./programs.html#funindex">funindex</A> program creates an +index on one or more columns in a binary table. For example, to create an index +for the column pi in the file huge.fits, use: +<PRE> + funindex huge.fits pi +</PRE> +This will create an index named huge_pi.idx. + +<P> +When a filter expression is initialized for row evaluation, funtools +looks for an index file for each column in the filter expression. If +found, and if the file modification date of the index file is later +than that of the data file, then the index will be used to reduce the +number of rows that are evaluated in the filter. When +<A HREF="./regions.html">Spatial Region Filtering</A> is part of the +expression, the columns associated with the region are checked for index +files. + +<P> +If an index file is not available for a given column, then in general, +all rows must be checked when that column is part of a filter +expression. This is not true, however, when a non-indexed column is +part of an AND expression. In this case, only the rows that pass the +other part of the AND expression need to be checked. Thus, in some cases, +filtering speed can increase significantly even if all columns are not +indexed. + +<P> +Also note that certain types of filter expression syntax cannot make +use of indices. For example, calling functions with column names as +arguments implies that all rows must be checked against the function +value. Once again, however, if this function is part of an AND +expression, then a significant improvement in speed still is possible +if the other part of the AND expression is indexed. + +<P> +For example, note below the dramatic speedup in searching a 1 Gb +file using an AND filter, even when one of the columns (pha) has no +index: + +<PRE> + time fundisp \ + huge.fits'[idx_activate=0,idx_debug=1,pha=2348&&cir 4000 4000 1]' \ + "x y pha" + x y pha + ---------- ----------- ---------- + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 42.36u 13.07s 6:42.89 13.7% + + time fundisp \ + huge.fits'[idx_activate=1,idx_debug=1,pha=2348&&cir 4000 4000 1]' \ + "x y pha" + x y pha + ---------- ----------- ---------- + idxeq: [INDEF] + idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] + idxand(1): INDEF [IDX_OR_SORT] + idxall(1): [IDX_OR_SORT] + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 1.55u 0.37s 1:19.80 2.4% +</PRE> + +When all columns are indexed, the increase in speed can be even more dramatic: +<PRE> + time fundisp \ + huge.fits'[idx_activate=0,idx_debug=1,pi=770&&cir 4000 4000 1]' \ + "x y pi" + x y pi + ---------- ----------- ---------- + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 42.60u 12.63s 7:28.63 12.3% + + time fundisp \ + huge.fits'[idx_activate=1,idx_debug=1,pi=770&&cir 4000 4000 1]' \ + "x y pi" + x y pi + ---------- ----------- ---------- + idxeq: pi start=9473025,stop=9492240 => pi[ROW 9473025:9492240] + idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] + idxor sort/merge: pi[ROW 9473025:9492240] [IDX_OR_SORT] + idxmerge(5): [IDX_OR_SORT] pi[ROW] + idxall(1): [IDX_OR_SORT] + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 1.67u 0.30s 0:24.76 7.9% +</PRE> + +<P> +The miracle of indexed filtering (and indeed, of any indexing) is the +speed of the binary search on the index, which is of order log2(n) +instead of n. (The funtools binary search method is taken from +http://www.tbray.org/ongoing/When/200x/2003/03/22/Binary, to whom +grateful acknowledgement is made.) This means that the larger the +file, the better the performance. Conversely, it also means that for +small files, using an index (and the overhead involved) can slow +filtering down somewhat. Our tests indicate that on a file containing +a few tens of thousands of rows, indexed filtering can be 10 to 20 +percent slower than non-indexed filtering. Of course, your mileage +will vary with conditions (disk access speed, amount of available +memory, process load, etc.) + +<p> +Any problem encountered during index processing will result in +indexing being turned off, and replaced by filtering all rows. You can turn +filtering off manually by setting the idx_activate variable to 0 (in a filter +expression) or the FILTER_IDX_ACTIVATE environment variable to 0 (in the global +environment). Debugging output showing how the indexes are being processed can +be displayed to stderr by setting the idx_debug variable to 1 (in a filter +expression) or the FILTER_IDX_DEBUG environment variable to 1 (in the global +environment). + +<p> +Currently, indexed filtering only works with FITS binary tables and raw +event files. It does not work with text files. This restriction might be +removed in a future release. + +<!-- =section funidx SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: August 3, 2007</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/imblank.c b/funtools/doc/imblank.c new file mode 100644 index 0000000..bdd736e --- /dev/null +++ b/funtools/doc/imblank.c @@ -0,0 +1,106 @@ +#include <funtools.h> +#include <stdlib.h> + +#ifdef ANSI_FUNC +int +main (int argc, char **argv) +#else +main(argc, argv) + int argc; + char **argv; +#endif +{ + int i; + int bitpix, dim1, dim2; + int total; + double blimit, bvalue; + char *buf; + unsigned char *cbuf; + short *sbuf; + int *ibuf; + float *fbuf; + double *dbuf; + Fun fun, fun2; + + if( argc < 4 ){ + fprintf(stderr, "usage: %s iname oname blimit bvalue\n", argv[0]); + exit(1); + } + + /* get blank limit and optional blank value */ + blimit = atof(argv[3]); + bvalue = 0; + if( argc >= 5 ) + bvalue = atof(argv[4]); + + /* exit on gio errors */ + setgerror(2); + + /* open the input FITS file */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + + /* open the output FITS image, preparing to copy input params */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + + /* extract and bin the data section into an image buffer */ + if( !(buf = FunImageGet(fun, NULL, NULL)) ) + gerror(stderr, "could not FunImageGet: %s\n", argv[1]); + + /* get required information from funtools structure. + this should come after the ImageGet call, in case that call + changed fun_sect_bitpix value */ + FunInfoGet(fun, + FUN_SECT_BITPIX, &bitpix, + FUN_SECT_DIM1, &dim1, + FUN_SECT_DIM2, &dim2, + 0); + + /* set appropriate data type buffer to point to image buffer */ + switch(bitpix){ + case 8: + cbuf = (unsigned char *)buf; break; + case 16: + sbuf = (short *)buf; break; + case 32: + ibuf = (int *)buf; break; + case -32: + fbuf = (float *)buf; break; + case -64: + dbuf = (double *)buf; break; + } + + /* loop through pixels and reset values below limit to value */ + total = dim1*dim2; + for(i=0; i<total; i++){ + switch(bitpix){ + case 8: + if( cbuf[i] <= blimit ) cbuf[i] = bvalue; + break; + case 16: + if( sbuf[i] <= blimit ) sbuf[i] = bvalue; + break; + case 32: + if( ibuf[i] <= blimit ) ibuf[i] = bvalue; + break; + case -32: + if( fbuf[i] <= blimit ) fbuf[i] = bvalue; + break; + case -64: + if( dbuf[i] <= blimit ) dbuf[i] = bvalue; + break; + } + } + + /* write the output image, updating the FITS header from the orig file */ + if( !FunImagePut(fun2, buf, 0, 0, 0, NULL) ) + gerror(stderr, "could not FunImagePut: %s\n", argv[2]); + + /* free up space */ + if( buf ) free(buf); + + /* close output first so that flush happens automatically */ + FunClose(fun2); + FunClose(fun); +} diff --git a/funtools/doc/library.html b/funtools/doc/library.html new file mode 100644 index 0000000..9bc725d --- /dev/null +++ b/funtools/doc/library.html @@ -0,0 +1,2823 @@ +<!-- =defdoc funlib funlib 3 --> +<HTML> +<HEAD> +<TITLE>Funtools Programming</TITLE> +</HEAD> +<BODY> + +<!-- =section funlib NAME --> +<H2><A NAME="funlib">FunLib: the Funtools Programming Interface</A></H2> + +<!-- =section funlib SYNOPSIS --> +<H2>Summary</H2> +A description of the Funtools library. + +<!-- =section funlib DESCRIPTION --> +<H2><A NAME="intro">Introduction to the Funtools Programming Interface</H2></A> +<P> +To create a Funtools application, you need to include +the funtools.h definitions file in your code: +<PRE> + #include <funtools.h> +</PRE> + +You then call Funtools subroutines and functions to access Funtools data. +The most important routines are: +<UL> + +<P> +<LI> <A HREF="./library.html#funopen">FunOpen</A>: open a Funtools file +<LI><A HREF="./library.html#funinfoget">FunInfoGet</A>: get info about an image or table +<LI><A HREF="./library.html#funimageget">FunImageGet</A>: retrieve image data +<LI><A HREF="./library.html#funimagerowget">FunImageRowGet</A>: retrieve image data by row +<LI><A HREF="./library.html#funimageput">FunImagePut</A>: output image data +<LI><A HREF="./library.html#funimagerowput">FunImageRowPut</A>: output image data by row +<LI><A HREF="./library.html#funcolumnselect">FunColumnSelect</A>: select columns in a table for access +<LI><A HREF="./library.html#funtablerowget">FunTableRowGet</A>: retrieve rows from a table +<LI><A HREF="./library.html#funtablerowput">FunTableRowPut</A>: output rows to a table +<LI> <A HREF="./library.html#funclose">FunClose</A>: close a Funtools file +</UL> + +Your program must be linked against the libfuntools.a library, +along with the math library. The following libraries also might be required +on your system: +<UL> +<LI> -lsocket -lnsl for socket support +<LI> -ldl for dynamic loading +</UL> +<P> +For example, on a Solaris system using gcc, use the following link line: +<PRE> + gcc -o foo foo.c -lfuntools -lsocket -lnsl -ldl -lm +</PRE> +On a Solaris system using Solaris cc, use the following link line: +<PRE> + gcc -o foo foo.c -lfuntools -lsocket -lnsl -lm +</PRE> +On a Linux system using gcc, use the following link line: +<PRE> + gcc -o foo foo.c -lfuntools -ldl -lm +</PRE> +Once configure has built a Makefile on your platform, the required +"extra" libraries (aside from -lm, which always is required) are +specified in that file's EXTRA_LIBS variable. For example, under +Linux you will find: +<PRE> + grep EXTRA_LIBS Makefile + EXTRA_LIBS = -ldl + ... +</PRE> + +<P> +The Funtools library contains both the zlib library +(http://www.gzip.org/zlib/) and Doug Mink's WCS library +(http://tdc-www.harvard.edu/software/wcstools/). It is not necessary +to put these libraries on a Funtools link line. Include files +necessary for using these libraries are installed in the Funtools +include directory. + +<H2><A NAME="tutorial">Funtools Programming Tutorial</A></H2> + +The +<A HREF="./library.html#funopen">FunOpen()</A> +function is used to open a FITS file, an array, or a raw event file: +<PRE> + /* open the input FITS file for reading */ + ifun = FunOpen(iname, "r", NULL); + /* open the output FITS file for writing, and connect it to the input file */ + ofun = FunOpen(iname, "w", ifun); +</PRE> +A new output file can inherit header parameters automatically from +existing input file by passing the input Funtools handle as the last +argument to the new file's +<A HREF="./library.html#funopen">FunOpen()</A> +call as shown above. + +<P> +For image data, you then can call +<A HREF="./library.html#funimageget">FunImageGet()</A> +to read an image into memory. +<PRE> + float buf=NULL; + /* extract and bin the data section into an image buffer */ + buf = FunImageGet(fun, NULL, "bitpix=-32"); +</PRE> +If the (second) buf argument to this call is NULL, buffer space is allocated +automatically. The (third) plist argument can be used to specify the +return data type of the array. If NULL is specified, the data type of +the input file is used. + +<P> +To process an image buffer, you would generally make a call to +<A HREF="./library.html#funinfoget">FunInfoGet()</A> to determine the +dimensions of the image (which may have been changed from the original +file dimensions due to <A HREF="./files.html#sections">Funtools image +sectioning</A> on the command line). In a FITS image, the index along +the dim1 axis varies most rapidly, followed by the dim2 axis, etc. +Thus, to access each pixel in an 2D image, use a double loop such as: +</PRE> + buf = FunImageGet(fun, NULL, "bitpix=-32"); + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + for(i=1; i<=dim2; i++){ + for(j=1; j<=dim1; j++){ + ... process buf[((i-1)*dim1)+(j-1)] ... + } + } +</PRE> +or: +<PRE> + buf = FunImageGet(fun, NULL, "bitpix=-32"); + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + for(i=0; i<(dim1*dim2); i++){ + ... process buf[i] ... + } +</PRE> +Finally, you can write the resulting image to disk using +<A HREF="./library.html#funimageput">FunImagePut()</A>: +<PRE> + FunImagePut(fun2, buf, dim1, dim2, -32, NULL); +</PRE> +Note that Funtools automatically takes care of book-keeping tasks such as +reading and writing FITS headers (although you can, of course, write +your own header or add your own parameters to a header). + +<P> +For binary tables and raw event files, a call to +<A HREF="./library.html#funopen">FunOpen()</A> +will be followed by a call to the +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +routine to select columns to be read from the input file and/or +written to the output file: + +<PRE> + typedef struct evstruct{ + double time; + int time2; + } *Ev, EvRec; + FunColumnSelect(fun, sizeof(EvRec), NULL, + "time", "D", "rw", FUN_OFFSET(Ev, time), + "time2", "J", "w", FUN_OFFSET(Ev, time2), + NULL); +</PRE> +Columns whose (third) mode argument contains an "r" are "readable", +i.e., columns will be read from the input file and converted into the +data type specified in the call's second argument. These columns +values then are stored in the specified offset of the user record +structure. Columns whose mode argument contains a "w" are +"writable", i.e., these values will be written to the output file. +The +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +routine also offers the option of automatically merging user +columns with the original input columns when writing the output +rows. + +<P> +Once a set of columns has been specified, you can retrieve rows using +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A>, +and write the rows using +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A>: +<PRE> + Ev ebuf, ev; + /* get rows -- let routine allocate the array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + /* time2 is generated here */ + ev->time2 = (int)(ev->time+.5); + /* change the input time as well */ + ev->time = -(ev->time/10.0); + } + /* write out this batch of rows with the new column */ + FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL); + /* free row data */ + if( ebuf ) free(ebuf); + } +</PRE> +The input rows are retrieved into an array of user structs, which +can be accessed serially as shown above. Once again, Funtools +automatically takes care of book-keeping tasks such as reading and writing +FITS headers (although you can, of course, write your own header or +add your own parameters to a header). + +<P> +When all processing is done, you can call +<A HREF="./library.html#funclose">FunClose()</A> +to close the file(s): +<PRE> + FunClose(fun2); + FunClose(fun); +</PRE> + +<P> +These are the basics of processing FITS files (and arrays or raw event +data) using Funtools. The routines in these examples are described in +more detail below, along with a few other routines that support +parameter access, data flushing, etc. + +<H2><A NAME="compiling">Compiling and Linking</A></H2> +<P> +To create a Funtools application, a software developer will include +the funtools.h definitions file in Funtools code: +<PRE> + #include <funtools.h> +</PRE> +The program is linked against the libfuntools.a library, along with the +math library (and the dynamic load library, if the latter is available +on your system): +<PRE> + gcc -o foo foo.c -lfuntools -ldl -lm +</PRE> +<P> +If gcc is used, Funtools filtering can be performed using dynamically +loaded shared objects that are built at run-time. Otherwise, filtering +is performed using a slave process. +<P> +Funtools has been built on the following systems: +<UL> +<LI> Sun/Solaris 5.X +<LI> Linux/RedHat Linux 5.X,6.X,7.X +<LI> Dec Alpha/OSF1 V4.X +<LI> WindowsNT/Cygwin 1.0 +<LI> SGI/IRIX64 6.5 +</UL> + +<H2><A NAME="order">A Short Digression on Subroutine Order</A></H2> +<P> +There is a natural order for all I/O access libraries. You would not +think of reading a file without first opening it, or writing a file +after closing it. A large part of the experiment in funtools is to use +the idea of "natural order" as a means of making programming +easier. We do this by maintaining the state of processing for a given +funtools file, so that we can do things like write headers and flush +extension padding at the right time, without you having to do it. + +<P> +For example, if you open a new funtools file for writing using +<A HREF="./library.html#funopen">FunOpen()</A>, +then generate an array of image data and call +<A HREF="./library.html#funimageput">FunImagePut()</A>, +funtools knows to write the image header automatically. +There is no need to think about writing a standard header. +Of course, you can add parameters to the file first by +calling one of the +<A HREF="./library.html#funparamput">FunParamPut()</A> +routines, and these parameters will automatically be added +to the header when it is written out. There still is no +need to write the header explicitly. + +<P> +Maintaining state in this way means that there are certain rules of +order which should be maintained in any funtools program. In particular, +we strongly recommend the following ordering rules be adhered to: + +<UL> +<LI> When specifying that input extensions be copied to an output file +via a reference handle, open the output file <B>before</B> reading the +input file. (Otherwise the initial copy will not occur). + +<LI> Always write parameters to an output file using one of the +<A HREF="./library.html#funparamput">FunParamPut()</A> calls +<B>before</B> writing any data. (This is a good idea for all FITS +libraries, to avoid having to recopy data is the FITS header needs +to be extended by adding a single parameter.) + +<LI> If you retrieve an image, and need to know the data +type, use the FUN_SECT_BITPIX option of +<A HREF="./library.html#funinfoget">FunInfoGet()</A>, +<B>after</B> calling +<A HREF="./library.html#funimageget">FunImageGet()</A>, since +it is possible to change the value of BITPIX from the latter. + +<LI> When specifying that input extensions be copied to an output file +via a reference handle, close the output file <B>before</B> closing +input file, or else use +<A HREF="./library.html#funflush">FunFlush()</A> +explicitly on the output file +<B>before</B> closing the input file. (Otherwise the final copy will +not occur). +</UL> + +<P> +We believe that these are the natural rules that are implied in most +FITS programming tasks. However, we recognize that making explicit use +of "natural order" to decide what automatic action to take on behalf +of the programmer is experimental. Therefore, if you find that your +needs are not compatible with our preferred order, please let us know +-- it will be most illuminating for us as we evaluate this experiment. + +<H2><A NAME="examples">Funtools Programming Examples</A></H2> +<P> +The following complete coding examples are provided to illustrate the +simplicity of Funtools applications. They can be found in the funtest +subdirectory of the Funtools distribution. In many cases, you should +be able to modify one of these programs to generate your own Funtools +program: +<UL> +<LI><A HREF="./evread.c">evread.c</A>: read and write binary tables +<LI><A HREF="./evcol.c">evcols.c</A>: add column and rows to binary tables +<LI><A HREF="./evmerge.c">evmerge.c</A>: merge new columns with existing columns +<LI><A HREF="./evnext.c">evnext.c</A>: manipulate raw data pointers +<LI><A HREF="./imblank.c">imblank.c</A>: blank out image values below a threshold +<LI><A HREF="./asc2fits.c">asc2fits.c</A>: convert a specific ASCII table to FITS binary table +</UL> + +<H2><A NAME="reference">The Funtools Programming Reference Manual</A></H2> +<P> +<PRE> +#include <funtools.h> + +Fun <A HREF="./library.html#funopen">FunOpen(char *name, char *mode, Fun ref)</A> + +void *<A HREF="./library.html#funimageget">FunImageGet(Fun fun, void *buf, char *plist)</A> + +int <A HREF="./library.html#funimageput">FunImagePut(Fun fun, void *buf, int dim1, int dim2, int bitpix, char *plist)</A> + +void * <A HREF="./library.html#funimagerowget">FunImageRowGet(Fun fun, void *buf, int rstart, int rstop, char *plist)</A> + +void * <A HREF="./library.html#funimagerowput">FunImageRowPut(Fun fun, void *buf, int rstart, int rstop, int dim1, int dim2, int bitpix, char *plist)</A> + +int <A HREF="./library.html#funcolumnselect">FunColumnSelect(Fun fun, int size, char *plist, ...)</A> + +void <A HREF="./library.html#funcolumnactivate">FunColumnActivate(Fun fun, char *s, char *plist)</A> + +int <A HREF="./library.html#funcolumnlookup">FunColumnLookup(Fun fun, char *s, int which, char **name, int *type, int *mode, int *offset, int *n, int *width)</A> + +void *<A HREF="./library.html#funtablerowget">FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, int *nrow)</A> + +int <A HREF="./library.html#funtablerowput">FunTableRowPut(Fun fun, void *rows, int nev, int idx, char *plist)</A> + +int <A HREF="./library.html#funparamget">FunParamGetb(Fun fun, char *name, int n, int defval, int *got)</A> + +int <A HREF="./library.html#funparamget">FunParamGeti(Fun fun, char *name, int n, int defval, int *got)</A> + +double <A HREF="./library.html#funparamget">FunParamGetd(Fun fun, char *name, int n, double defval, int *got)</A> + +char *<A HREF="./library.html#funparamget">FunParamGets(Fun fun, char *name, int n, char *defval, int *got)</A> + +int <A HREF="./library.html#funparamput">FunParamPutb(Fun fun, char *name, int n, int value, char *comm, int append)</A> + +int <A HREF="./library.html#funparamput">FunParamPuti(Fun fun, char *name, int n, int value, char *comm, int append)</A> + +int <A HREF="./library.html#funparamput">FunParamPutd(Fun fun, char *name, int n, double value, int prec, char *comm, int append)</A> + +int <A HREF="./library.html#funparamput">FunParamPuts(Fun fun, char *name, int n, char *value, char *comm, int append)</A> + +int <A HREF="./library.html#funinfoget">FunInfoGet(Fun fun, int type, ...)</A> + +int <A HREF="./library.html#funinfoput">FunInfoPut(Fun fun, int type, ...)</A> + +void <A HREF="./library.html#funflush">FunFlush(Fun fun, char *plist)</A> + +void <A HREF="./library.html#funclose">FunClose(Fun fun)</A> +</PRE> + +<!-- =defdoc funopen funopen 3 --> + +<!-- =section funopen NAME --> +<H2><A NAME="funopen">FunOpen - open a Funtools data file</A></H2> + +<!-- =section funopen SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + Fun FunOpen(char *name, char *mode, Fun ref); +</PRE> +</B> + +<!-- =section funopen DESCRIPTION --> +<P> +The <B>FunOpen()</B> routine opens a Funtools data file for reading or +appending, or creates a new FITS file for writing. The <B>name</B> +argument specifies the name of the Funtools data file to open. You can +use IRAF-style bracket notation to specify +<A HREF="./files.html">Funtools Files, Extensions, and Filters</A>. +A separate call should be made each time a different FITS extension is +accessed: +<PRE> + Fun fun; + char *iname; + ... + if( !(fun = FunOpen(iname, "r", NULL)) ){ + fprintf(stderr, "could not FunOpen input file: %s\n", iname); + exit(1); + } +</PRE> +<P> +If <B>mode</B> is "r", the file is opened for reading, and processing +is set up to begin at the specified extension. For reading, +<B>name</B> can be <B>stdin</B>, in which case the standard input is read. + +<P> +If <B>mode</B> is "w", the file is created if it does not exist, or +opened and truncated for writing if it does exist. Processing starts +at the beginning of the file. The <B>name</B> can be <B>stdout</B>, +in which case the standard output is readied for processing. + +<P> +If <B>mode</B> is "a", the file is created if it does not exist, or +opened if it does exist. Processing starts at the end of the file. +The <B>name</B> can be <B>stdout</B>, in which case the standard +output is readied for processing. + +<P> +When a Funtools file is opened for writing or appending, a previously +opened <A HREF="./library.html#refhandle">Funtools reference +handle</A> can be specified as the third argument. This handle +typically is associated with the input Funtools file that will be used +to generate the data for the output data. When a reference file is +specified in this way, the output file will inherit the (extension) +header parameters from the input file: +<PRE> + Fun fun, fun2; + ... + /* open input file */ + if( !(fun = FunOpen(argv[1], "r", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + /* open the output FITS image, inheriting params from input */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); +</PRE> +Thus, in the above example, the output FITS binary table file will +inherit all of the parameters associated with the input binary table +extension. +<P> +A file opened for writing with a +<A HREF="./library.html#refhandle">Funtools reference handle</A> also +inherits the selected columns (i.e. those columns chosen for +processing using the +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> routine) +from the reference file as its default columns. This makes it easy to +open an output file in such a way that the columns written to the +output file are the same as the columns read in the input file. Of +course, column selection can easily be tailored using the +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> routine. +In particular, it is easy to merge user-defined columns with the input +columns to generate a new file. See the +<A HREF="./evmerge.c">evmerge</A> for a complete example. + +<P> +In addition, when a +<A HREF="./library.html#refhandle">Funtools reference handle</A> +is supplied in a <A HREF="./library.html#funopen">FunOpen()</A> call, +it is possible also to specify that all other extensions from the +reference file (other than the input extension being processed) should +be copied from the reference file to the output file. This is useful, +for example, in a case where you are processing a FITS binary table +or image and you want to copy all of the other extensions to +the output file as well. Copy of other extensions is controlled by +adding a "C" or "c" to the mode string of the +<A HREF="./library.html#funopen">FunOpen()</A> call <B>of the input +reference file</B>. If "C" is specified, then other extensions are +<B>always</B> copied (i.e., copy is forced by the application). If +"c" is used, then other extensions are copied if the user requests +copying by adding a plus sign "+" to the extension name in the bracket +specification. For example, the <B>funtable</B> program utilizes +"c" mode, giving users the option of copying all other extensions: +<PRE> + /* open input file -- allow user copy of other extensions */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + /* open the output FITS image, inheriting params from input */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); +</PRE> + +Thus, <B>funtable</B> supports either of these command lines: +<PRE> + # copy only the EVENTS extension + csh> funtable "test.ev[EVENTS,circle(512,512,10)]" foo.ev + # copy ALL extensions + csh> funtable "test.ev[EVENTS+,circle(512,512,10)]" foo.ev +</PRE> + +<P> +Use of a <A HREF="./library.html#refhandle">Funtools reference +handle</A> implies that the input file is opened before the output +file. However, it is important to note that if copy mode ("c" or "C") +is specified for the input file, the actual input file open is delayed +until just after the output file is opened, since the copy of prior +extensions to the output file takes place while Funtools is seeking to +the specified input extension. This implies that the output file +should be opened before any I/O is done on the input file or else the +copy will fail. Note also that the copy of subsequent extension will +be handled automatically by +<A HREF="./library.html#funclose">FunClose()</A> +if the output file is +closed before the input file. Alternatively, it can be done explicitly +by <A HREF="./library.html#funflush">FunFlush()</A>, but again, this +assumes that the input file still is open. + +<P> +Upon success <A HREF="./library.html#funopen">FunOpen()</A> returns a +Fun handle that is used in subsequent Funtools calls. On error, NULL +is returned. + +<!-- =defdoc funimageget funimageget 3 --> + +<!-- =section funimageget NAME --> +<H2><A NAME="funimageget">FunImageGet - get an image or image section</A></H2> + +<!-- =section funimageget SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void *FunImageGet(Fun fun, void *buf, char *plist) +</PRE> +</B> + +<!-- =section funimageget DESCRIPTION --> +<P> +The <B>FunImageGet()</B> routine returns an binned image array of the +specified section of a Funtools data file. If the input data are +already of type image, the array is generated by extracting the +specified image section and then binning it according to the specified +bin factor. If the input data are contained in a binary table or raw +event file, the rows are binned on the columns specified by the +<B>bincols=</B> keyword (using appropriate default columns as +necessary), after which the image section and bin factors are +applied. In both cases, the data is automatically converted from FITS +to native format, if necessary. +<P> +The first argument is the Funtools handle returned by +<A HREF="./library.html#funopen">FunOpen()</A>. The second <B>buf</B> +argument is a pointer to a data buffer to fill. If NULL is specified, +FunImageGet will allocate a buffer of the appropriate size. Generally +speaking, you always want Funtools to allocate the buffer because +the image dimensions will be determined by +<A HREF="./files.html#sections">Funtools image sectioning</A> +on the command line. +<P> +The third <B>plist</B> (i.e., parameter list) argument is a string +containing one or more comma-delimited <B>keyword=value</B> +parameters. It can be used to specify the return data type using the +<B>bitpix=</B> keyword. If no such keyword is specified in the plist +string, the data type of the returned image is the same as the data type +of the original input file, or is of type int for FITS binary tables. + +<P> +If the <B>bitpix=</B> keyword is supplied in the plist string, the data +type of the returned image will be one of the supported FITS image +data types: +<UL> +<LI> 8 unsigned char +<LI> 16 short +<LI> 32 int +<LI> -32 float +<LI> -64 double +</UL> +For example: +<PRE> + void *buf; + /* extract data section into an image buffer */ + if( !(buf = FunImageGet(fun, NULL, NULL)) ) + gerror(stderr, "could not FunImageGet: %s\n", iname); +</PRE> +will allocate buf and retrieve the image in the file data format. In +this case, you will have to determine the data type (using the +FUN_SECT_BITPIX value in the +<A HREF="./library.html#funinfoget">FunInfoGet()</A> +routine) +and then use a switch statement to process each data type: +<PRE> + int bitpix; + void *buf; + unsigned char *cbuf; + short *sbuf; + int *ibuf; + ... + buf = FunImageGet(fun, NULL, NULL); + FunInfoGet(fun, FUN_SECT_BITPIX, &bitpix, 0); + /* set appropriate data type buffer to point to image buffer */ + switch(bitpix){ + case 8: + cbuf = (unsigned char *)buf; break; + case 16: + sbuf = (short *)buf; break; + case 32: + ibuf = (int *)buf; break; + ... +</PRE> +See the +<A HREF="./imblank.c">imblank example code</A> +for more details on how to process an image when the data type is not +specified beforehand. + +<P> +It often is easier to specify the data type directly: +<PRE> + double *buf; + /* extract data section into a double image buffer */ + if( !(buf = FunImageGet(fun, NULL, "bitpix=-64")) ) + gerror(stderr, "could not FunImageGet: %s\n", iname); +</PRE> +will extract the image while converting to type double. + +<P> +On success, a pointer to the image buffer is returned. (This will be +the same as the second argument, if NULL is not passed to the latter.) +On error, NULL is returned. + +<P> +In summary, to retrieve image or row data into a binned image, you simply +call FunOpen() followed by +<A HREF="./library.html#funimageget">FunImageGet()</A>. Generally, you +then will want to call +<A HREF="./library.html#funinfoget">FunInfoGet()</A> +to retrieve the +axis dimensions (and data type) of the section you are processing +(so as to take account of sectioning and blocking of the original data): +<PRE> + double *buf; + int i, j; + int dim1, dim2; + ... other declarations, etc. + + /* open the input FITS file */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + + /* extract and bin the data section into a double float image buffer */ + if( !(buf = FunImageGet(fun, NULL, "bitpix=-64")) ) + gerror(stderr, "could not FunImageGet: %s\n", argv[1]); + + /* get dimension information from funtools structure */ + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + + /* loop through pixels and reset values below limit to value */ + for(i=0; i<dim1*dim2; i++){ + if( buf[i] <= blimit ) buf[i] = bvalue; + } +</PRE> + +<P> +Another useful plist string value is "mask=all", which returns an +image populated with regions id values. Image pixels within a region +will contain the associated region id (region values start at 1), and +otherwise will contain a 0 value. Thus, the returned image is a +region mask which can be used to process the image data (which +presumably is retrieved by a separate call to FunImageGet) pixel by +pixel. + +<P> +If a FITS binary table or a non-FITS raw event file is being binned +into an image, it is necessary to specify the two columns that will be +used in the 2D binning. This usually is done on the command line +using the <B>bincols=(x,y)</B> keyword: +<PRE> + funcnts "foo.ev[EVENTS,bincols=(detx,dety)]" +</PRE> + +<P> +The full form of the <B>bincols=</B> specifier is: +<PRE> + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]) +</PRE> +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: +<PRE> + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) +</PRE> +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters (respectively) are present in the +FITS binary table header for the column in question. Note that if +only one parameter is specified, it is assumed to be tlmax, and tlmin +defaults to 1. If two parameters are specified, they are assumed to be +tlmin and tlmax. + +<P> +If <B>bincols</B> is not specified on the command line, Funtools tries +to use appropriate defaults: it looks for the environment variable +FITS_BINCOLS (or FITS_BINKEY). Then it looks for the Chandra +parameters CPREF (or PREFX) in the FITS binary table header. Failing +this, it looks for columns named "X" and "Y" and if these are not +found, it looks for columns containing the characters "X" and "Y". +<P> +See <A HREF="./files.html#binning">Binning FITS Binary Tables and +Non-FITS Event Files</A> for more information. + +<!-- =defdoc funimageput funimageput 3 --> + +<!-- =section funimageput NAME --> +<H2><A NAME="funimageput">FunImagePut - put an image to a Funtools file</A></H2> +<!-- =section funimageput SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunImagePut(Fun fun, void *buf, int dim1, int dim2, int bitpix, + char *plist) +</PRE> +</B> + +<!-- =section funimageput DESCRIPTION --> +The <B>FunImagePut()</B> routine outputs an image array to a FITS +file. The image is written either as a primary header/data unit or as +an image extension, depending on whether other data have already been +written to the file. That is, if the current file position is at the +beginning of the file, a primary HDU is written. Otherwise, an +image extension is written. + +<P> +The first argument is the Funtools handle returned by +<A HREF="./library.html#funopen">FunOpen()</A>. The second <B>buf</B> +argument is a pointer to a data buffer to write. The <B>dim1</B>and +<B>dim2</B> arguments that follow specify the dimensions of the image, +where dim1 corresponds to naxis1 and dim2 corresponds to naxis2. The +<B>bitpix</B> argument specifies the data type of the image and can +have the following FITS-standard values: +<UL> +<LI> 8 unsigned char +<LI> 16 short +<LI> 32 int +<LI> -32 float +<LI> -64 double +</UL> + +<P> +When <A HREF="./library.html#funtablerowput">FunTableRowPut()</A> is first +called for a given image, Funtools checks to see if the primary header +has already been written (by having previously written an image or a +binary table.) If not, this image is written to the primary HDU. +Otherwise, it is written to an image extension. +<P> +Thus, a simple program to generate a FITS image might look like this: +<PRE> + int i; + int dim1=512, dim2=512; + double *dbuf; + Fun fun; + dbuf = malloc(dim1*dim2*sizeof(double)); + /* open the output FITS image, preparing to copy input params */ + if( !(fun = FunOpen(argv[1], "w", NULL)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[1]); + for(i=0; i<(dim1*dim2); i++){ + ... fill dbuf ... + } + /* put the image (header will be generated automatically */ + if( !FunImagePut(fun, buf, dim1, dim2, -64, NULL) ) + gerror(stderr, "could not FunImagePut: %s\n", argv[1]); + FunClose(fun); + free(dbuf); +</PRE> + +<P> +In addition, if a +<A HREF="./library.html#refhandle">Funtools reference handle</A> +was specified when this table was opened, the +parameters from this +<A HREF="./library.html#refhandle">Funtools reference handle</A> +are merged into the new image +header. Furthermore, if a reference image was specified during +<A HREF="./library.html#funopen">FunOpen()</A>, the values of +<B>dim1</B>, <B>dim2</B>, and <B>bitpix</B> in the calling sequence +can all be set to 0. In this case, default values are taken from the +reference image section. This is useful if you are reading an image +section in its native data format, processing it, and then writing +that section to a new FITS file. See the +<A HREF="./imblank.c">imblank example code</A>. + +<P> +The data are assumed to be in the native machine format and will +automatically be swapped to FITS big-endian format if necessary. This +behavior can be over-ridden with the <B>convert=[true|false]</B> +keyword in the <B>plist</B> param list string. + +<p> +When you are finished writing the image, you should call +<A HREF="./library.html#funflush">FunFlush()</A> to write out the FITS +image padding. However, this is not necessary if you subsequently call +FunClose() without doing any other I/O to the FITS file. + +<!-- =defdoc funimagerowget funimagerowget 3 --> + +<!-- =section funimagerowget NAME --> +<H2><A NAME="funimagerowget">FunImageRowGet - get row(s) of an image</A></H2> + +<!-- =section funimagerowget SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void *FunImageRowGet(Fun fun, void *buf, int rstart, int rstop, + char *plist) +</PRE> +</B> + +<!-- =section funimagerowget DESCRIPTION --> +<P> +The <B>FunImageRowGet()</B> routine returns one or more image rows +from the specified section of a Funtools data file. If the input data +are of type image, the array is generated by extracting the specified +image rows and then binning them according to the specified bin +factor. If the input data are contained in a binary table or raw +event file, the rows are binned on the columns specified by the +<B>bincols=</B> keyword (using appropriate default columns as needed), +after which the image section and bin factors are applied. + +<P> +The first argument is the Funtools handle returned by +<A HREF="./library.html#funopen">FunOpen()</A>. The second <B>buf</B> +argument is a pointer to a data buffer to fill. If NULL is specified, +FunImageGet() will allocate a buffer of the appropriate size. + +<P> +The third and fourth arguments specify the first and last row to +retrieve. Rows are counted starting from 1, up to the value of +FUN_YMAX(fun). The final <B>plist</B> (i.e., parameter list) argument +is a string containing one or more comma-delimited +<B>keyword=value</B> parameters. It can be used to specify the return +data type using the <B>bitpix=</B> keyword. If no such keyword is +specified in the plist string, the data type of the image is the same +as the data type of the original input file, or is of type int for +FITS binary tables. + +<P> +If the <B>bitpix=</B>value is supplied in the plist string, the data +type of the returned image will be one of the supported FITS image +data types: +<UL> +<LI> 8 unsigned char +<LI> 16 short +<LI> 32 int +<LI> -32 float +<LI> -64 double +</UL> + +<P> +For example: +<PRE> + double *drow; + Fun fun; + ... open files ... + /* get section dimensions */ + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + /* allocate one line's worth */ + drow = malloc(dim1*sizeof(double)); + /* retrieve and process each input row (starting at 1) */ + for(i=1; i <= dim2; i++){ + if( !FunImageRowGet(fun, drow, i, i, "bitpix=-64") ) + gerror(stderr, "can't FunImageRowGet: %d %s\n", i, iname); + /* reverse the line */ + for(j=1; j<=dim1; j++){ + ... process drow[j-1] ... + } + } + ... +</PRE> + +<P> +On success, a pointer to the image buffer is returned. (This will be +the same as the second argument, if NULL is not passed to the latter.) +On error, NULL is returned. Note that the considerations described +above for specifying binning columns in +<A HREF="./library.html#funimageget">FunImageGet()</A> also apply to +<B>FunImageRowGet()</B>. + +<!-- =defdoc funimagerowput funimagerowput 3 --> + +<!-- =section funimagerowput NAME --> +<H2><A NAME="funimagerowput">FunImageRowPut - put row(s) of an image</A></H2> + +<!-- =section funimagerowput SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void *FunImageRowPut(Fun fun, void *buf, int rstart, int rstop, + int dim1, int dim2, int bitpix, char *plist) +</PRE> +</B> + +<!-- =section funimagerowput DESCRIPTION --> +<P> +The <B>FunImageRowPut()</B> routine writes one or more image rows to +the specified FITS image file. The first argument is the Funtools +handle returned by <A HREF="./library.html#funopen">FunOpen()</A>. +The second <B>buf</B> argument is a pointer to the row data buffer, +while the third and fourth arguments specify the starting and ending +rows to write. Valid rows values range from 1 to dim2, i.e., row is +one-valued. + +<P> +The <B>dim1</B>and <B>dim2</B> arguments that follow specify the +dimensions, where dim1 corresponds to naxis1 and dim2 corresponds to +naxis2. The <B>bitpix</B> argument data type of the image and can +have the following FITS-standard values: +<UL> +<LI> 8 unsigned char +<LI> 16 short +<LI> 32 int +<LI> -32 float +<LI> -64 double +</UL> + +For example: +<PRE> + double *drow; + Fun fun, fun2; + ... open files ... + /* get section dimensions */ + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + /* allocate one line's worth */ + drow = malloc(dim1*sizeof(double)); + /* retrieve and process each input row (starting at 1) */ + for(i=1; i <= dim2; i++){ + if( !FunImageRowGet(fun, drow, i, i, "bitpix=-64") ) + gerror(stderr, "can't FunImageRowGet: %d %s\n", i, iname); + ... process drow ... + if( !FunImageRowPut(fun2, drow, i, i, 64, NULL) ) + gerror(stderr, "can't FunImageRowPut: %d %s\n", i, oname); + } + ... +</PRE> + +<P> +The data are assumed to be in the native machine format and will +automatically be swapped to big-endian FITS format if necessary. This +behavior can be over-ridden with the <B>convert=[true|false]</B> +keyword in the <B>plist</B> param list string. + +<p> +When you are finished writing the image, you should call +<A HREF="./library.html#funflush">FunFlush()</A> to write out the FITS +image padding. However, this is not necessary if you subsequently call +FunClose() without doing any other I/O to the FITS file. + +<!-- =defdoc funcolumnselect funcolumnselect 3 --> + +<!-- =section funcolumnselect NAME --> +<H2><A NAME="funcolumnselect">FunColumnSelect - select Funtools columns</A></H2> +<!-- =section funcolumnselect SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunColumnSelect(Fun fun, int size, char *plist, + char *name1, char *type1, char *mode1, int offset1, + char *name2, char *type2, char *mode2, int offset2, + ..., + NULL) + + int FunColumnSelectArr(Fun fun, int size, char *plist, + char **names, char **types, char **modes, + int *offsets, int nargs); +</PRE> +</B> + +<!-- =section funcolumnselect DESCRIPTION --> +The <B>FunColumnSelect()</B> routine is used to select the columns +from a Funtools binary table extension or raw event file for +processing. This routine allows you to specify how columns in a file +are to be read into a user record structure or written from a user +record structure to an output FITS file. + +<P> +The first argument is the Fun handle associated with this set of +columns. The second argument specifies the size of the user record +structure into which columns will be read. Typically, the sizeof() +macro is used to specify the size of a record structure. The third +argument allows you to specify keyword directives for the selection +and is described in more detail below. + +<P> +Following the first three required arguments is a variable length list of +column specifications. Each column specification will consist of four +arguments: +<UL> +<LI> <B>name</B>: the name of the column + +<LI> <B>type</B>: the data type of the column as it will be stored in +the user record struct (not the data type of the input file). The +following basic data types are recognized: +<UL> +<LI>A: ASCII characters +<LI>B: unsigned 8-bit char +<LI>I: signed 16-bit int +<LI>U: unsigned 16-bit int (not standard FITS) +<LI>J: signed 32-bit int +<LI>V: unsigned 32-bit int (not standard FITS) +<LI>E: 32-bit float +<LI>D: 64-bit float +</UL> +The syntax used is similar to that which defines the TFORM parameter +in FITS binary tables. That is, a numeric repeat value can precede +the type character, so that "10I" means a vector of 10 short ints, "E" +means a single precision float, etc. Note that the column value from +the input file will be converted to the specified data type as the +data is read by +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A>. + +<P> +[ A short digression regarding bit-fields: Special attention is +required when reading or writing the FITS bit-field type +("X"). Bit-fields almost always have a numeric repeat character +preceding the 'X' specification. Usually this value is a multiple of 8 +so that bit-fields fit into an integral number of bytes. For all +cases, the byte size of the bit-field B is (N+7)/8, where N is the +numeric repeat character. + +<p> +A bit-field is most easily declared in the user struct as an array of +type char of size B as defined above. In this case, bytes are simply +moved from the file to the user space. If, instead, a short or int +scalar or array is used, then the algorithm for reading the bit-field +into the user space depends on the size of the data type used along +with the value of the repeat character. That is, if the user data +size is equal to the byte size of the bit-field, then the data is +simply moved (possibly with endian-based byte-swapping) from one to +the other. If, on the other hand, the data storage is larger than the +bit-field size, then a data type cast conversion is performed to move +parts of the bit-field into elements of the array. Examples will help +make this clear: + +<UL> +<LI> If the file contains a 16X bit-field and user space specifies a 2B +char array[2], then the bit-field is moved directly into the char array. + +<LI> If the file contains a 16X bit-field and user space specifies a 1I +scalar short int, then the bit-field is moved directly into the short int. + +<LI> If the file contains a 16X bit-field and user space specifies a 1J +scalar int, then the bit-field is type-cast to unsigned int before +being moved (use of unsigned avoids possible sign extension). + +<LI> If the file contains a 16X bit-field and user space specifies a 2J +int array[2], then the bit-field is handled as 2 chars, each of which +are type-cast to unsigned int before being moved (use of unsigned +avoids possible sign extension). + +<LI> If the file contains a 16X bit-field and user space specifies a 1B +char, then the bit-field is treated as a char, i.e., truncation will +occur. + +<LI> If the file contains a 16X bit-field and user space specifies a 4J +int array[4], then the results are undetermined. + +</UL> +For all user data types larger than char, the bit-field is byte-swapped +as necessary to convert to native format, so that bits in the +resulting data in user space can be tested, masked, etc. in the same +way regardless of platform.] + +<P> +In addition to setting data type and size, the <B>type</B> +specification allows a few ancillary parameters to be set, using the +full syntax for <B>type</B>: +<PRE> + [@][n]<type>[[['B']poff]][:[tlmin[:tlmax[:binsiz]]]] +</PRE> + +<P> +The special character "@" can be prepended to this specification to +indicated that the data element is a pointer in the user record, +rather than an array stored within the record. + +<P> +The [n] value is an integer that specifies the +number of elements that are in this column (default is 1). TLMIN, +TLMAX, and BINSIZ values also can be specified for this column after +the type, separated by colons. If only one such number is specified, +it is assumed to be TLMAX, and TLMIN and BINSIZ are set to 1. + +<P> +The [poff] value can be used to specify the offset into an +array. By default, this offset value is set to zero and the data +specified starts at the beginning of the array. The offset usually +is specified in terms of the data type of the column. Thus an offset +specification of [5] means a 20-byte offset if the data type is a +32-bit integer, and a 40-byte offset for a double. If you want to +specify a byte offset instead of an offset tied to the column data type, +precede the offset value with 'B', e.g. [B6] means a 6-bye offset, +regardless of the column data type. + +The [poff] is especially useful in conjunction with the pointer @ +specification, since it allows the data element to anywhere stored +anywhere in the allocated array. For example, a specification such as +"@I[2]" specifies the third (i.e., starting from 0) element in the +array pointed to by the pointer value. A value of "@2I[4]" specifies +the fifth and sixth values in the array. For example, consider the +following specification: <PRE> + + typedef struct EvStruct{ + short x[4], *atp; + } *Event, EventRec; + /* set up the (hardwired) columns */ + FunColumnSelect( fun, sizeof(EventRec), NULL, + "2i", "2I ", "w", FUN_OFFSET(Event, x), + "2i2", "2I[2]", "w", FUN_OFFSET(Event, x), + "at2p", "@2I", "w", FUN_OFFSET(Event, atp), + "at2p4", "@2I[4]", "w", FUN_OFFSET(Event, atp), + "atp9", "@I[9]", "w", FUN_OFFSET(Event, atp), + "atb20", "@I[B20]", "w", FUN_OFFSET(Event, atb), + NULL); +</PRE> +Here we have specified the following columns: +<UL> +<LI> 2i: two short ints in an array which is stored as part the +record +<LI> 2i2: the 3rd and 4th elements of an array which is stored +as part of the record +<LI> an array of at least 10 elements, not stored in the record but +allocated elsewhere, and used by three different columns: +<UL> +<LI> at2p: 2 short ints which are the first 2 elements of the allocated array +<LI> at2p4: 2 short ints which are the 5th and 6th elements of +the allocated array +<LI> atp9: a short int which is the 10th element of the allocated array +</UL> +<LI> atb20: a short int which is at byte offset 20 of another allocated array +</UL> +In this way, several columns can be specified, all of which are in a +single array. <B>NB</B>: it is the programmer's responsibility to +ensure that specification of a positive value for poff does not point +past the end of valid data. + +<LI> <B>read/write mode</B>: "r" means that the column is read from an +input file into user space by +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A>, "w" means that +the column is written to an output file. Both can specified at the same +time. + +<LI> <B>offset</B>: the offset into the user data to store +this column. Typically, the macro FUN_OFFSET(recname, colname) is used +to define the offset into a record structure. +</UL> + +<P> +When all column arguments have been specified, a final NULL argument +must added to signal the column selection list. + +<P> +As an alternative to the varargs +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +routine, a non-varargs routine called +<A HREF="./library.html#funcolumnselect">FunColumnSelectArr()</A> +also is available. The first three arguments (fun, size, plist) of this +routine are the same as in +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A>. +Instead of a variable +argument list, however, +<A HREF="./library.html#funcolumnselect">FunColumnSelectArr()</A> +takes 5 additional arguments. The first 4 arrays arguments contain the +names, types, modes, and offsets, respectively, of the columns being +selected. The final argument is the number of columns that are +contained in these arrays. It is the user's responsibility to free +string space allocated in these arrays. + +<P> +Consider the following example: +<PRE> + typedef struct evstruct{ + int status; + float pi, pha, *phas; + double energy; + } *Ev, EvRec; + + FunColumnSelect(fun, sizeof(EvRec), NULL, + "status", "J", "r", FUN_OFFSET(Ev, status), + "pi", "E", "r", FUN_OFFSET(Ev, pi), + "pha", "E", "r", FUN_OFFSET(Ev, pha), + "phas", "@9E", "r", FUN_OFFSET(Ev, phas), + NULL); +</PRE> +<P> +Each time a row is read into the Ev struct, the "status" column is +converted to an int data type (regardless of its data type in the +file) and stored in the status value of the struct. Similarly, "pi" +and "pha", and the phas vector are all stored as floats. Note that the +"@" sign indicates that the "phas" vector is a pointer to a 9 element +array, rather than an array allocated in the struct itself. The row +record can then be processed as required: +<PRE> + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ev->pi = (ev->pi+.5); + ev->pha = (ev->pi-.5); + } +</PRE> + +<P> +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +can also be called to define "writable" columns in order to generate a FITS +Binary Table, without reference to any input columns. For +example, the following will generate a 4-column FITS binary table when +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A> is used to +write Ev records: + +<PRE> + typedef struct evstruct{ + int status; + float pi, pha + double energy; + } *Ev, EvRec; + + FunColumnSelect(fun, sizeof(EvRec), NULL, + "status", "J", "w", FUN_OFFSET(Ev, status), + "pi", "E", "w", FUN_OFFSET(Ev, pi), + "pha", "E", "w", FUN_OFFSET(Ev, pha), + "energy", "D", "w", FUN_OFFSET(Ev, energy), + NULL); +</PRE> +All columns are declared to be write-only, so presumably the column +data is being generated or read from some other source. + +<P> +In addition, +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +can be called to define <B>both</B> "readable" and "writable" columns. +In this case, the "read" columns +are associated with an input file, while the "write" columns are +associated with the output file. Of course, columns can be specified as both +"readable" and "writable", in which case they are read from input +and (possibly modified data values are) written to the output. +The +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +call itself is made by passing the input Funtools handle, and it is +assumed that the output file has been opened using this input handle +as its +<A HREF="./library.html#refhandle">Funtools reference handle</A>. + +<P> +Consider the following example: +<PRE> + typedef struct evstruct{ + int status; + float pi, pha, *phas; + double energy; + } *Ev, EvRec; + + FunColumnSelect(fun, sizeof(EvRec), NULL, + "status", "J", "r", FUN_OFFSET(Ev, status), + "pi", "E", "rw", FUN_OFFSET(Ev, pi), + "pha", "E", "rw", FUN_OFFSET(Ev, pha), + "phas", "@9E", "rw", FUN_OFFSET(Ev, phas), + "energy", "D", "w", FUN_OFFSET(Ev, energy), + NULL); +</PRE> +As in the "read" example above, each time an row is read into the Ev +struct, the "status" column is converted to an int data type +(regardless of its data type in the file) and stored in the status +value of the struct. Similarly, "pi" and "pha", and the phas vector +are all stored as floats. Since the "pi", "pha", and "phas" variables +are declared as "writable" as well as "readable", they also will be +written to the output file. Note, however, that the "status" variable +is declared as "readable" only, and hence it will not be written to +an output file. Finally, the "energy" column is declared as +"writable" only, meaning it will not be read from the input file. In +this case, it can be assumed that "energy" will be calculated in the +program before being output along with the other values. + +<P> +In these simple cases, only the columns specified as "writable" will +be output using +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A>. However, +it often is the case that you want to merge the user columns back in +with the input columns, even in cases where not all of the input +column names are explicitly read or even known. For this important +case, the <B>merge=[type]</B> keyword is provided in the plist string. + +<P> +The <B>merge=[type]</B> keyword tells Funtools to merge the columns from +the input file with user columns on output. It is normally used when +an input and output file are opened and the input file provides the +<A HREF="./library.html#refhandle">Funtools reference handle</A> +for the output file. In this case, each time +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A> is called, the +raw input rows are saved in a special buffer. If +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A> then is called +(before another call to +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A>), the contents +of the raw input rows are merged with the user rows according to the +value of <B>type</B> as follows: + +<UL> +<LI> <B>update</B>: add new user columns, and update value of existing ones (maintaining the input data type) + +<LI> <B>replace</B>: add new user columns, and replace the data type +and value of existing ones. (Note that if tlmin/tlmax values are not +specified in the replacing column, but are specified in the original +column being replaced, then the original tlmin/tlmax values are used +in the replacing column.) + +<LI> <B>append</B>: only add new columns, do not "replace" or "update" existing ones +</UL> + +<P> +Consider the example above. If <B>merge=update</B> is specified in the +plist string, then "energy" will be added to the input columns, and +the values of "pi", "pha", and "phas" will be taken from the user +space (i.e., the values will be updated from the original values, if +they were changed by the program). The data type for "pi", "pha", and +"phas" will be the same as in the original file. If +<B>merge=replace</B> is specified, both the data type and value of +these three input columns will be changed to the data type and value +in the user structure. If <B>merge=append</B> is specified, none of +these three columns will be updated, and only the "energy" column will +be added. Note that in all cases, "status" will be written from the +input data, not from the user record, since it was specified as read-only. + +<P> +Standard applications will call +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +to define user columns. However, if this routine is not called, the +default behavior is to transfer all input columns into user space. For +this purpose a default record structure is defined such that each data +element is properly aligned on a valid data type boundary. This +mechanism is used by programs such as fundisp and funtable to process +columns without needing to know the specific names of those columns. +It is not anticipated that users will need such capabilities (contact +us if you do!) + +<p> +By default, <A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +reads/writes rows to/from an "array of structs", where each struct contains +the column values for a single row of the table. This means that the +returned values for a given column are not contiguous. You can +set up the IO to return a "struct of arrays" so that each of the +returned columns are contiguous by specifying <B>org=structofarrays</B> +(abbreviation: <B>org=soa</B>) in the plist. +(The default case is <B>org=arrayofstructs</B> or <B>org=aos</B>.) + +<P> +For example, the default setup to retrieve rows from a table would be +to define a record structure for a single event and then call + <A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +as follows: +<PRE> + typedef struct evstruct{ + short region; + double x, y; + int pi, pha; + double time; + } *Ev, EvRec; + + got = FunColumnSelect(fun, sizeof(EvRec), NULL, + "x", "D:10:10", mode, FUN_OFFSET(Ev, x), + "y", "D:10:10", mode, FUN_OFFSET(Ev, y), + "pi", "J", mode, FUN_OFFSET(Ev, pi), + "pha", "J", mode, FUN_OFFSET(Ev, pha), + "time", "1D", mode, FUN_OFFSET(Ev, time), + NULL); +</PRE> +Subsequently, each call to +<HREF="./library.html#funtablerowget">FunTableRowGet()</A> +will return an array of structs, one for each returned row. If instead you +wanted to read columns into contiguous arrays, you specify <B>org=soa</B>: +<PRE> + typedef struct aevstruct{ + short region[MAXROW]; + double x[MAXROW], y[MAXROW]; + int pi[MAXROW], pha[MAXROW]; + double time[MAXROW]; + } *AEv, AEvRec; + + got = FunColumnSelect(fun, sizeof(AEvRec), "org=soa", + "x", "D:10:10", mode, FUN_OFFSET(AEv, x), + "y", "D:10:10", mode, FUN_OFFSET(AEv, y), + "pi", "J", mode, FUN_OFFSET(AEv, pi), + "pha", "J", mode, FUN_OFFSET(AEv, pha), + "time", "1D", mode, FUN_OFFSET(AEv, time), + NULL); +</PRE> +Note that the only modification to the call is in the plist string. + +<P> +Of course, instead of using staticly allocated arrays, you also can specify +dynamically allocated pointers: +<PRE> + /* pointers to arrays of columns (used in struct of arrays) */ + typedef struct pevstruct{ + short *region; + double *x, *y; + int *pi, *pha; + double *time; + } *PEv, PEvRec; + + got = FunColumnSelect(fun, sizeof(PEvRec), "org=structofarrays", + "$region", "@I", mode, FUN_OFFSET(PEv, region), + "x", "@D:10:10", mode, FUN_OFFSET(PEv, x), + "y", "@D:10:10", mode, FUN_OFFSET(PEv, y), + "pi", "@J", mode, FUN_OFFSET(PEv, pi), + "pha", "@J", mode, FUN_OFFSET(PEv, pha), + "time", "@1D", mode, FUN_OFFSET(PEv, time), + NULL); +</PRE> +Here, the actual storage space is either allocated by the user or by the +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> call). + +<P> +In all of the above cases, the same call is made to retrieve rows, e.g.: +<PRE> + buf = (void *)FunTableRowGet(fun, NULL, MAXROW, NULL, &got); +</PRE> +However, the individual data elements are accessed differently. +For the default case of an "array of structs", the +individual row records are accessed using: +<PRE> + for(i=0; i<got; i++){ + ev = (Ev)buf+i; + fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + ev->x, ev->y, ev->pi, ev->pha, ev->dx, ev->dy, ev->time); + } +</PRE> +For a struct of arrays or a struct of array pointers, we have a single struct +through which we access individual columns and rows using: +<PRE> + aev = (AEv)buf; + for(i=0; i<got; i++){ + fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + aev->x[i], aev->y[i], aev->pi[i], aev->pha[i], + aev->dx[i], aev->dy[i], aev->time[i]); + } +</PRE> +Support for struct of arrays in the +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A> +call is handled analogously. + +<P> +See the <A HREF="./evread.c">evread example code</A> +and +<A HREF="./evmerge.c">evmerge example code</A> +for working examples of how +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> is used. + +<!-- =defdoc funcolumnactivate funcolumnactivate 3 --> + +<!-- =section funcolumnactivate NAME --> +<H2><A NAME="funcolumnactivate">FunColumnActivate - activate Funtools columns</A></H2> + +<!-- =section funcolumnactivate SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void FunColumnActivate(Fun fun, char *s, char *plist) +</PRE> +</B> + +<!-- =section funcolumnactivate DESCRIPTION --> +<P> +The <B>FunColumnActivate()</B> routine determines which columns (set up +by <A HREF="./library.html#funcolumnselect">FunColumnSelect()</A>) +ultimately will be read and/or written. By default, all columns that +are selected using +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +are activated. The +<A HREF="./library.html#funcolumnactivate">FunColumnActivate()</A> +routine can be used to turn off/off activation of specific columns. + +<P> +The first argument is the Fun handle associated with this set of +columns. The second argument is a space-delimited list of columns to +activate or de-activate. Columns preceded by "+" are activated and +columns preceded by a "-" are de-activated. If a column is named +without "+" or "-", it is activated. The reserved strings "$region" +and '$n' are used to activate a special columns containing the filter +region value and row value, respectively, associated with +this row. For example, if a filter containing two circular regions is +specified as part of the Funtools file name, this column will contain +a value of 1 or 2, depending on which region that row was in. The +reserved strings "$x" and "$y" are used to activate the current +binning columns. Thus, if the columns DX and DY are specified as +binning columns: +<PRE> + [sh $] fundisp foo.fits[bincols=(DX,DY)] +</PRE> +then "$x" and "$y" will refer to these columns in a call to +<A HREF="./library.html#funcolumnactivate">FunColumnActivate()</A>. + +<P> +In addition, if the activation string contains only columns to be +activated, then the routine will de-activate all other columns. +Similarly, if the activation string contains only +columns to de-activate, then the routine will activate all other columns +before activating the list. This makes it simple to change the +activation state of all columns without having to know all of the +column names. For example: +<UL> +<LI> <B>"pi pha time"</B> # only these three columns will be active +<LI> <B>"-pi -pha -time"</B> # all but these columns will be active +<LI> <B>"pi -pha"</B> # only pi is active, pha is not, others are not +<LI> <B>"+pi -pha"</B> # same as above +<LI> <B>"pi -pha -time"</B> # only pi is active, all others are not +<LI> <B>"pi pha"</B> # pha and pi are active, all others are not +<LI> <B>"pi pha -x -y"</B> # pha and pi are active, all others are not +</UL> + +<p> +You can use the column activation list to reorder columns, since +columns are output in the order specified. For example: +<PRE> + # default output order + fundisp snr.ev'[cir 512 512 .1]' + X Y PHA PI TIME DX DY + -------- -------- -------- -------- --------------------- -------- -------- + 512 512 6 7 79493997.45854475 578 574 + 512 512 8 9 79494575.58943175 579 573 + 512 512 5 6 79493631.03866175 578 575 + 512 512 5 5 79493290.86521725 578 575 + 512 512 8 9 79493432.00990875 579 573 + + # re-order the output by specifying explicit order + fundisp snr.ev'[cir 512 512 .1]' "time x y dy dx pi pha" + TIME X Y DY DX PI PHA + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 512 512 574 578 7 6 + 79494575.58943175 512 512 573 579 9 8 + 79493631.03866175 512 512 575 578 6 5 + 79493290.86521725 512 512 575 578 5 5 + 79493432.00990875 512 512 573 579 9 8 +</PRE> + +<P> +A "+" sign by itself means to activate all columns, so that you can reorder +just a few columns without specifying all of them: +<PRE> + # reorder 3 columns and then output the rest + fundisp snr.ev'[cir 512 512 .1]' "time pi pha +" + TIME PI PHA Y X DX DY + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 7 6 512 512 578 574 + 79494575.58943175 9 8 512 512 579 573 + 79493631.03866175 6 5 512 512 578 575 + 79493290.86521725 5 5 512 512 578 575 + 79493432.00990875 9 8 512 512 579 573 +</PRE> +The column activation/deactivation is performed in the order of the +specified column arguments. This means you can mix "+", "-" (which +de-activates all columns) and specific column names to reorder and +select columns in one command. For example, consider the following: +<PRE> + # reorder and de-activate + fundisp snr.ev'[cir 512 512 .1]' "time pi pha + -x -y" + TIME PI PHA DX DY + --------------------- -------- -------- -------- -------- + 79493997.45854475 7 6 578 574 + 79494575.58943175 9 8 579 573 + 79493631.03866175 6 5 578 575 + 79493290.86521725 5 5 578 575 + 79493432.00990875 9 8 579 573 +</PRE> +We first activate "time", "pi", and "pha" so that they are output first. +We then activate all of the other columns, and then de-activate "x" and "y". +Note that this is different from: +<PRE> + # probably not what you want ... + fundisp snr.ev'[cir 512 512 .1]' "time pi pha -x -y +" + TIME PI PHA Y X DX DY + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 7 6 512 512 578 574 + 79494575.58943175 9 8 512 512 579 573 + 79493631.03866175 6 5 512 512 578 575 + 79493290.86521725 5 5 512 512 578 575 + 79493432.00990875 9 8 512 512 579 573 +</PRE> +Here, "x" and "y" are de-activated, but then all columns including "x" and +"y" are again re-activated. + +<p> +Typically, +<A HREF="./library.html#funcolumnactivate">FunColumnActivate()</A> uses a +list of columns that are passed into the program from the command line. For +example, the code for funtable contains the following: +<PRE> + char *cols=NULL; + + /* open the input FITS file */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + + /* set active flag for specified columns */ + if( argc >= 4 ) cols = argv[3]; + FunColumnActivate(fun, cols, NULL); +</PRE> + +The <A HREF="./library.html#funopen">FunOpen()</A> call sets the +default columns to be all columns in the input file. The +<A HREF="./library.html#funcolumnactivate">FunColumnActivate()</A> call +then allows the user to control which columns ultimately will be +activated (i.e., in this case, written to the new file). For example: +<PRE> + funtable test.ev foo.ev "pi pha time" +</PRE> +will process only the three columns mentioned, while: +<PRE> + funtable test.ev foo.ev "-time" +</PRE> +will process all columns except "time". + +<P> +If <A HREF="./library.html#funcolumnactivate">FunColumnActivate()</A> +is called with a null string, then the environment variable +<B>FUN_COLUMNS</B> will be used to provide a global value, if present. +This is the reason why we call the routine even if no columns +are specified on the command line (see example above), instead +of calling it this way: +<PRE> + /* set active flag for specified columns */ + if( argc >= 4 ){ + FunColumnActivate(fun, argv[3], NULL); + } +</PRE> + +<!-- =defdoc funcolumnlookup funcolumnlookup 3 --> + +<!-- =section funcolumnlookup NAME --> +<H2><A NAME="funcolumnlookup">FunColumnLookup - lookup a Funtools column</A></H2> +<!-- =section funcolumnlookup SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunColumnLookup(Fun fun, char *s, int which, + char **name, int *type, int *mode, + int *offset, int *n, int *width) +</PRE> +</B> + +<!-- =section funcolumnlookup DESCRIPTION --> +<P> +The <B>FunColumnLookup()</B> routine returns information about a named +(or indexed) column. The first argument is the Fun handle associated +with this set of columns. The second argument is the name of the +column to look up. If the name argument is NULL, the argument that +follows is the zero-based index into the column array of the column +for which information should be returned. The next argument is a +pointer to a char *, which will contain the name of the column. The +arguments that follow are the addresses of int values into which +the following information will be returned: +<UL> +<LI> <B>type</B>: data type of column: +<UL> +<LI>A: ASCII characters +<LI>B: unsigned 8-bit char +<LI>I: signed 16-bit int +<LI>U: unsigned 16-bit int (not standard FITS) +<LI>J: signed 32-bit int +<LI>V: unsigned 32-bit int (not standard FITS) +<LI>E: 32-bit float +<LI>D: 64-bit float +</UL> +<LI> <B>mode</B>: bit flag status of column, including: +<UL> +<LI> COL_ACTIVE 1 is column activated? +<LI> COL_IBUF 2 is column in the raw input data? +<LI> COL_PTR 4 is column a pointer to an array? +<LI> COL_READ 010 is read mode selected? +<LI> COL_WRITE 020 is write mode selected? +<LI> COL_REPLACEME 040 is this column being replaced by user data? +</UL> +<LI> <B>offset</B>: byte offset in struct +<LI> <B>n</B>: number of elements (i.e. size of vector) in this column +<LI> <B>width</B>: size in bytes of this column +</UL> +If the named column exists, the routine returns a positive integer, +otherwise zero is returned. (The positive integer is the index+1 into +the column array where this column was located.) + +If NULL is passed as the return address of one (or more) of these +values, no data is passed back for that information. For +example: +<PRE> + if( !FunColumnLookup(fun, "phas", 0, NULL NULL, NULL, NULL, &npha, NULL) ) + gerror(stderr, "can't find phas column\n"); +</PRE> +only returns information about the size of the phas vector. + +<!-- =defdoc funtablerowget funtablerowget 3 --> + +<!-- =section funtablerowget NAME --> +<H2><A NAME="funtablerowget">FunTableRowGet - get Funtools rows</A></H2> + +<!-- =section funtablerowget SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void *FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, + int *nrow) +</PRE> +</B> + +<!-- =section funtablerowget DESCRIPTION --> +<P> +The <B>FunTableRowGet()</B> routine retrieves rows from a Funtools +binary table or raw event file, and places the values of columns +selected by <A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> +into an array of user structs. Selected column values are +automatically converted to the specified user data type (and to native +data format) as necessary. + +<P> +The first argument is the Fun handle associated with this row data. +The second <B>rows</B> argument is the array of user structs into +which the selected columns will be stored. If NULL is passed, the +routine will automatically allocate space for this array. (This +includes proper allocation of pointers within each struct, if the "@" +pointer type is used in the selection of columns. Note that if you +pass NULL in the second argument, you should free this space using the +standard free() system call when you are finished with the array of +rows.) The third <B>maxrow</B> argument specifies the maximum number +of rows to be returned. Thus, if <B>rows</B> is allocated by the +user, it should be at least of size maxrow*sizeof(evstruct). + +<P> +The fourth <B>plist</B> argument is a param list string. Currently, +the keyword/value pair "mask=transparent" is supported in the plist +argument. If this string is passed in the call's plist argument, then +all rows are passed back to the user (instead of just rows passing +the filter). This is only useful when +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> also is +used to specify "$region" as a column to return for each row. In +such a case, rows found within a region have a returned region value +greater than 0 (corresponding to the region id of the region in which +they are located), rows passing the filter but not in a region have +region value of -1, and rows not passing any filter have region +value of 0. Thus, using "mask=transparent" and the returned region +value, a program can process all rows and decide on an action based +on whether a given row passed the filter or not. + +<P> +The final argument is a pointer to an int variable that will return +the actual number of rows returned. The routine returns a pointer to +the array of stored rows, or NULL if there was an error. (This pointer +will be the same as the second argument, if the latter is non-NULL). +<PRE> + /* get rows -- let routine allocate the row array */ + while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = buf+i; + /* rearrange some values. etc. */ + ev->energy = (ev->pi+ev->pha)/2.0; + ev->pha = -ev->pha; + ev->pi = -ev->pi; + } + /* write out this batch of rows */ + FunTableRowPut(fun2, buf, got, 0, NULL); + /* free row data */ + if( buf ) free(buf); + } +</PRE> +As shown above, successive calls to +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A> will return the +next set of rows from the input file until all rows have been read, +i.e., the routine behaves like sequential Unix I/O calls such as +fread(). See <A HREF="./evmerge.c">evmerge example code</A> for a +more complete example. + +<P> +Note that FunTableRowGet() also can be called as FunEventsGet(), for +backward compatibility. + +<!-- =defdoc funtablerowput funtablerowput 3 --> + +<!-- =section funtablerowput NAME --> +<H2><A NAME="funtablerowput">FunTableRowPut - put Funtools rows</A></H2> + +<!-- =section funtablerowput SYNOPSIS --> +<PRE> +<B> +int FunTableRowPut(Fun fun, void *rows, int nev, int idx, char *plist) +</B> +</PRE> + +<!-- =section funtablerowput DESCRIPTION --> +The <B>FunTableRowPut()</B> routine writes rows to a FITS binary +table, taking its input from an array of user structs that contain +column values selected by a previous call to +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A>. Selected +column values are automatically converted from native data format to +FITS data format as necessary. + +<P> +The first argument is the Fun handle associated with this row data. +The second <B>rows</B> argument is the array of user structs to +output. The third <B>nrow</B> argument specifies the number number of +rows to write. The routine will write <B>nrow</B> records, starting +from the location specified by <B>rows</B>. + +<P> +The fourth <B>idx</B> argument is the index of the first raw input +row to write, in the case where rows from the user buffer are +being merged with their raw input row counterparts (see below). Note +that this <B>idx</B> value is has nothing to do with the +row buffer specified in argument 1. It merely matches the row +being written with its corresponding (hidden) raw row. Thus, if you +read a number of rows, process them, and then write them out all at +once starting from the first user row, the value of <B>idx</B> +should be 0: +<PRE> + Ev ebuf, ev; + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ... + } + /* write out this batch of rows, starting with the first */ + FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL); + /* free row data */ + if( ebuf ) free(ebuf); + } +</PRE> + +<P> +On the other hand, if you write out the rows one at a time (possibly +skipping rows), then, when writing the i'th row from the input +array of rows, set <B>idx</B> to the value of i: +<PRE> + Ev ebuf, ev; + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ... + /* write out the current (i.e., i'th) row */ + FunTableRowPut(fun2, (char *)ev, 1, i, NULL); + } + /* free row data */ + if( ebuf ) free(ebuf); + } +</PRE> + +<P> +The final argument is a param list string that is not currently used. +The routine returns the number of rows output. This should be equal +to the value passed in the third <B>nrow</B argument. + +<P> +When <A HREF="./library.html#funtablerowput">FunTableRowPut()</A> is first +called for a given binary table, Funtools checks to see of the primary +header has already been written (either by writing a previous row +table or by writing an image.) If not, a dummy primary header is +written to the file specifying that an extension should be expected. +After this, a binary table header is automatically written containing +information about the columns that will populate this table. In +addition, if a +<A HREF="./library.html#refhandle">Funtools reference handle</A> +was specified when this table was opened, the parameters from this +<A HREF="./library.html#refhandle">Funtools reference handle</A> +are merged into the new binary table header. + +<P> +In a typical Funtools row loop, you read rows using +<A HREF="./library.html#funtablerowget">FunTableRowGet()</A>() and write +rows using FunTableRowPut(). The columns written by +FunTableRowPut()() are those defined as writable by a previous call to +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A>. If +that call to FunColumnSelect also specified +<B>merge=[update|replace|append]</B>, then the entire corresponding +raw input row record will be merged with the output row according +to the <B>merge</B> specification (see +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> above). + +<P> +A call to write rows can either be done once, after all rows in +the input batch have been processed, or it can be done (slightly less +efficiently) one row at a time (or anything in between). We do +recommend that you write all rows associated with a given batch of +input rows before reading new rows. This is <B>required</B> if +you are merging the output rows with the raw input rows (since +the raw rows are destroyed with each successive call to get new rows). + +For example: +<PRE> + Ev buf, ev; + ... + /* get rows -- let routine allocate the row array */ + while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* point to the i'th row */ + ev = buf + i; + .... process + } + /* write out this batch of rows */ + FunTableRowPut(fun2, buf, got, 0, NULL); + /* free row data */ + if( buf ) free(buf); + } +</PRE> + +or + +<PRE> + Ev buf, ev; + ... + /* get rows -- let routine allocate the row array */ + while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = buf + i; + ... process + /* write out this batch of rows with the new column */ + if( dowrite ) + FunTableRowPut(fun2, buf, 1, i, NULL); + } + /* free row data */ + if( buf ) free(buf); + } +</PRE> + +<P> +Note that the difference between these calls is that the first one +outputs <B>got</B> rows all at once and therefore passes +<B>idx=0</B> in argument four, so that merging starts at the first raw +input row. In the second case, a check it made on each row to see +if it needs to be output. If so, the value of <B>idx</B> is passed as +the value of the <B>i</B> variable which points to the current row +being processed in the batch of input rows. + +<P> +As shown above, successive calls to +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A> will write +rows sequentially. When you are finished writing all rows in a +table, you should call +<A HREF="./library.html#funflush">FunFlush()</A> to write out the FITS +binary table padding. However, this is not necessary if you +subsequently call FunClose() without doing any other I/O to the FITS +file. + +<P> +Note that FunTableRowPut() also can be called as FunEventsPut(), for +backward compatibility. + +<!-- =defdoc funparamget funparamget 3 --> + +<!-- =section funparamget NAME --> +<H2><A NAME="funparamget">FunParamGet - get a Funtools param value</A></H2> + +<!-- =section funparamget SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunParamGetb(Fun fun, char *name, int n, int defval, int *got) + + int FunParamGeti(Fun fun, char *name, int n, int defval, int *got) + + double FunParamGetd(Fun fun, char *name, int n, double defval, int *got) + + char *FunParamGets(Fun fun, char *name, int n, char *defval, int *got) +</PRE> +</B> + +<!-- =section funparamget DESCRIPTION --> +<P> +The four routines <B>FunParamGetb()</B>, <B>FunParamGeti()</B>, +<B>FunParamGetd()</B>, and <B>FunParamGets()</B>, return the value of +a FITS header parameter as a boolean, int, double, and string, +respectively. The string returned by <B>FunParamGets()</B> is a malloc'ed +copy of the header value and should be freed when no longer needed. + +<P> +The first argument is the Fun handle associated with the FITS header +being accessed. Normally, the header is associated with the FITS +extension that you opened with <B>FunOpen()</B>. However, you can use +FunInfoPut() to specify access of the primary header. In particular, +if you set the FUN_PRIMARYHEADER parameter to 1, then the primary +header is used for all parameter access until the value is reset to +0. For example: +<PRE> + int val; + FunParamGeti(fun, "NAXIS", 1, 0, &got); # current header + val=1; + FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0); # switch to ... + FunParamGeti(fun, "NAXIS", 1, 0, &got); # ... primary header + FunParamGeti(fun, "NAXIS", 2, 0, &got); # ... primary header + val=0; + FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0); # switch back to ... + FunParamGeti(fun, "NAXIS", 2, 0, &got); # current header +</PRE> + +<P> +Alternatively, you can use the FUN_PRIMARY macro to access parameters +from the primary header on a per-parameter basis: +<PRE> + FunParamGeti(fun, "NAXIS1", 0, 0, &got); # current header + FunParamGeti(FUN_PRIMARY(fun), "NAXIS1", 0, 0, &got); # primary header +</PRE> +<B>NB: FUN_PRIMARY is deprecated.</B> +It makes use of a global parameter and therefore will not not +appropriate for threaded applications, when we make funtools +thread-safe. We recommend use of FunInfoPut() to switch between the +extension header and the primary header. + +<P> +For output data, access to the primary header is only possible until +the header is written out, which usually takes place when the first +data are written. + +<P> +The second argument is the name of the parameter to access. The third +<B>n</B> argument, if non-zero, is an integer that will be added as a +suffix to the parameter name. This makes it easy to use a simple loop +to process parameters having the same root name. For example, to +gather up all values of TLMIN and TLMAX for each column in a binary +table, you can use: +<PRE> + for(i=0, got=1; got; i++){ + fun->cols[i]->tlmin = (int)FunParamGeti(fun, "TLMIN", i+1, 0.0, &got); + fun->cols[i]->tlmax = (int)FunParamGeti(fun, "TLMAX", i+1, 0.0, &got); + } +</PRE> + +<P> +The fourth <B>defval</B> argument is the default value to return if +the parameter does not exist. Note that the data type of this +parameter is different for each specific FunParamGet() call. The final +<B>got</B> argument will be 0 if no param was found. Otherwise the +data type of the parameter is returned as follows: FUN_PAR_UNKNOWN +('u'), FUN_PAR_COMMENT ('c'), FUN_PAR_LOGICAL ('l'), FUN_PAR_INTEGER +('i'), FUN_PAR_STRING ('s'), FUN_PAR_REAL ('r'), FUN_PAR_COMPLEX ('x'). + +<p> +These routines return the value of the header parameter, or the +specified default value if the header parameter does not exist. The +returned value is a malloc'ed string and should be freed when no +longer needed. + +<P> +By default, <B>FunParamGets()</B> returns the string value of the +named parameter. However, you can use FunInfoPut() to retrieve the +raw 80-character FITS card instead. In particular, if you set the +FUN_RAWPARAM parameter to 1, then card images will be returned by +FunParamGets() until the value is reset to 0. + +<P> +Alternatively, if the FUN_RAW macro is applied to the name, then the +80-character raw FITS card is returned instead. +<B>NB: FUN_RAW is deprecated.</B> +It makes use of a global parameter and therefore will not not +appropriate for threaded applications, when we make funtools +thread-safe. We recommend use of FunInfoPut() to switch between the +extension header and the primary header. + +<P> +Note that in addition to the behaviors described above, the +routine <B>FunParamGets()</B> will return the 80 raw characters of the +<B>nth</B> FITS card (including the comment) if <B>name</B> is specified as +NULL and <B>n</B> is positive. For example, to loop through all FITS +header cards in a given extension and print out the raw card, use: +<PRE> + for(i=1; ;i++){ + if( (s = FunParamGets(fun, NULL, i, NULL, &got)) ){ + fprintf(stdout, "%.80s\n", s); + free(s); + } + else{ + break; + } + } +</PRE> + +<!-- =defdoc funparamput funparamput 3 --> + +<!-- =section funparamput NAME --> +<H2><A NAME="funparamput">FunParamPut - put a Funtools param value</A></H2> + +<!-- =section funparamput SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunParamPutb(Fun fun, char *name, int n, int value, char *comm, + int append) + + int FunParamPuti(Fun fun, char *name, int n, int value, char *comm, + int append) + + int FunParamPutd(Fun fun, char *name, int n, double value, int prec, + char *comm, int append) + + int FunParamPuts(Fun fun, char *name, int n, char *value, char *comm, + int append) +</PRE> +</B> + +<!-- =section funparamput DESCRIPTION --> +<P> +The four routines <B>FunParamPutb()</B>, <B>FunParamPuti()</B>, +<B>FunParamPutd()</B>, and <B>FunParamPuts()</B>, will set the value +of a FITS header parameter as a boolean, int, double, and string, +respectively. + +<P> +The first argument is the Fun handle associated with the FITS header +being accessed. Normally, the header is associated with the FITS +extension that you opened with <B>FunOpen()</B>. +However, you can use FunInfoPut() to specify that use of the primary +header. In particular, if you set the FUN_PRIMARYHEADER parameter to +1, then the primary header is used for all parameter access until the +value is reset to 0. For example: +<PRE> + int val; + FunParamPuti(fun, "NAXIS1", 0, 10, NULL, 1); # current header + val=1; + FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0); # switch to ... + FunParamPuti(fun, "NAXIS1", 0, 10, NULL, 1); # primary header +</PRE> +(You also can use the deprecated FUN_PRIMARY macro, to access +parameters from the primary header.) + +<p> +The second argument is the <b>name</b> of the parameter. ( +In accordance with FITS standards, the special names <b>COMMENT</b> +and <b>HISTORY</b>, as well as blank names, are output without the "= " +value indicator in columns 9 and 10. + +<p> +The third <B>n</B> argument, if non-zero, is an integer that will be +added as a suffix to the parameter name. This makes it easy to use a +simple loop to process parameters having the same root name. For +example, to set the values of TLMIN and TLMAX for each column in a +binary table, you can use: +<PRE> + for(i=0; i<got; i++){ + FunParamPutd(fun, "TLMIN", i+1, tlmin[i], 7, "min column val", 1); + FunParamPutd(fun, "TLMAX", i+1, tlmax[i], 7, "max column val", 1); + } +</PRE> + +<P> +The fourth <B>defval</B> argument is the value to set. Note that the +data type of this argument is different for each specific +FunParamPut() call. The <B>comm</B> argument is the comment +string to add to this header parameter. Its value can be NULL. The +final <B>append</B> argument determines whether the parameter is added +to the header if it does not exist. If set to a non-zero value, the +header parameter will be appended to the header if it does not exist. +If set to 0, the value will only be used to change an existing parameter. + +<P> +Note that the double precision routine FunParamPutd() supports an +extra <B>prec</B> argument after the <B>value</B> argument, in order +to specify the precision when converting the double value to ASCII. In +general a 20.[prec] format is used (since 20 characters are alloted to +a floating point number in FITS) as follows: if the double value being +put to the header is less than 0.1 or greater than or equal to +10**(20-2-[prec]), then %20.[prec]e format is used (i.e., scientific +notation); otherwise %20.[prec]f format is used (i.e., numeric +notation). + +<P> +As a rule, parameters should be set before writing the table or image. +It is, however, possible to update the value of an <B>existing</B> +parameter after writing an image or table (but not to add a new +one). Such updating only works if the parameter already exists and if +the output file is seekable, i.e. if it is a disk file or is stdout +being redirected to a disk file. + +<P> +It is possible to add a new parameter to a header after the data has +been written, but only if space has previously been reserved. To reserve +space, add a blank parameter whose value is the name of the parameter you +eventually will update. Then, when writing the new parameter, specify a +value of 2 for the append flag. The parameter writing routine will +first look to update an existing parameter, as usual. If an existing +parameter is not found, an appropriately-valued blank parameter will be +searched for and replaced. For example: +<PRE> + /* add blank card to be used as a place holder for IPAR1 update */ + FunParamPuts(fun, NULL, 0, "IPAR1", "INTEGER Param", 0); + ... + /* write header and data */ + FunTableRowPut(fun, events, got, 0, NULL); + ... + /* update param in file after writing data -- note append = 2 here */ + FunParamPuti(fun, "IPAR", 1, 400, "INTEGER Param", 2); +</PRE> + +<P> +The parameter routines return a 1 if the routine was successful and a 0 on +failure. In general, the major reason for failure is that you did not +set the append argument to a non-zero value and the parameter did not +already exist in the file. + +<!-- =defdoc funinfoget funinfoget 3 --> + +<!-- =section funinfoget NAME --> +<H2><A NAME="funinfoget">FunInfoGet - get information from Funtools struct</A></H2> + +<!-- =section funinfoget SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunInfoGet(Fun fun, int type, char *addr, ...) +</PRE> +</B> + +<!-- =section funinfoget DESCRIPTION --> +<P> +The <B>FunInfoGet()</B> routine returns information culled from the +Funtools structure. The first argument is the Fun handle from which +information is to be retrieved. This first required argument is followed +by a variable length list of pairs of arguments. Each pair consists +of an integer representing the type of information to retrieve and the +address where the information is to be stored. The list is terminated by a 0. +The routine returns the number of get actions performed. + +<P> +The full list of available information is described below. Please note +that only a few of these will be useful to most application developers. +For imaging applications, the most important types are: +<PRE> + FUN_SECT_DIM1 int /* dim1 for section */ + FUN_SECT_DIM2 int /* dim2 for section */ + FUN_SECT_BITPIX int /* bitpix for section */ +</PRE> +These would be used to determine the dimensions and data type of image +data retrieved using the +<A HREF="./library.html#funimageget">FunImageGet()</A> routine. For +example: +<PRE> + /* extract and bin the data section into an image buffer */ + buf = FunImageGet(fun, NULL, NULL); + /* get required information from funtools structure. + this should come after the FunImageGet() call, in case the call + changed sect_bitpix */ + FunInfoGet(fun, + FUN_SECT_BITPIX, &bitpix, + FUN_SECT_DIM1, &dim1, + FUN_SECT_DIM2, &dim2, + 0); + /* loop through pixels and reset values below limit to value */ + for(i=0; i<dim1*dim2; i++){ + switch(bitpix){ + case 8: + if( cbuf[i] <= blimit ) cbuf[i] = bvalue; + ... + } +</PRE> +It is important to bear in mind that the call to +<A HREF="./library.html#funimageget">FunImageGet()</A> +can change the value of FUN_SECT_BITPIX (e.g. if "bitpix=n" is passed +in the param list). Therefore, a call to +<A HREF="./library.html#funinfoget">FunInfoGet()</A> +should be made <B>after</B> the call to +<A HREF="./library.html#funimageget">FunImageGet()</A>, +in order to retrieve the updated bitpix value. +See the <A HREF="./imblank.c">imblank example code</A> for more +details. + +<P> +It also can be useful to retrieve the World Coordinate System +information from the Funtools structure. Funtools uses the the WCS +Library developed by Doug Mink at SAO, which is available +<A HREF="ftp://cfa-ftp.harvard.edu/pub/gsc/WCSTools/home.html">here</A>. +(More information about the WCSTools project in general can be found +<A HREF="http://tdc-www.harvard.edu/software/wcstools/">here</A>.) +The <A HREF="./library.html#funopen">FunOpen()</A> routine initializes +two WCS structures that can be used with this WCS Library. +Applications can retrieve either of these two WCS structures using +<B>FunInfoGet()</B>: +<PRE> + FUN_WCS struct WorldCoor * /* wcs structure, for image coordinates*/ + FUN_WCS0 struct WorldCoor * /* wcs structure, for physical coordinates */ +</PRE> +The structure retrieved by FUN_WCS is a WCS library handle containing +parameters suitable for use with image coordinates, regardless of whether the +data are images or tables. For this structure, the WCS reference point +(CRPIX) has been converted to image coordinates if the underlying file +is a table (and therefore in physical coordinates). You therefore must +ensure that the positions being passed to a routine like pix2wcs are in +image coordinates. The FUN_WCS0 structure has not had its WCS +reference point converted to image coordinates. It therefore is useful +when passing processing physical coordinates from a table. + +<P> +Once a WCS structure has been retrieved, it can be used as the first +argument to the WCS library routines. (If the structure is NULL, no +WCS information was contained in the file.) The two important WCS routines +that Funtools uses are: +<PRE> + #include <wcs.h> + void pix2wcs (wcs,xpix,ypix,xpos,ypos) + struct WorldCoor *wcs; /* World coordinate system structure */ + double xpix,ypix; /* x and y coordinates in pixels */ + double *xpos,*ypos; /* RA and Dec in degrees (returned) */ +</PRE> + +which converts pixel coordinates to sky coordinates, and: + +<PRE> + void wcs2pix (wcs, xpos, ypos, xpix, ypix, offscl) + struct WorldCoor *wcs; /* World coordinate system structure */ + double xpos,ypos; /* World coordinates in degrees */ + double *xpix,*ypix; /* coordinates in pixels */ + int *offscl; /* 0 if within bounds, else off scale */ +</PRE> +which converts sky coordinates to pixel coordinates. Again, please note +that the wcs structure returned by FUN_WCS assumes that image coordinates +are passed to the pix2wcs routine, while FUN_WCS0 assumes that physical +coordinates are passed. + +<P> +Note that funtools.h file automatically includes wcs.h. An example +program that utilizes these WCS structure to call WCS Library routines +is <A HREF="./twcs.c">twcs.c</A>. + +<P> +The following is the complete list of information that can be returned: +<PRE> + name type comment + --------- -------- --------------------------------------------- + FUN_FNAME char * /* file name */ + FUN_GIO GIO /* gio handle */ + FUN_HEADER FITSHead /* fitsy header struct */ + FUN_TYPE int /* TY_TABLE,TY_IMAGE,TY_EVENTS,TY_ARRAY */ + FUN_BITPIX int /* bits/pixel in file */ + FUN_MIN1 int /* tlmin of axis1 -- tables */ + FUN_MAX1 int /* tlmax of axis1 -- tables */ + FUN_MIN2 int /* tlmin of axis2 -- tables */ + FUN_MAX2 int /* tlmax of axis2 -- tables */ + FUN_DIM1 int /* dimension of axis1 */ + FUN_DIM2 int /* dimension of axis2 */ + FUN_ENDIAN int /* 0=little, 1=big endian */ + FUN_FILTER char * /* supplied filter */ + FUN_IFUN FITSHead /* pointer to reference header */ + FUN_IFUN0 FITSHead /* same as above, but no reset performed */ + /* image information */ + FUN_DTYPE int /* data type for images */ + FUN_DLEN int /* length of image in bytes */ + FUN_DPAD int /* padding to end of extension */ + FUN_DOBLANK int /* was blank keyword defined? */ + FUN_BLANK int /* value for blank */ + FUN_SCALED int /* was bscale/bzero defined? */ + FUN_BSCALE double /* bscale value */ + FUN_BZERO double /* bzero value */ + /* table information */ + FUN_NROWS int /* number of rows in file (naxis2) */ + FUN_ROWSIZE int /* size of user row struct */ + FUN_BINCOLS char * /* specified binning columns */ + FUN_OVERFLOW int /* overflow detected during binning? */ + /* array information */ + FUN_SKIP int /* bytes to skip in array header */ + /* section information */ + FUN_SECT_X0 int /* low dim1 value of section */ + FUN_SECT_X1 int /* hi dim1 value of section */ + FUN_SECT_Y0 int /* low dim2 value of section */ + FUN_SECT_Y1 int /* hi dim2 value of section */ + FUN_SECT_BLOCK int /* section block factor */ + FUN_SECT_BTYPE int /* 's' (sum), 'a' (average) for binning */ + FUN_SECT_DIM1 int /* dim1 for section */ + FUN_SECT_DIM2 int /* dim2 for section */ + FUN_SECT_BITPIX int /* bitpix for section */ + FUN_SECT_DTYPE int /* data type for section */ + FUN_RAWBUF char * /* pointer to raw row buffer */ + FUN_RAWSIZE int /* byte size of raw row records */ + /* column information */ + FUN_NCOL int /* number of row columns defined */ + FUN_COLS FunCol /* array of row columns */ + /* WCS information */ + FUN_WCS struct WorldCoor * /* wcs structure, converted for images*/ + FUN_WCS0 struct WorldCoor * /* wcs structure, not converted */ +</PRE> + +<P> +Row applications would not normally need any of this information. +An example of how these values can be used in more complex programs is +the <A HREF="./evnext.c">evnext example code</A>. In this program, the +time value for each row is changed to be the value of the succeeding +row. The program thus reads the time values for a batch of rows, +changes the time values to be the value for the succeeding row, and +then merges these changed time values back with the other columns to +the output file. It then reads the next batch, etc. + +<P> +This does not work for the last row read in each batch, since there +is no succeeding row until the next batch is read. Therefore, the +program saves that last row until it has read the next batch, then +processes the former before starting on the new batch. In order to +merge the last row successfully, the code uses FUN_RAWBUF to save +and restore the raw input data associated with each batch of +rows. Clearly, this requires some information about how funtools +works internally. We are happy to help you write such programs as the +need arises. + +<!-- =defdoc funinfoput funinfoput 3 --> + +<!-- =section funinfoput NAME --> +<H2><A NAME="funinfoput">FunInfoPut - put information into a Funtools struct</A></H2> + +<!-- =section funinfoput SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + int FunInfoPut(Fun fun, int type, char *addr, ...) +</PRE> +</B> + +<!-- =section funinfoput DESCRIPTION --> +<P> +The <B>FunInfoPut()</B> routine puts information into a Funtools +structure. The first argument is the Fun handle from which +information is to be retrieved. After this first required argument +comes a variable length list of pairs of arguments. Each pair consists +of an integer representing the type of information to store and the +address of the new information to store in the struct. The variable +list is terminated by a 0. The routine returns the number of put +actions performed. + +<P> +The full list of available information is described above with the +<A HREF="./library.html#funinfoput">FunInfoPut()</A> routine. Although +use of this routine is expected to be uncommon, there is one +important situation in which it plays an essential part: writing +multiple extensions to a single output file. + +<P> +For input, multiple extensions are handled by calling +<A HREF="./library.html#funopen">FunOpen()</A> for each extension to be +processed. When opening multiple inputs, it sometimes is the case that +you will want to process them and then write them (including their +header parameters) to a single output file. To accomplish this, you +open successive input extensions using +<A HREF="./library.html#funopen">FunOpen()</A> and then call +<B>FunInfoPut()</B> to set the +<A HREF="./library.html#refhandle">Funtools reference handle</A> +of the output file to that of the newly opened input extension: +<PRE> + /* open a new input extension */ + ifun=FunOpen(tbuf, "r", NULL)) ) + /* make the new extension the reference handle for the output file */ + FunInfoPut(ofun, FUN_IFUN, &ifun, 0); +</PRE> + +Resetting FUN_IFUN has same effect as when a funtools handle is passed +as the final argument to +<A HREF="./library.html#funopen">FunOpen()</A>. The state of the output +file is reset so that a new extension is ready to be written. +Thus, the next I/O call on the output extension will output the +header, as expected. + +<P> +For example, in a binary table, after resetting FUN_IFUN you can then +call <A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> to +select the columns for output. When you then call +<A HREF="./library.html#funimageput">FunImagePut()</A> or <A +HREF="./library.html#funtablerowput">FunTableRowPut()</A>, a new +extension will be written that contains the header parameters from the +reference extension. Remember to call +<A HREF="./library.html#funflush">FunFlush()</A> to complete output of a +given extension. + +<P> +A complete example of this capability is given +in the <A HREF="./evcol.c">evcol example code</A>. +The central algorithm is: +<UL> +<LI> open the output file without a reference handle +<LI> loop: open each input extension in turn +<UL> +<LI> set the reference handle for output to the newly opened input extension +<LI> read the input rows or image and perform processing +<LI> write new rows or image to the output file +<LI> flush the output +<LI> close input extension +</UL> +<LI> close output file +</UL> +Note that <A HREF="./library.html#funflush">FunFlush()</A> is called +after processing each input extension in order to ensure that the +proper padding is written to the output file. A call to +<A HREF="./library.html#funflush">FunFlush()</A> also ensures that the +extension header is written to the output file in the case where there +are no rows to output. + +<P> +If you wish to output a new extension without using a +<A HREF="./library.html#refhandle">Funtools reference handle</A>, you can +call FunInfoPut() to reset the FUN_OPS value directly. For a binary +table, you would then call FunColumnSelect() to set up the columns for +this new extension. +<PRE> + /* reset the operations performed on this handle */ + int ops=0; + FunInfoPut(ofun, FUN_OPS, &ops, 0); + FunColumnSelect(fun, sizeof(EvRec), NULL, + "MYCOL", "J", "w", FUN_OFFSET(Ev, mycol), + NULL); +</PRE> +Once the FUN_OPS variable has been reset, the next I/O call on the +output extension will output the header, as expected. + +<!-- =defdoc funflush funflush 3 --> + +<!-- =section funflush NAME --> +<H2><A NAME="funflush">FunFlush - flush data to output file</A></H2> + +<!-- =section funflush SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void FunFlush(Fun fun, char *plist) +</PRE> +</B> + +<!-- =section funflush DESCRIPTION --> +<P> +The <B>FunFlush</B> routine will flush data to a FITS output file. In +particular, it can be called after all rows have been written (using +the <A HREF="./library.html#funtablerowput">FunTableRowPut()</A> routine) +in order to add the null padding that is required to complete a FITS +block. It also should be called after completely writing an image using +<A HREF="./library.html#funimageput">FunImagePut()</A> or after writing +the final row of an image using +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A>. + +<P> +The <B>plist</B> (i.e., parameter list) argument is a string +containing one or more comma-delimited <B>keyword=value</B> +parameters. If the plist string contains the parameter +"copy=remainder" and the file was opened with a reference file, which, +in turn, was opened for extension copying (i.e. the input +<A HREF="./library.html#funopen">FunOpen()</A> mode also was "c" or "C"), +then FunFlush also will copy the remainder of the FITS extensions from +the input reference file to the output file. This normally would be +done only at the end of processing. + +<P> +Note that <A HREF="./library.html#funflush">FunFlush()</A> is called +with "copy=remainder" in the mode string by FunClose(). This means +that if you close the output file before the reference input file, it +is not necessary to call +<A HREF="./library.html#funflush">FunFlush()</A> explicitly, unless you +are writing more than one extension. See the +<A HREF="./evmerge.c">evmerge example code</A>. However, it is safe to +call <A HREF="./library.html#funflush">FunFlush()</A> more than once +without fear of re-writing either the padding or the copied +extensions. + +<P> +In addition, if <A HREF="./library.html#funflush">FunFlush()</A> is +called on an output file with the plist set to "copy=reference" and if +the file was opened with a reference file, the reference extension is +written to the output file. This mechanism provides a simple way to +copy input extensions to an output file without processing the former. +For example, in the code fragment below, an input extension is set to +be the reference file for a newly opened output extension. If that +reference extension is not a binary table, it is written to the output +file: +<PRE> + /* process each input extension in turn */ + for(ext=0; ;ext++){ + /* get new extension name */ + sprintf(tbuf, "%s[%d]", argv[1], ext); + /* open input extension -- if we cannot open it, we are done */ + if( !(ifun=FunOpen(tbuf, "r", NULL)) ) + break; + /* make the new extension the reference handle for the output file */ + FunInfoPut(ofun, FUN_IFUN, &ifun, 0); + /* if its not a binary table, just write it out */ + if( !(s=FunParamGets(ifun, "XTENSION", 0, NULL, &got)) || + strcmp(s, "BINTABLE")){ + if( s ) free(s); + FunFlush(ofun, "copy=reference"); + FunClose(ifun); + continue; + } + else{ + /* process binary table */ + .... + } + } +</PRE> + +<!-- =defdoc funclose funclose 3 --> + +<!-- =section funclose NAME --> +<H2><A NAME="funclose">FunClose - close a Funtools data file</A></H2> + +<!-- =section funclose SYNOPSIS --> +<B> +<PRE> + #include <funtools.h> + + void FunClose(Fun fun) +</PRE> +</B> + +<!-- =section funclose DESCRIPTION --> +<P> +The <B>FunClose()</B> routine closes a previously-opened Funtools data +file, freeing control structures. If a +<A HREF="./library.html#refhandle">Funtools reference handle</A> +was passed to +the <A HREF="./library.html#funopen">FunOpen()</A> call for this file, +and if copy mode also was specified for that file, then +<A HREF="./library.html#funclose">FunClose()</A> also will copy the +remaining extensions from the input file to the output file (if the +input file still is open). Thus, we recommend always closing the +output Funtools file <B>before</B> the input file. (Alternatively, +you can call <A HREF="./library.html#funflush">FunFlush()</A> +explicitly). + +<!-- =defdoc funref funref 3 --> + +<!-- =section funref NAME --> +<H2><A NAME="refhandle">FunRef: the Funtools Reference Handle</A></H2> + +<!-- =section funref SYNOPSIS --> +<H2>Summary</H2> +A description of how to use a Funtools reference handle to connect a +Funtools input file to an output file. + +<!-- =section funref DESCRIPTION --> +<H2>Description</H2> +<P> +The Funtools reference handle connects a Funtools input file to a +Funtools output file so that parameters (or even whole extensions) can +be copied from the one to the other. To make the connection, the Funtools +handle of the input file is passed to the +final argument of the +<A HREF="./library.html#funopen">FunOpen()</A> call for the output file: +<PRE> + if( !(ifun = FunOpen(argv[1], "r", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + if( !(ofun = FunOpen(argv[2], "w", ifun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); +</PRE> +It does not matter what type of input or output file (or extension) is +opened, or whether they are the same type. When the output image or +binary table is written using +<A HREF="./library.html#funimageput">FunImagePut()</A> +or +<A HREF="./library.html#funtablerowput">FunTableRowPut()</A> +an appropriate header will be written first, with parameters copied +from the input extension. Of course, invalid parameters will be +removed first, e.g., if the input is a binary table and the output is +an image, then binary table parameters such as TFORM, TUNIT, +etc. parameters will not be copied to the output. + +<P> +Use of a reference handle also allows default values to be passed +to +<A HREF="./library.html#funimageput">FunImagePut()</A> in order to +write out an output image with the same dimensions and data type +as the input image. To use the defaults from the input, a value +of 0 is entered for dim1, dim2, and bitpix. For example: +<PRE> + fun = FunOpen(argv[1], "r", NULL); + fun2 = FunOpen(argv[2], "w", fun); + buf = FunImageGet(fun, NULL, NULL); + ... process image data ... + FunImagePut(fun2, buf, 0, 0, 0, NULL); +</PRE> +Of course, you often want to get information about the data type +and dimensions of the image for processing. The above code +is equivalent to the following: +<PRE> + fun = FunOpen(argv[1], "r", NULL); + fun2 = FunOpen(argv[2], "w", fun); + buf = FunImageGet(fun, NULL, NULL); + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, + FUN_SECT_BITPIX, &bitpix, 0); + ... process image data ... + FunImagePut(fun2, buf, dim1, dim2, bitpix, NULL); +</PRE> + +<P> +It is possible to change the reference handle for a given output Funtools +handle using the +<A HREF="./library.html#funinfoput">FunInfoPut()</A> routine: +<PRE> + /* make the new extension the reference handle for the output file */ + FunInfoPut(fun2, FUN_IFUN, &fun, 0); +</PRE> +When this is done, Funtools specially resets the output file to start +a new output extension, which is connected to the new input reference +handle. You can use this mechanism to process multiple input extensions +into a single output file, by successively opening the former and +setting the reference handle for the latter. For example: +<PRE> + /* open a new output FITS file */ + if( !(fun2 = FunOpen(argv[2], "w", NULL)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + /* process each input extension in turn */ + for(ext=0; ;ext++){ + /* get new extension name */ + sprintf(tbuf, "%s[%d]", argv[1], ext); + /* open it -- if we cannot open it, we are done */ + if( !(fun=FunOpen(tbuf, "r", NULL)) ) + break; + /* make the new extension the reference handle for the output file */ + FunInfoPut(fun2, FUN_IFUN, &fun, 0); + ... process ... + /* flush output extension (write padding, etc.) */ + FunFlush(fun2, NULL); + /* close the input extension */ + FunClose(fun); + } +</PRE> +In this example, the output file is opened first. Then each successive +input extension is opened, and the output reference handle is set to +the newly opened input handle. After data processing is performed, the +output extension is flushed and the input extension is closed, in +preparation for the next input extension. +<P> +Finally, a reference handle can be used to copy other extensions from +the input file to the output file. Copy of other extensions is +controlled by adding a "C" or "c" to the mode string of the +<A HREF="./library.html#funopen">FunOpen()</A> +call <B>of the input reference file</B>. If "C" is specified, then +other extensions are <B>always</B> copied (i.e., copy is forced by the +application). If "c" is used, then other extensions are copied if the +user requests copying by adding a plus sign "+" to the extension name +in the bracket specification. For example, the <B>funtable</B> +program utilizes user-specified "c" mode so that the second example +below will copy all extensions: +<PRE> + # copy only the EVENTS extension + csh> funtable "test.ev[EVENTS,circle(512,512,10)]" foo.ev + # copy ALL extensions + csh> funtable "test.ev[EVENTS+,circle(512,512,10)]" foo.ev +</PRE> +When extension copy is specified in the input file, the call to +<A HREF="./library.html#funopen">FunOpen()</A> +on the input file delays the actual file open until the output file +also is opened (or until I/O is performed on the input file, which +ever happens first). Then, when the output file is opened, the input +file is also opened and input extensions are copied to the output +file, up to the specific extension being opened. Processing of input +and output extensions then proceed. +<P> +When extension processing is complete, the remaining extensions need to +be copied from input to output. This can be done explicitly, using the +<A HREF="./library.html#funflush">FunFlush()</A> +call with the "copy=remaining" plist: +<PRE> + FunFlush(fun, "copy=remaining"); +</PRE> +Alternatively, this will happen automatically, if the output file +is closed <B>before</B> the input file: +<PRE> + /* we could explicitly flush remaining extensions that need copying */ + /* FunFlush(fun2, "copy=remaining"); */ + /* but if we close output before input, end flush is done automatically */ + FunClose(fun2); + FunClose(fun); +</PRE> + +<!-- =section funlib SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funopen SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funimageget SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funimageput SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funimagerowget SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funimagerowput SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funcolumnselect SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funcolumnactivate SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funcolumnlookup SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funtablerowget SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funtablerowput SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funparamget SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funparamput SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funinfoget SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funinfoput SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funflush SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funclose SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funref SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: December 1, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/pod/funcalc.pod b/funtools/doc/pod/funcalc.pod new file mode 100644 index 0000000..f63cfc8 --- /dev/null +++ b/funtools/doc/pod/funcalc.pod @@ -0,0 +1,572 @@ +=pod + +=head1 NAME + + + +B<funcalc - Funtools calculator (for binary tables)> + + +=head1 SYNOPSIS + + + + + +funcalc [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] <iname> [oname [columns]] + + + + + +=head1 OPTIONS + + + + + + -a argstr # user arguments to pass to the compiled program + -e expr # funcalc expression + -f file # file containing funcalc expression + -l libs # libs to add to link command + -n # output generated code instead of compiling and executing + -p prog # generate named program, no execution + -u # die if any variable is undeclared (don't auto-declare) + + + + +=head1 DESCRIPTION + + + + +B<funcalc> is a calculator program that allows arbitrary +expressions to be constructed, compiled, and executed on columns in a +Funtools table (FITS binary table or raw event file). It works by +integrating user-supplied expression(s) into a template C program, +then compiling and executing the program. B<funcalc> expressions +are C statements, although some important simplifications (such +as automatic declaration of variables) are supported. + + +B<funcalc> expressions can be specified in three ways: on the +command line using the B<-e [expression]> switch, in a file using +the B<-f [file]> switch, or from stdin (if neither B<-e> nor +B<-f> is specified). Of course a file containing B<funcalc> +expressions can be read from stdin. + + +Each invocation of B<funcalc> requires an input Funtools table +file to be specified as the first command line argument. The output +Funtools table file is the second optional argument. It is needed only +if an output FITS file is being created (i.e., in cases where the +B<funcalc> expression only prints values, no output file is +needed). If input and output file are both specified, a third optional +argument can specify the list of columns to activate (using +FunColumnActivate()). Note +that B<funcalc> determines whether or not to generate code for +writing an output file based on the presence or absence of an +output file argument. + + +A B<funcalc> expression executes on each row of a table and +consists of one or more C statements that operate on the columns of +that row (possibly using temporary variables). Within an expression, +reference is made to a column of the B<current> row using the C +struct syntax B<cur->[colname]>, e.g. cur->x, cur->pha, etc. +Local scalar variables can be defined using C declarations at very the +beginning of the expression, or else they can be defined automatically +by B<funcalc> (to be of type double). Thus, for example, a swap of +columns x and y in a table can be performed using either of the +following equivalent B<funcalc> expressions: + + + double temp; + temp = cur->x; + cur->x = cur->y; + cur->y = temp; + + +or: + + + temp = cur->x; + cur->x = cur->y; + cur->y = temp; + + +When this expression is executed using a command such as: + + funcalc -f swap.expr itest.ev otest.ev + +the resulting file will have values of the x and y columns swapped. + + +By default, the data type of the variable for a column is the same as +the data type of the column as stored in the file. This can be changed +by appending ":[dtype]" to the first reference to that column. In the +example above, to force x and y to be output as doubles, specify the +type 'D' explicitly: + + temp = cur->x:D; + cur->x = cur->y:D; + cur->y = temp; + + +Data type specifiers follow standard FITS table syntax for defining +columns using TFORM: + + +=over 4 + + + + +=item * + +A: ASCII characters + + +=item * + +B: unsigned 8-bit char + + +=item * + +I: signed 16-bit int + + +=item * + +U: unsigned 16-bit int (not standard FITS) + + +=item * + +J: signed 32-bit int + + +=item * + +V: unsigned 32-bit int (not standard FITS) + + +=item * + +E: 32-bit float + + +=item * + +D: 64-bit float + + +=item * + +X: bits (treated as an array of chars) + + +=back + + +Note that only the first reference to a column should contain the +explicit data type specifier. + + +Of course, it is important to handle the data type of the columns +correctly. One of the most frequent cause of error in B<funcalc> +programming is the implicit use of the wrong data type for a column in +expression. For example, the calculation: + + dx = (cur->x - cur->y)/(cur->x + cur->y); + +usually needs to be performed using floating point arithmetic. In +cases where the x and y columns are integers, this can be done by +reading the columns as doubles using an explicit type specification: + + dx = (cur->x:D - cur->y:D)/(cur->x + cur->y); + + +Alternatively, it can be done using C type-casting in the expression: + + dx = ((double)cur->x - (double)cur->y)/((double)cur->x + (double)cur->y); + + + +In addition to accessing columns in the current row, reference also +can be made to the B<previous> row using B<prev->[colname]>, +and to the B<next> row using B<next->[colname]>. Note that if +B<prev->[colname]> is specified in the B<funcalc> +expression, the very first row is not processed. If +B<next->[colname]> is specified in the B<funcalc> +expression, the very last row is not processed. In this way, +B<prev> and B<next> are guaranteed always to point to valid +rows. For example, to print out the values of the current x column +and the previous y column, use the C fprintf function in a +B<funcalc> expression: + + fprintf(stdout, "%d %d\n", cur->x, prev->y); + + + +New columns can be specified using the same B<cur->[colname]> +syntax by appending the column type (and optional tlmin/tlmax/binsiz +specifiers), separated by colons. For example, cur->avg:D will define +a new column of type double. Type specifiers are the same those +used above to specify new data types for existing columns. + + +For example, to create and output a new column that is the average value of the +x and y columns, a new "avg" column can be defined: + + cur->avg:D = (cur->x + cur->y)/2.0 + +Note that the final ';' is not required for single-line expressions. + + +As with FITS TFORM data type specification, the column data type +specifier can be preceded by a numeric count to define an array, e.g., +"10I" means a vector of 10 short ints, "2E" means two single precision +floats, etc. A new column only needs to be defined once in a +B<funcalc> expression, after which it can be used without +re-specifying the type. This includes reference to elements of a +column array: + + + cur->avg[0]:2D = (cur->x + cur->y)/2.0; + cur->avg[1] = (cur->x - cur->y)/2.0; + + + +The 'X' (bits) data type is treated as a char array of dimension +(numeric_count/8), i.e., 16X is processed as a 2-byte char array. Each +8-bit array element is accessed separately: + + cur->stat[0]:16X = 1; + cur->stat[1] = 2; + +Here, a 16-bit column is created with the MSB is set to 1 and the LSB set to 2. + + +By default, all processed rows are written to the specified output +file. If you want to skip writing certain rows, simply execute the C +"continue" statement at the end of the B<funcalc> expression, +since the writing of the row is performed immediately after the +expression is executed. For example, to skip writing rows whose +average is the same as the current x value: + + + cur->avg[0]:2D = (cur->x + cur->y)/2.0; + cur->avg[1] = (cur->x - cur->y)/2.0; + if( cur->avg[0] == cur->x ) + continue; + + + +If no output file argument is specified on the B<funcalc> command +line, no output file is opened and no rows are written. This is useful +in expressions that simply print output results instead of generating +a new file: + + fpv = (cur->av3:D-cur->av1:D)/(cur->av1+cur->av2:D+cur->av3); + fbv = cur->av2/(cur->av1+cur->av2+cur->av3); + fpu = ((double)cur->au3-cur->au1)/((double)cur->au1+cur->au2+cur->au3); + fbu = cur->au2/(double)(cur->au1+cur->au2+cur->au3); + fprintf(stdout, "%f\t%f\t%f\t%f\n", fpv, fbv, fpu, fbu); + +In the above example, we use both explicit type specification +(for "av" columns) and type casting (for "au" columns) to ensure that +all operations are performed in double precision. + + +When an output file is specified, the selected input table is +processed and output rows are copied to the output file. Note that +the output file can be specified as "stdout" in order to write the +output rows to the standard output. If the output file argument is +passed, an optional third argument also can be passed to specify which +columns to process. + + +In a FITS binary table, it sometimes is desirable to copy all of the +other FITS extensions to the output file as well. This can be done by +appending a '+' sign to the name of the extension in the input file +name. See B<funtable> for a related example. + + +B<funcalc> works by integrating the user-specified expression +into a template C program called tabcalc.c. +The completed program then is compiled and executed. Variable +declarations that begin the B<funcalc> expression are placed in +the local declaration section of the template main program. All other +lines are placed in the template main program's inner processing +loop. Other details of program generation are handled +automatically. For example, column specifiers are analyzed to build a +C struct for processing rows, which is passed to +FunColumnSelect() and used +in FunTableRowGet(). If +an unknown variable is used in the expression, resulting in a +compilation error, the program build is retried after defining the +unknown variable to be of type double. + + +Normally, B<funcalc> expression code is added to +B<funcalc> row processing loop. It is possible to add code +to other parts of the program by placing this code inside +special directives of the form: + + [directive name] + ... code goes here ... + end + + +The directives are: + + +=over 4 + + + + +=item * + +B<global> add code and declarations in global space, before the main routine. + + + +=item * + +B<local> add declarations (and code) just after the local declarations in +main + + + +=item * + +B<before> add code just before entering the main row processing loop + + + +=item * + +B<after> add code just after exiting the main row processing loop + + +=back + + + +Thus, the following B<funcalc> expression will declare global +variables and make subroutine calls just before and just after the +main processing loop: + + global + double v1, v2; + double init(void); + double finish(double v); + end + before + v1 = init(); + end + ... process rows, with calculations using v1 ... + after + v2 = finish(v1); + if( v2 < 0.0 ){ + fprintf(stderr, "processing failed %g -> %g\n", v1, v2); + exit(1); + } + end + +Routines such as init() and finish() above are passed to the generated +program for linking using the B<-l [link directives ...]> +switch. The string specified by this switch will be added to the link +line used to build the program (before the funtools library). For +example, assuming that init() and finish() are in the library +libmysubs.a in the /opt/special/lib directory, use: + + funcalc -l "-L/opt/special/lib -lmysubs" ... + + + +User arguments can be passed to a compiled funcalc program using a string +argument to the "-a" switch. The string should contain all of the +user arguments. For example, to pass the integers 1 and 2, use: + + funcalc -a "1 2" ... + +The arguments are stored in an internal array and are accessed as +strings via the ARGV(n) macro. For example, consider the following +expression: + + local + int pmin, pmax; + end + + before + pmin=atoi(ARGV(0)); + pmax=atoi(ARGV(1)); + end + + if( (cur->pha >= pmin) && (cur->pha <= pmax) ) + fprintf(stderr, "%d %d %d\n", cur->x, cur->y, cur->pha); + +This expression will print out x, y, and pha values for all rows in which +the pha value is between the two user-input values: + + funcalc -a '1 12' -f foo snr.ev'[cir 512 512 .1]' + 512 512 6 + 512 512 8 + 512 512 5 + 512 512 5 + 512 512 8 + + funcalc -a '5 6' -f foo snr.ev'[cir 512 512 .1]' + 512 512 6 + 512 512 5 + 512 512 5 + + + +Note that it is the user's responsibility to ensure that the correct +number of arguments are passed. The ARGV(n) macro returns a NULL if a +requested argument is outside the limits of the actual number of args, +usually resulting in a SEGV if processed blindly. To check the +argument count, use the ARGC macro: + + local + long int seed=1; + double limit=0.8; + end + + before + if( ARGC >= 1 ) seed = atol(ARGV(0)); + if( ARGC >= 2 ) limit = atof(ARGV(1)); + srand48(seed); + end + + if ( drand48() > limit ) continue; + + + +The macro WRITE_ROW expands to the FunTableRowPut() call that writes +the current row. It can be used to write the row more than once. In +addition, the macro NROW expands to the row number currently being +processed. Use of these two macros is shown in the following example: + + if( cur->pha:I == cur->pi:I ) continue; + a = cur->pha; + cur->pha = cur->pi; + cur->pi = a; + cur->AVG:E = (cur->pha+cur->pi)/2.0; + cur->NR:I = NROW; + if( NROW < 10 ) WRITE_ROW; + + + +If the B<-p [prog]> switch is specified, the expression is not +executed. Rather, the generated executable is saved with the specified +program name for later use. + + +If the B<-n> switch is specified, the expression is not +executed. Rather, the generated code is written to stdout. This is +especially useful if you want to generate a skeleton file and add your +own code, or if you need to check compilation errors. Note that the +comment at the start of the output gives the compiler command needed +to build the program on that platform. (The command can change from +platform to platform because of the use of different libraries, +compiler switches, etc.) + + +As mentioned previously, B<funcalc> will declare a scalar +variable automatically (as a double) if that variable has been used +but not declared. This facility is implemented using a sed script +named funcalc.sed, which processes the +compiler output to sense an undeclared variable error. This script +has been seeded with the appropriate error information for gcc, and for +cc on Solaris, DecAlpha, and SGI platforms. If you find that automatic +declaration of scalars is not working on your platform, check this sed +script; it might be necessary to add to or edit some of the error +messages it senses. + + +In order to keep the lexical analysis of B<funcalc> expressions +(reasonably) simple, we chose to accept some limitations on how +accurately C comments, spaces, and new-lines are placed in the +generated program. In particular, comments associated with local +variables declared at the beginning of an expression (i.e., not in a +B<local...end> block) will usually end up in the inner loop, not +with the local declarations: + + /* this comment will end up in the wrong place (i.e, inner loop) */ + double a; /* also in wrong place */ + /* this will be in the the right place (inner loop) */ + if( cur->x:D == cur->y:D ) continue; /* also in right place */ + a = cur->x; + cur->x = cur->y; + cur->y = a; + cur->avg:E = (cur->x+cur->y)/2.0; + +Similarly, spaces and new-lines sometimes are omitted or added in a +seemingly arbitrary manner. Of course, none of these stylistic +blemishes affect the correctness of the generated code. + + +Because B<funcalc> must analyze the user expression using the data +file(s) passed on the command line, the input file(s) must be opened +and read twice: once during program generation and once during +execution. As a result, it is not possible to use stdin for the +input file: B<funcalc> cannot be used as a filter. We will +consider removing this restriction at a later time. + + +Along with C comments, B<funcalc> expressions can have one-line +internal comments that are not passed on to the generated C +program. These internal comment start with the B<#> character and +continue up to the new-line: + + double a; # this is not passed to the generated C file + # nor is this + a = cur->x; + cur->x = cur->y; + cur->y = a; + /* this comment is passed to the C file */ + cur->avg:E = (cur->x+cur->y)/2.0; + + + +As previously mentioned, input columns normally are identified by +their being used within the inner event loop. There are rare cases +where you might want to read a column and process it outside the main +loop. For example, qsort might use a column in its sort comparison +routine that is not processed inside the inner loop (and therefore not +implicitly specified as a column to be read). To ensure that such a +column is read by the event loop, use the B<explicit> keyword. +The arguments to this keyword specify columns that should be read into +the input record structure even though they are not mentioned in the +inner loop. For example: + + explicit pi pha + +will ensure that the pi and pha columns are read for each row, +even if they are not processed in the inner event loop. The B<explicit> +statement can be placed anywhere. + + +Finally, note that B<funcalc> currently works on expressions +involving FITS binary tables and raw event files. We will consider +adding support for image expressions at a later point, if there is +demand for such support from the community. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funcen.pod b/funtools/doc/pod/funcen.pod new file mode 100644 index 0000000..3f10795 --- /dev/null +++ b/funtools/doc/pod/funcen.pod @@ -0,0 +1,145 @@ +=pod + +=head1 NAME + + + +B<funcen - find centroid (for binary tables)> + + +=head1 SYNOPSIS + + + + + +funcen [-i] [-n iter] [-t tol] [-v lev] <iname> <region> + + + + + +=head1 OPTIONS + + + + + + -i # use image filtering (default: event filtering) + -n iter # max number of iterations (default: 0) + -t tol # pixel tolerance distance (default: 1.0) + -v [0,1,2,3] # output verbosity level (default: 0) + + + + +=head1 DESCRIPTION + + + + +B<funcen> iteratively calculates the centroid position within one +or more regions of a Funtools table (FITS binary table or raw event +file). Starting with an input table, an initial region specification, +and an iteration count, the program calculates the average x and y +position within the region and then uses this new position as the +region center for the next iteration. Iteration terminates when the +maximum number of iterations is reached or when the input tolerance +distance is met for that region. A count of events in the final region +is then output, along with the pixel position value (and, where +available, WCS position). + + +The first argument to the program specifies the Funtools table file to +process. Since the file must be read repeatedly, a value of "stdin" +is not permitted when the number of iterations is non-zero. Use +Funtools Bracket Notation to specify FITS +extensions and filters. + + +The second required argument is the initial region descriptor. Multiple +regions are permitted. However, compound regions (accelerators, +variable argument regions and regions connected via boolean algebra) +are not permitted. Points and polygons also are illegal. These +restrictions might be lifted in a future version, if warranted. + + +The B<-n> (iteration number) switch specifies the maximum number of +iterations to perform. The default is 0, which means that the program will +simply count and display the number of events in the initial region(s). +Note that when iterations is 0, the data can be input via stdin. + + +The B<-t> (tolerance) switch specifies a floating point tolerance +value. If the distance between the current centroid position value and +the last position values is less than this value, iteration terminates. +The default value is 1 pixel. + + +The B<-v> (verbosity) switch specifies the verbosity level of the +output. The default is 0, which results in a single line of output for +each input region consisting of the following values: + + counts x y [ra dec coordsys] + +The last 3 WCS values are output if WCS information is available in the +data file header. Thus, for example: + + [sh] funcen -n 0 snr.ev "cir 505 508 5" + 915 505.00 508.00 345.284038 58.870920 j2000 + + [sh] funcen -n 3 snr.ev "cir 505 508 5" + 1120 504.43 509.65 345.286480 58.874587 j2000 + +The first example simply counts the number of events in the initial region. +The second example iterates the centroid calculation three times to determine +a final "best" position. + + +Higher levels of verbosity obviously imply more verbose output. At +level 1, the output essentially contains the same information as level +0, but with keyword formatting: + + [sh] funcen -v 1 -n 3 snr.ev "cir 505 508 5" + event_file: snr.ev + initial_region: cir 505 508 5 + tolerance: 1.0000 + iterations: 1 + + events: 1120 + x,y(physical): 504.43 509.65 + ra,dec(j2000): 345.286480 58.874587 + final_region1: cir 504.43 509.65 5 + +Level 2 outputs results from intermediate calculations as well. + + +Ordinarily, region filtering is performed using analytic (event) +filtering, i.e. that same style of filtering as is performed by +B<fundisp> and B<funtable>. Use the B<-i> switch to specify image +filtering, i.e. the same style filtering as is performed by B<funcnts>. +Thus, you can perform a quick calculation of counts in regions, using +either the analytic or image filtering method, by specifying the + B<-n 0> and optional B<-i> switches. These two method often +give different results because of how boundary events are processed: + + [sh] funcen snr.ev "cir 505 508 5" + 915 505.00 508.00 345.284038 58.870920 j2000 + + [sh] funcen -i snr.ev "cir 505 508 5" + 798 505.00 508.00 345.284038 58.870920 j2000 + +See Region Boundaries for more information +about how boundaries are calculated using these two methods. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funclose.pod b/funtools/doc/pod/funclose.pod new file mode 100644 index 0000000..48357a3 --- /dev/null +++ b/funtools/doc/pod/funclose.pod @@ -0,0 +1,53 @@ +=pod + +=head1 NAME + + + +B<FunClose - close a Funtools data file> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void FunClose(Fun fun) + + + + + +=head1 DESCRIPTION + + + + +The B<FunClose()> routine closes a previously-opened Funtools data +file, freeing control structures. If a +Funtools reference handle +was passed to +the FunOpen() call for this file, +and if copy mode also was specified for that file, then +FunClose() also will copy the +remaining extensions from the input file to the output file (if the +input file still is open). Thus, we recommend always closing the +output Funtools file B<before> the input file. (Alternatively, +you can call FunFlush() +explicitly). + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funcnts.pod b/funtools/doc/pod/funcnts.pod new file mode 100644 index 0000000..7d5dfe5 --- /dev/null +++ b/funtools/doc/pod/funcnts.pod @@ -0,0 +1,657 @@ +=pod + +=head1 NAME + + + +B<funcnts - count photons in specified regions, with bkgd subtraction> + + + +=head1 SYNOPSIS + + + + + +funcnts [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_value] + + + + + +=head1 OPTIONS + + + + + + -e "source_exposure[;bkgd_exposure]" + # source (bkgd) FITS exposure image using matching files + -w "source_exposure[;bkgd_exposure]" + # source (bkgd) FITS exposure image using WCS transform + -t "source_timecorr[;bkgd_timecorr]" + # source (bkgd) time correction value or header parameter name + -g # output using nice g format + -G # output using %.14g format (maximum precision) + -i "[column;]int1;int2..." # column-based intervals + -m # match individual source and bkgd regions + -p # output in pixels, even if wcs is present + -r # output inner/outer radii (and angles) for annuli (and pandas) + -s # output summed values + -v "scol[;bcol]" # src and bkgd value columns for tables + -T # output in starbase/rdb format + -z # output regions with zero area + + + + + +=head1 DESCRIPTION + + + + +B<funcnts> counts photons in the specified source regions and +reports the results for each region. Regions are specified using the +Spatial Region Filtering mechanism. +Photons are also counted in the specified bkgd regions applied to the +same data file or a different data file. (Alternatively, a constant +background value in counts/pixel**2 can be specified.) The bkgd regions +are either paired one-to-one with source regions or pooled and +normalized by area, and then subtracted from the source counts in each +region. Displayed results include the bkgd-subtracted counts in each +region, as well as the error on the counts, the area in +each region, and the surface brightness (cnts/area**2) calculated for +each region. + + +The first argument to the program specifies the FITS input image, array, or +raw event file to process. If "stdin" is specified, data are read from +the standard input. Use Funtools Bracket +Notation to specify FITS extensions, image sections, and filters. + + +The optional second argument is the source region descriptor. If no +region is specified, the entire field is used. + + +The background arguments can take one of two forms, depending on +whether a separate background file is specified. If the source +file is to be used for background as well, the third argument can be +either the background region, or a constant value denoting background +cnts/pixel. Alternatively, the third argument can be a background +data file, in which case the fourth argument is the background region. +If no third argument is specified, a constant value of 0 is used +(i.e., no background). + + +In summary, the following command arguments are valid: + + [sh] funcnts sfile # counts in source file + [sh] funcnts sfile sregion # counts in source region + [sh] funcnts sfile sregion bregion # bkgd reg. is from source file + [sh] funcnts sfile sregion bvalue # bkgd reg. is constant + [sh] funcnts sfile sregion bfile bregion # bkgd reg. is from separate file + + + +NB: unlike other Funtools programs, source and background regions are +specified as separate arguments on the command line, rather than being +placed inside brackets as part of the source and background filenames. +This is because regions in funcnts are not simply used as data +filters, but also are used to calculate areas, exposure, etc. If you +put the source region inside the brackets (i.e. use it simply as a +filter) rather than specifying it as argument two, the program still +will only count photons that pass the region filter. However, the area +calculation will be performed on the whole field, since field() is the +default source region. This rarely is the desired behavior. On the +other hand, with FITS binary tables, it often is useful to put a column +filter in the filename brackets, so that only events matching the +column filter are counted inside the region. + + +For example, to extract the counts within a radius of 22 pixels from the +center of the FITS binary table snr.ev and subtract the background determined +from the same image within an annulus of radii 50-100 pixels: + + [sh] funcnts snr.ev "circle(502,512,22)" "annulus(502,512,50,100)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001 + + + # the following source and background components were used: + source region(s) + ---------------- + circle(502,512,22) + + reg counts pixels + ---- ------------ --------- + 1 4382.000 1513 + + background region(s) + -------------------- + annulus(502,512,50,100) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 + +The area units for the output columns labeled "area", "surf_bri" +(surface brightness) and "surf_err" will be given either in +arc-seconds (if appropriate WCS information is in the data file +header(s)) or in pixels. If the data file has WCS info, but you do not +want arc-second units, use the B<-p> switch to force output in +pixels. Also, regions having zero area are not normally included in +the primary (background-subtracted) table, but are included in the +secondary source and bkgd tables. If you want these regions to be +included in the primary table, use the B<-z> switch. + + +Note that a simple sed command will extract the background-subtracted results +for further analysis: + + [sh] cat funcnts.sed + 1,/---- .*/d + /^$/,$d + + [sh] sed -f funcnts.sed funcnts.out + 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001 + + + +If separate source and background files are specified, B<funcnts> will +attempt to normalize the the background area so that the background +pixel size is the same as the source pixel size. This normalization +can only take place if the appropriate WCS information is contained in +both files (e.g. degrees/pixel values in CDELT). If either +file does not contain the requisite size information, the normalization +is not performed. In this case, it is the user's responsibility to +ensure that the pixel sizes are the same for the two files. + + +Normally, if more than one background region is specified, B<funcnts> +will combine them all into a single region and use this background +region to produce the background-subtracted results for each source +region. The B<-m> (match multiple backgrounds) switch tells +B<funcnts> to make a one to one correspondence between background and +source regions, instead of using a single combined background region. +For example, the default case is to combine 2 background +regions into a single region and then apply that region to each of the +source regions: + + + [sh] funcnts snr.ev "annulus(502,512,0,22,n=2)" "annulus(502,512,50,100,n=2)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 + 2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,22,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region(s) + -------------------- + annulus(502,512,50,100,n=2) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 + +Note that the basic region filter rule "each photon is counted once +and no photon is counted more than once" still applies when using The +B<-m> to match background regions. That is, if two background +regions overlap, the overlapping pixels will be counted in only one of +them. In a worst-case scenario, if two background regions are the same +region, the first will get all the counts and area and the second +will get none. + + +Using the B<-m> switch causes B<funcnts> to use each of the two +background regions independently with each of the two source regions: + + + [sh] funcnts -m snr.ev "annulus(502,512,0,22,n=2)" "ann(502,512,50,100,n=2)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3087.015 56.954 150.985 2.395 23872.00 0.129 0.002 + 2 755.959 34.295 388.041 5.672 72959.99 0.010 0.000 + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,22,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region(s) + -------------------- + ann(502,512,50,100,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3975.000 9820 + 2 4681.000 13752 + + + +Note that most floating point quantities are displayed using "f" +format. You can change this to "g" format using the B<-g> +switch. This can be useful when the counts in each pixel is very +small or very large. If you want maximum precision and don't care +about the columns lining up nicely, use B<-G>, which outputs +all floating values as %.14g. + + +When counting photons using the annulus and panda (pie and annuli) +shapes, it often is useful to have access to the radii (and panda +angles) for each separate region. The B<-r> switch will add radii +and angle columns to the output table: + + + [sh] funcnts -r snr.ev "annulus(502,512,0,22,n=2)" "ann(502,512,50,100,n=2)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + # radii: arcsecs + # angles: degrees + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err radius1 radius2 angle1 angle2 + ---- ------------ --------- ------------ --------- --------- --------- --------- --------- --------- --------- --------- + 1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 0.00 88.00 NA NA + 2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 88.00 176.00 NA NA + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,22,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region(s) + -------------------- + ann(502,512,50,100,n=2) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 + + + +Radii are given in units of pixels or arc-seconds (depending on the +presence of WCS info), while the angle values (when present) are in +degrees. These columns can be used to plot radial profiles. For +example, the script B<funcnts.plot> in the funtools +distribution) will plot a radial profile using gnuplot (version 3.7 or +above). A simplified version of this script is shown below: + + + #!/bin/sh + + if [ x"$1" = xgnuplot ]; then + if [ x`which gnuplot 2>/dev/null` = x ]; then + echo "ERROR: gnuplot not available" + exit 1 + fi + awk ' + BEGIN{HEADER=1; DATA=0; FILES=""; XLABEL="unknown"; YLABEL="unknown"} + HEADER==1{ + if( $1 == "#" && $2 == "data" && $3 == "file:" ){ + if( FILES != "" ) FILES = FILES "," + FILES = FILES $4 + } + else if( $1 == "#" && $2 == "radii:" ){ + XLABEL = $3 + } + else if( $1 == "#" && $2 == "surf_bri:" ){ + YLABEL = $3 + } + else if( $1 == "----" ){ + printf "set nokey; set title \"funcnts(%s)\"\n", FILES + printf "set xlabel \" radius(%s)\"\n", XLABEL + printf "set ylabel \"surf_bri(%s)\"\n", YLABEL + print "plot \"-\" using 3:4:6:7:8 with boxerrorbars" + HEADER = 0 + DATA = 1 + next + } + } + DATA==1{ + if( NF == 12 ){ + print $9, $10, ($9+$10)/2, $7, $8, $7-$8, $7+$8, $10-$9 + } + else{ + exit + } + } + ' | gnuplot -persist - 1>/dev/null 2>&1 + + elif [ x"$1" = xds9 ]; then + awk ' + BEGIN{HEADER=1; DATA=0; XLABEL="unknown"; YLABEL="unknown"} + HEADER==1{ + if( $1 == "#" && $2 == "data" && $3 == "file:" ){ + if( FILES != "" ) FILES = FILES "," + FILES = FILES $4 + } + else if( $1 == "#" && $2 == "radii:" ){ + XLABEL = $3 + } + else if( $1 == "#" && $2 == "surf_bri:" ){ + YLABEL = $3 + } + else if( $1 == "----" ){ + printf "funcnts(%s) radius(%s) surf_bri(%s) 3\n", FILES, XLABEL, YLABEL + HEADER = 0 + DATA = 1 + next + } + } + DATA==1{ + if( NF == 12 ){ + print $9, $7, $8 + } + else{ + exit + } + } + ' + else + echo "funcnts -r ... | funcnts.plot [ds9|gnuplot]" + exit 1 + fi + + +Thus, to run B<funcnts> and plot the results using gnuplot (version 3.7 +or above), use: + + funcnts -r snr.ev "annulus(502,512,0,50,n=5)" ... | funcnts.plot gnuplot + + + +The B<-s> (sum) switch causes B<funcnts> to produce an +additional table of summed (integrated) background subtracted values, +along with the default table of individual values: + + + [sh] funcnts -s snr.ev "annulus(502,512,0,50,n=5)" "annulus(502,512,50,100)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # summed background-subtracted results + upto net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003 + 2 3776.817 65.254 457.183 4.914 79679.98 0.047 0.001 + 3 4025.492 71.972 1031.508 11.087 179775.96 0.022 0.000 + 4 4185.149 80.109 1840.851 19.786 320831.94 0.013 0.000 + 5 4415.540 90.790 2873.460 30.885 500799.90 0.009 0.000 + + + # background-subtracted results + reg counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003 + 2 895.818 35.423 345.182 3.710 60159.99 0.015 0.001 + 3 248.675 29.345 574.325 6.173 100095.98 0.002 0.000 + 4 159.657 32.321 809.343 8.699 141055.97 0.001 0.000 + 5 230.390 37.231 1032.610 11.099 179967.96 0.001 0.000 + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,50,n=5) + + reg counts pixels sumcnts sumpix + ---- ------------ --------- ------------ --------- + 1 2993.000 305 2993.000 305 + 2 1241.000 940 4234.000 1245 + 3 823.000 1564 5057.000 2809 + 4 969.000 2204 6026.000 5013 + 5 1263.000 2812 7289.000 7825 + + background region(s) + -------------------- + annulus(502,512,50,100) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 + + + +The B<-t> and B<-e> switches can be used to apply timing and +exposure corrections, respectively, to the data. Please note that +these corrections are meant to be used qualitatively, since +application of more accurate correction factors is a complex and +mission-dependent effort. The algorithm for applying these simple +corrections is as follows: + + C = Raw Counts in Source Region + Ac= Area of Source Region + Tc= Exposure time for Source Data + Ec= Average exposure in Source Region, from exposure map + + B= Raw Counts in Background Region + Ab= Area of Background Region + Tb= (Exposure) time for Background Data + Eb= Average exposure in Background Region, from exposure map + +Then, Net Counts in Source region is + + Net= C - B * (Ac*Tc*Ec)/(Ab*Tb*Eb) + +with the standard propagation of errors for the Error on Net. +The net rate would then be + + Net Rate = Net/(Ac*Tc*Ec) + +The average exposure in each region is calculated by summing up the +pixel values in the exposure map for the given region and then +dividing by the number of pixels in that region. Exposure maps often +are generated at a block factor > 1 (e.g., block 4 means that each +exposure pixel contains 4x4 pixels at full resolution) and +B<funcnts> will deal with the blocking automatically. Using the +B<-e> switch, you can supply both source and background exposure +files (separated by ";"), if you have separate source and background +data files. If you do not supply a background exposure file to go with +a separate background data file, B<funcnts> assumes that exposure +already has been applied to the background data file. In addition, it +assumes that the error on the pixels in the background data file is +zero. + + +NB: The B<-e> switch assumes that the exposure map overlays the +image file B<exactly>, except for the block factor. Each pixel in +the image is scaled by the block factor to access the corresponding +pixel in the exposure map. If your exposure map does not line up +exactly with the image, B<do not use> the B<-e> exposure +correction. In this case, it still is possible to perform exposure +correction B<if> both the image and the exposure map have valid +WCS information: use the B<-w> switch so that the transformation +from image pixel to exposure pixel uses the WCS information. That is, +each pixel in the image region will be transformed first from image +coordinates to sky coordinates, then from sky coordinates to exposure +coordinates. Please note that using B<-w> can increase the time +required to process the exposure correction considerably. + + +A time correction can be applied to both source and +background data using the B<-t> switch. The value for the correction can +either be a numeric constant or the name of a header parameter in +the source (or background) file: + + [sh] funcnts -t 23.4 ... # number for source + [sh] funcnts -t "LIVETIME;23.4" ... # param for source, numeric for bkgd + +When a time correction is specified, it is applied to the net counts +as well (see algorithm above), so that the units of surface brightness +become cnts/area**2/sec. + + +The B<-i> (interval) switch is used to run B<funcnts> on multiple +column-based intervals with only a single pass through the data. It is +equivalent to running B<funcnts> several times with a different column +filter added to the source and background data each time. For each +interval, the full B<funcnts> output is generated, with a linefeed +character (^L) inserted between each run. In addition, the output for +each interval will contain the interval specification in its header. +Intervals are very useful for generating X-ray hardness ratios +efficiently. Of course, they are only supported when the input data +are contained in a table. + + +Two formats are supported for interval specification. The most general +format is semi-colon-delimited list of filters to be used as intervals: + + funcnts -i "pha=1:5;pha=6:10;pha=11:15" snr.ev "circle(502,512,22)" ... + +Conceptually, this will be equivalent to running B<funcnts> three times: + + funcnts snr.ev'[pha=1:5]' "circle(502,512,22)" + funcnts snr.ev'[pha=6:10]' "circle(502,512,22)" + funcnts snr.ev'[pha=11:15]' "circle(502,512,22)" + +However, using the B<-i> switch will require only one pass through +the data. + + +Note that complex filters can be used to specify intervals: + + funcnts -i "pha=1:5&&pi=4;pha=6:10&&pi=5;pha=11:15&&pi=6" snr.ev ... + +The program simply runs the data through each filter in turn and generates +three B<funcnts> outputs, separated by the line-feed character. + + +In fact, although the intent is to support intervals for hardness ratios, +the specified filters do not have to be intervals at all. Nor does one +"interval" filter have to be related to another. For example: + + funcnts -i "pha=1:5;pi=6:10;energy=11:15" snr.ev "circle(502,512,22)" ... + +is equivalent to running B<funcnts> three times with unrelated filter +specifications. + + +A second interval format is supported for the simple case in which a +single column is used to specify multiple homogeneous intervals for +that column. In this format, a column name is specified first, +followed by intervals: + + funcnts -i "pha;1:5;6:10;11:15" snr.ev "circle(502,512,22)" ... + +This is equivalent to the first example, but requires less typing. The +B<funcnts> program will simply prepend "pha=" before each of the specified +intervals. (Note that this format does not contain the "=" character in +the column argument.) + + +Ordinarily, when B<funcnts> is run on a FITS binary table (or a +raw event table), one integral count is accumulated for each row +(event) contained within a given region. The B<-v "scol[;bcol]"> +(value column) switch will accumulate counts using the value from the +specified column for the given event. If only a single column is +specified, it is used for both the source and background regions. Two +separate columns, separated by a semi-colon, can be specified for source +and background. The special token '$none' can be used to specify that +a value column is to be used for one but not the other. For example, +'pha;$none' will use the pha column for the source but use integral +counts for the background, while '$none;pha' will do the converse. +If the value column is of type logical, then the value used will be 1 +for T and 0 for F. Value columns are used, for example, to integrate +probabilities instead of integral counts. + + +If the B<-T> (rdb table) switch is used, the output will conform +to starbase/rdb data base format: tabs will be inserted between +columns rather than spaces and line-feed will be inserted between +tables. + + +Finally, note that B<funcnts> is an image program, even though it +can be run directly on FITS binary tables. This means that image +filtering is applied to the rows in order to ensure that the same +results are obtained regardless of whether a table or the equivalent +binned image is used. Because of this, however, the number of counts +found using B<funcnts> can differ from the number of events found +using row-filter programs such as B<fundisp> or B<funtable> +For more information about these difference, see the discussion of +Region Boundaries. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funcolumnactivate.pod b/funtools/doc/pod/funcolumnactivate.pod new file mode 100644 index 0000000..d5f6e3a --- /dev/null +++ b/funtools/doc/pod/funcolumnactivate.pod @@ -0,0 +1,239 @@ +=pod + +=head1 NAME + + + +B<FunColumnActivate - activate Funtools columns> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void FunColumnActivate(Fun fun, char *s, char *plist) + + + + + +=head1 DESCRIPTION + + + + +The B<FunColumnActivate()> routine determines which columns (set up +by FunColumnSelect()) +ultimately will be read and/or written. By default, all columns that +are selected using +FunColumnSelect() +are activated. The +FunColumnActivate() +routine can be used to turn off/off activation of specific columns. + + +The first argument is the Fun handle associated with this set of +columns. The second argument is a space-delimited list of columns to +activate or de-activate. Columns preceded by "+" are activated and +columns preceded by a "-" are de-activated. If a column is named +without "+" or "-", it is activated. The reserved strings "$region" +and '$n' are used to activate a special columns containing the filter +region value and row value, respectively, associated with +this row. For example, if a filter containing two circular regions is +specified as part of the Funtools file name, this column will contain +a value of 1 or 2, depending on which region that row was in. The +reserved strings "$x" and "$y" are used to activate the current +binning columns. Thus, if the columns DX and DY are specified as +binning columns: + + [sh $] fundisp foo.fits[bincols=(DX,DY)] + +then "$x" and "$y" will refer to these columns in a call to +FunColumnActivate(). + + +In addition, if the activation string contains only columns to be +activated, then the routine will de-activate all other columns. +Similarly, if the activation string contains only +columns to de-activate, then the routine will activate all other columns +before activating the list. This makes it simple to change the +activation state of all columns without having to know all of the +column names. For example: + + +=over 4 + + + + +=item * + +B<"pi pha time"> # only these three columns will be active + + +=item * + +B<"-pi -pha -time"> # all but these columns will be active + + +=item * + +B<"pi -pha"> # only pi is active, pha is not, others are not + + +=item * + +B<"+pi -pha"> # same as above + + +=item * + +B<"pi -pha -time"> # only pi is active, all others are not + + +=item * + +B<"pi pha"> # pha and pi are active, all others are not + + +=item * + +B<"pi pha -x -y"> # pha and pi are active, all others are not + + +=back + + + + +You can use the column activation list to reorder columns, since +columns are output in the order specified. For example: + + # default output order + fundisp snr.ev'[cir 512 512 .1]' + X Y PHA PI TIME DX DY + -------- -------- -------- -------- --------------------- -------- -------- + 512 512 6 7 79493997.45854475 578 574 + 512 512 8 9 79494575.58943175 579 573 + 512 512 5 6 79493631.03866175 578 575 + 512 512 5 5 79493290.86521725 578 575 + 512 512 8 9 79493432.00990875 579 573 + + # re-order the output by specifying explicit order + fundisp snr.ev'[cir 512 512 .1]' "time x y dy dx pi pha" + TIME X Y DY DX PI PHA + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 512 512 574 578 7 6 + 79494575.58943175 512 512 573 579 9 8 + 79493631.03866175 512 512 575 578 6 5 + 79493290.86521725 512 512 575 578 5 5 + 79493432.00990875 512 512 573 579 9 8 + + + +A "+" sign by itself means to activate all columns, so that you can reorder +just a few columns without specifying all of them: + + # reorder 3 columns and then output the rest + fundisp snr.ev'[cir 512 512 .1]' "time pi pha +" + TIME PI PHA Y X DX DY + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 7 6 512 512 578 574 + 79494575.58943175 9 8 512 512 579 573 + 79493631.03866175 6 5 512 512 578 575 + 79493290.86521725 5 5 512 512 578 575 + 79493432.00990875 9 8 512 512 579 573 + +The column activation/deactivation is performed in the order of the +specified column arguments. This means you can mix "+", "-" (which +de-activates all columns) and specific column names to reorder and +select columns in one command. For example, consider the following: + + # reorder and de-activate + fundisp snr.ev'[cir 512 512 .1]' "time pi pha + -x -y" + TIME PI PHA DX DY + --------------------- -------- -------- -------- -------- + 79493997.45854475 7 6 578 574 + 79494575.58943175 9 8 579 573 + 79493631.03866175 6 5 578 575 + 79493290.86521725 5 5 578 575 + 79493432.00990875 9 8 579 573 + +We first activate "time", "pi", and "pha" so that they are output first. +We then activate all of the other columns, and then de-activate "x" and "y". +Note that this is different from: + + # probably not what you want ... + fundisp snr.ev'[cir 512 512 .1]' "time pi pha -x -y +" + TIME PI PHA Y X DX DY + --------------------- -------- -------- -------- -------- -------- -------- + 79493997.45854475 7 6 512 512 578 574 + 79494575.58943175 9 8 512 512 579 573 + 79493631.03866175 6 5 512 512 578 575 + 79493290.86521725 5 5 512 512 578 575 + 79493432.00990875 9 8 512 512 579 573 + +Here, "x" and "y" are de-activated, but then all columns including "x" and +"y" are again re-activated. + + +Typically, +FunColumnActivate() uses a +list of columns that are passed into the program from the command line. For +example, the code for funtable contains the following: + + char *cols=NULL; + + /* open the input FITS file */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + + /* set active flag for specified columns */ + if( argc >= 4 ) cols = argv[3]; + FunColumnActivate(fun, cols, NULL); + + +The FunOpen() call sets the +default columns to be all columns in the input file. The +FunColumnActivate() call +then allows the user to control which columns ultimately will be +activated (i.e., in this case, written to the new file). For example: + + funtable test.ev foo.ev "pi pha time" + +will process only the three columns mentioned, while: + + funtable test.ev foo.ev "-time" + +will process all columns except "time". + + +If FunColumnActivate() +is called with a null string, then the environment variable +B<FUN_COLUMNS> will be used to provide a global value, if present. +This is the reason why we call the routine even if no columns +are specified on the command line (see example above), instead +of calling it this way: + + /* set active flag for specified columns */ + if( argc >= 4 ){ + FunColumnActivate(fun, argv[3], NULL); + } + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funcolumnlookup.pod b/funtools/doc/pod/funcolumnlookup.pod new file mode 100644 index 0000000..bfdf5c0 --- /dev/null +++ b/funtools/doc/pod/funcolumnlookup.pod @@ -0,0 +1,188 @@ +=pod + +=head1 NAME + + + +B<FunColumnLookup - lookup a Funtools column> + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunColumnLookup(Fun fun, char *s, int which, + char **name, int *type, int *mode, + int *offset, int *n, int *width) + + + + + +=head1 DESCRIPTION + + + + +The B<FunColumnLookup()> routine returns information about a named +(or indexed) column. The first argument is the Fun handle associated +with this set of columns. The second argument is the name of the +column to look up. If the name argument is NULL, the argument that +follows is the zero-based index into the column array of the column +for which information should be returned. The next argument is a +pointer to a char *, which will contain the name of the column. The +arguments that follow are the addresses of int values into which +the following information will be returned: + + +=over 4 + + + + +=item * + +B<type>: data type of column: + + +=over 4 + + + + +=item * + +A: ASCII characters + + +=item * + +B: unsigned 8-bit char + + +=item * + +I: signed 16-bit int + + +=item * + +U: unsigned 16-bit int (not standard FITS) + + +=item * + +J: signed 32-bit int + + +=item * + +V: unsigned 32-bit int (not standard FITS) + + +=item * + +E: 32-bit float + + +=item * + +D: 64-bit float + + +=back + + + + +=item * + +B<mode>: bit flag status of column, including: + + +=over 4 + + + + +=item * + +COL_ACTIVE 1 is column activated? + + +=item * + +COL_IBUF 2 is column in the raw input data? + + +=item * + +COL_PTR 4 is column a pointer to an array? + + +=item * + +COL_READ 010 is read mode selected? + + +=item * + +COL_WRITE 020 is write mode selected? + + +=item * + +COL_REPLACEME 040 is this column being replaced by user data? + + +=back + + + + +=item * + +B<offset>: byte offset in struct + + +=item * + +B<n>: number of elements (i.e. size of vector) in this column + + +=item * + +B<width>: size in bytes of this column + + +=back + + +If the named column exists, the routine returns a positive integer, +otherwise zero is returned. (The positive integer is the index+1 into +the column array where this column was located.) + +If NULL is passed as the return address of one (or more) of these +values, no data is passed back for that information. For +example: + + if( !FunColumnLookup(fun, "phas", 0, NULL NULL, NULL, NULL, &npha, NULL) ) + gerror(stderr, "can't find phas column\n"); + +only returns information about the size of the phas vector. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funcolumnselect.pod b/funtools/doc/pod/funcolumnselect.pod new file mode 100644 index 0000000..ce65d38 --- /dev/null +++ b/funtools/doc/pod/funcolumnselect.pod @@ -0,0 +1,686 @@ +=pod + +=head1 NAME + + + +B<FunColumnSelect - select Funtools columns> + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunColumnSelect(Fun fun, int size, char *plist, + char *name1, char *type1, char *mode1, int offset1, + char *name2, char *type2, char *mode2, int offset2, + ..., + NULL) + + int FunColumnSelectArr(Fun fun, int size, char *plist, + char **names, char **types, char **modes, + int *offsets, int nargs); + + + + + +=head1 DESCRIPTION + + + +The B<FunColumnSelect()> routine is used to select the columns +from a Funtools binary table extension or raw event file for +processing. This routine allows you to specify how columns in a file +are to be read into a user record structure or written from a user +record structure to an output FITS file. + + +The first argument is the Fun handle associated with this set of +columns. The second argument specifies the size of the user record +structure into which columns will be read. Typically, the sizeof() +macro is used to specify the size of a record structure. The third +argument allows you to specify keyword directives for the selection +and is described in more detail below. + + +Following the first three required arguments is a variable length list of +column specifications. Each column specification will consist of four +arguments: + + +=over 4 + + + + +=item * + +B<name>: the name of the column + + + +=item * + +B<type>: the data type of the column as it will be stored in +the user record struct (not the data type of the input file). The +following basic data types are recognized: + + +=over 4 + + + + +=item * + +A: ASCII characters + + +=item * + +B: unsigned 8-bit char + + +=item * + +I: signed 16-bit int + + +=item * + +U: unsigned 16-bit int (not standard FITS) + + +=item * + +J: signed 32-bit int + + +=item * + +V: unsigned 32-bit int (not standard FITS) + + +=item * + +E: 32-bit float + + +=item * + +D: 64-bit float + + +=back + + +The syntax used is similar to that which defines the TFORM parameter +in FITS binary tables. That is, a numeric repeat value can precede +the type character, so that "10I" means a vector of 10 short ints, "E" +means a single precision float, etc. Note that the column value from +the input file will be converted to the specified data type as the +data is read by +FunTableRowGet(). + + +[ A short digression regarding bit-fields: Special attention is +required when reading or writing the FITS bit-field type +("X"). Bit-fields almost always have a numeric repeat character +preceding the 'X' specification. Usually this value is a multiple of 8 +so that bit-fields fit into an integral number of bytes. For all +cases, the byte size of the bit-field B is (N+7)/8, where N is the +numeric repeat character. + + +A bit-field is most easily declared in the user struct as an array of +type char of size B as defined above. In this case, bytes are simply +moved from the file to the user space. If, instead, a short or int +scalar or array is used, then the algorithm for reading the bit-field +into the user space depends on the size of the data type used along +with the value of the repeat character. That is, if the user data +size is equal to the byte size of the bit-field, then the data is +simply moved (possibly with endian-based byte-swapping) from one to +the other. If, on the other hand, the data storage is larger than the +bit-field size, then a data type cast conversion is performed to move +parts of the bit-field into elements of the array. Examples will help +make this clear: + + + +=over 4 + + + + +=item * + +If the file contains a 16X bit-field and user space specifies a 2B +char array[2], then the bit-field is moved directly into the char array. + + + +=item * + +If the file contains a 16X bit-field and user space specifies a 1I +scalar short int, then the bit-field is moved directly into the short int. + + + +=item * + +If the file contains a 16X bit-field and user space specifies a 1J +scalar int, then the bit-field is type-cast to unsigned int before +being moved (use of unsigned avoids possible sign extension). + + + +=item * + +If the file contains a 16X bit-field and user space specifies a 2J +int array[2], then the bit-field is handled as 2 chars, each of which +are type-cast to unsigned int before being moved (use of unsigned +avoids possible sign extension). + + + +=item * + +If the file contains a 16X bit-field and user space specifies a 1B +char, then the bit-field is treated as a char, i.e., truncation will +occur. + + + +=item * + +If the file contains a 16X bit-field and user space specifies a 4J +int array[4], then the results are undetermined. + + + +=back + + +For all user data types larger than char, the bit-field is byte-swapped +as necessary to convert to native format, so that bits in the +resulting data in user space can be tested, masked, etc. in the same +way regardless of platform.] + + +In addition to setting data type and size, the B<type> +specification allows a few ancillary parameters to be set, using the +full syntax for B<type>: + + [@][n]<type>[[['B']poff]][:[tlmin[:tlmax[:binsiz]]]] + + + +The special character "@" can be prepended to this specification to +indicated that the data element is a pointer in the user record, +rather than an array stored within the record. + + +The [n] value is an integer that specifies the +number of elements that are in this column (default is 1). TLMIN, +TLMAX, and BINSIZ values also can be specified for this column after +the type, separated by colons. If only one such number is specified, +it is assumed to be TLMAX, and TLMIN and BINSIZ are set to 1. + + +The [poff] value can be used to specify the offset into an +array. By default, this offset value is set to zero and the data +specified starts at the beginning of the array. The offset usually +is specified in terms of the data type of the column. Thus an offset +specification of [5] means a 20-byte offset if the data type is a +32-bit integer, and a 40-byte offset for a double. If you want to +specify a byte offset instead of an offset tied to the column data type, +precede the offset value with 'B', e.g. [B6] means a 6-bye offset, +regardless of the column data type. + +The [poff] is especially useful in conjunction with the pointer @ +specification, since it allows the data element to anywhere stored +anywhere in the allocated array. For example, a specification such as +"@I[2]" specifies the third (i.e., starting from 0) element in the +array pointed to by the pointer value. A value of "@2I[4]" specifies +the fifth and sixth values in the array. For example, consider the +following specification: + + typedef struct EvStruct{ + short x[4], *atp; + } *Event, EventRec; + /* set up the (hardwired) columns */ + FunColumnSelect( fun, sizeof(EventRec), NULL, + "2i", "2I ", "w", FUN_OFFSET(Event, x), + "2i2", "2I[2]", "w", FUN_OFFSET(Event, x), + "at2p", "@2I", "w", FUN_OFFSET(Event, atp), + "at2p4", "@2I[4]", "w", FUN_OFFSET(Event, atp), + "atp9", "@I[9]", "w", FUN_OFFSET(Event, atp), + "atb20", "@I[B20]", "w", FUN_OFFSET(Event, atb), + NULL); + +Here we have specified the following columns: + + +=over 4 + + + + +=item * + +2i: two short ints in an array which is stored as part the +record + + +=item * + +2i2: the 3rd and 4th elements of an array which is stored +as part of the record + + +=item * + +an array of at least 10 elements, not stored in the record but +allocated elsewhere, and used by three different columns: + + +=over 4 + + + + +=item * + +at2p: 2 short ints which are the first 2 elements of the allocated array + + +=item * + +at2p4: 2 short ints which are the 5th and 6th elements of +the allocated array + + +=item * + +atp9: a short int which is the 10th element of the allocated array + + +=back + + + + +=item * + +atb20: a short int which is at byte offset 20 of another allocated array + + +=back + + +In this way, several columns can be specified, all of which are in a +single array. B<NB>: it is the programmer's responsibility to +ensure that specification of a positive value for poff does not point +past the end of valid data. + + + +=item * + +B<read/write mode>: "r" means that the column is read from an +input file into user space by +FunTableRowGet(), "w" means that +the column is written to an output file. Both can specified at the same +time. + + + +=item * + +B<offset>: the offset into the user data to store +this column. Typically, the macro FUN_OFFSET(recname, colname) is used +to define the offset into a record structure. + + +=back + + + + +When all column arguments have been specified, a final NULL argument +must added to signal the column selection list. + + +As an alternative to the varargs +FunColumnSelect() +routine, a non-varargs routine called +FunColumnSelectArr() +also is available. The first three arguments (fun, size, plist) of this +routine are the same as in +FunColumnSelect(). +Instead of a variable +argument list, however, +FunColumnSelectArr() +takes 5 additional arguments. The first 4 arrays arguments contain the +names, types, modes, and offsets, respectively, of the columns being +selected. The final argument is the number of columns that are +contained in these arrays. It is the user's responsibility to free +string space allocated in these arrays. + + +Consider the following example: + + typedef struct evstruct{ + int status; + float pi, pha, *phas; + double energy; + } *Ev, EvRec; + + FunColumnSelect(fun, sizeof(EvRec), NULL, + "status", "J", "r", FUN_OFFSET(Ev, status), + "pi", "E", "r", FUN_OFFSET(Ev, pi), + "pha", "E", "r", FUN_OFFSET(Ev, pha), + "phas", "@9E", "r", FUN_OFFSET(Ev, phas), + NULL); + + +Each time a row is read into the Ev struct, the "status" column is +converted to an int data type (regardless of its data type in the +file) and stored in the status value of the struct. Similarly, "pi" +and "pha", and the phas vector are all stored as floats. Note that the +"@" sign indicates that the "phas" vector is a pointer to a 9 element +array, rather than an array allocated in the struct itself. The row +record can then be processed as required: + + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ev->pi = (ev->pi+.5); + ev->pha = (ev->pi-.5); + } + + + +FunColumnSelect() +can also be called to define "writable" columns in order to generate a FITS +Binary Table, without reference to any input columns. For +example, the following will generate a 4-column FITS binary table when +FunTableRowPut() is used to +write Ev records: + + + typedef struct evstruct{ + int status; + float pi, pha + double energy; + } *Ev, EvRec; + + FunColumnSelect(fun, sizeof(EvRec), NULL, + "status", "J", "w", FUN_OFFSET(Ev, status), + "pi", "E", "w", FUN_OFFSET(Ev, pi), + "pha", "E", "w", FUN_OFFSET(Ev, pha), + "energy", "D", "w", FUN_OFFSET(Ev, energy), + NULL); + +All columns are declared to be write-only, so presumably the column +data is being generated or read from some other source. + + +In addition, +FunColumnSelect() +can be called to define B<both> "readable" and "writable" columns. +In this case, the "read" columns +are associated with an input file, while the "write" columns are +associated with the output file. Of course, columns can be specified as both +"readable" and "writable", in which case they are read from input +and (possibly modified data values are) written to the output. +The +FunColumnSelect() +call itself is made by passing the input Funtools handle, and it is +assumed that the output file has been opened using this input handle +as its +Funtools reference handle. + + +Consider the following example: + + typedef struct evstruct{ + int status; + float pi, pha, *phas; + double energy; + } *Ev, EvRec; + + FunColumnSelect(fun, sizeof(EvRec), NULL, + "status", "J", "r", FUN_OFFSET(Ev, status), + "pi", "E", "rw", FUN_OFFSET(Ev, pi), + "pha", "E", "rw", FUN_OFFSET(Ev, pha), + "phas", "@9E", "rw", FUN_OFFSET(Ev, phas), + "energy", "D", "w", FUN_OFFSET(Ev, energy), + NULL); + +As in the "read" example above, each time an row is read into the Ev +struct, the "status" column is converted to an int data type +(regardless of its data type in the file) and stored in the status +value of the struct. Similarly, "pi" and "pha", and the phas vector +are all stored as floats. Since the "pi", "pha", and "phas" variables +are declared as "writable" as well as "readable", they also will be +written to the output file. Note, however, that the "status" variable +is declared as "readable" only, and hence it will not be written to +an output file. Finally, the "energy" column is declared as +"writable" only, meaning it will not be read from the input file. In +this case, it can be assumed that "energy" will be calculated in the +program before being output along with the other values. + + +In these simple cases, only the columns specified as "writable" will +be output using +FunTableRowPut(). However, +it often is the case that you want to merge the user columns back in +with the input columns, even in cases where not all of the input +column names are explicitly read or even known. For this important +case, the B<merge=[type]> keyword is provided in the plist string. + + +The B<merge=[type]> keyword tells Funtools to merge the columns from +the input file with user columns on output. It is normally used when +an input and output file are opened and the input file provides the +Funtools reference handle +for the output file. In this case, each time +FunTableRowGet() is called, the +raw input rows are saved in a special buffer. If +FunTableRowPut() then is called +(before another call to +FunTableRowGet()), the contents +of the raw input rows are merged with the user rows according to the +value of B<type> as follows: + + + +=over 4 + + + + +=item * + +B<update>: add new user columns, and update value of existing ones (maintaining the input data type) + + + +=item * + +B<replace>: add new user columns, and replace the data type +and value of existing ones. (Note that if tlmin/tlmax values are not +specified in the replacing column, but are specified in the original +column being replaced, then the original tlmin/tlmax values are used +in the replacing column.) + + + +=item * + +B<append>: only add new columns, do not "replace" or "update" existing ones + + +=back + + + + +Consider the example above. If B<merge=update> is specified in the +plist string, then "energy" will be added to the input columns, and +the values of "pi", "pha", and "phas" will be taken from the user +space (i.e., the values will be updated from the original values, if +they were changed by the program). The data type for "pi", "pha", and +"phas" will be the same as in the original file. If +B<merge=replace> is specified, both the data type and value of +these three input columns will be changed to the data type and value +in the user structure. If B<merge=append> is specified, none of +these three columns will be updated, and only the "energy" column will +be added. Note that in all cases, "status" will be written from the +input data, not from the user record, since it was specified as read-only. + + +Standard applications will call +FunColumnSelect() +to define user columns. However, if this routine is not called, the +default behavior is to transfer all input columns into user space. For +this purpose a default record structure is defined such that each data +element is properly aligned on a valid data type boundary. This +mechanism is used by programs such as fundisp and funtable to process +columns without needing to know the specific names of those columns. +It is not anticipated that users will need such capabilities (contact +us if you do!) + + +By default, FunColumnSelect() +reads/writes rows to/from an "array of structs", where each struct contains +the column values for a single row of the table. This means that the +returned values for a given column are not contiguous. You can +set up the IO to return a "struct of arrays" so that each of the +returned columns are contiguous by specifying B<org=structofarrays> +(abbreviation: B<org=soa>) in the plist. +(The default case is B<org=arrayofstructs> or B<org=aos>.) + + +For example, the default setup to retrieve rows from a table would be +to define a record structure for a single event and then call + FunColumnSelect() +as follows: + + typedef struct evstruct{ + short region; + double x, y; + int pi, pha; + double time; + } *Ev, EvRec; + + got = FunColumnSelect(fun, sizeof(EvRec), NULL, + "x", "D:10:10", mode, FUN_OFFSET(Ev, x), + "y", "D:10:10", mode, FUN_OFFSET(Ev, y), + "pi", "J", mode, FUN_OFFSET(Ev, pi), + "pha", "J", mode, FUN_OFFSET(Ev, pha), + "time", "1D", mode, FUN_OFFSET(Ev, time), + NULL); + +Subsequently, each call to +FunTableRowGet() +will return an array of structs, one for each returned row. If instead you +wanted to read columns into contiguous arrays, you specify B<org=soa>: + + typedef struct aevstruct{ + short region[MAXROW]; + double x[MAXROW], y[MAXROW]; + int pi[MAXROW], pha[MAXROW]; + double time[MAXROW]; + } *AEv, AEvRec; + + got = FunColumnSelect(fun, sizeof(AEvRec), "org=soa", + "x", "D:10:10", mode, FUN_OFFSET(AEv, x), + "y", "D:10:10", mode, FUN_OFFSET(AEv, y), + "pi", "J", mode, FUN_OFFSET(AEv, pi), + "pha", "J", mode, FUN_OFFSET(AEv, pha), + "time", "1D", mode, FUN_OFFSET(AEv, time), + NULL); + +Note that the only modification to the call is in the plist string. + + +Of course, instead of using staticly allocated arrays, you also can specify +dynamically allocated pointers: + + /* pointers to arrays of columns (used in struct of arrays) */ + typedef struct pevstruct{ + short *region; + double *x, *y; + int *pi, *pha; + double *time; + } *PEv, PEvRec; + + got = FunColumnSelect(fun, sizeof(PEvRec), "org=structofarrays", + "$region", "@I", mode, FUN_OFFSET(PEv, region), + "x", "@D:10:10", mode, FUN_OFFSET(PEv, x), + "y", "@D:10:10", mode, FUN_OFFSET(PEv, y), + "pi", "@J", mode, FUN_OFFSET(PEv, pi), + "pha", "@J", mode, FUN_OFFSET(PEv, pha), + "time", "@1D", mode, FUN_OFFSET(PEv, time), + NULL); + +Here, the actual storage space is either allocated by the user or by the +FunColumnSelect() call). + + +In all of the above cases, the same call is made to retrieve rows, e.g.: + + buf = (void *)FunTableRowGet(fun, NULL, MAXROW, NULL, &got); + +However, the individual data elements are accessed differently. +For the default case of an "array of structs", the +individual row records are accessed using: + + for(i=0; i<got; i++){ + ev = (Ev)buf+i; + fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + ev->x, ev->y, ev->pi, ev->pha, ev->dx, ev->dy, ev->time); + } + +For a struct of arrays or a struct of array pointers, we have a single struct +through which we access individual columns and rows using: + + aev = (AEv)buf; + for(i=0; i<got; i++){ + fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n", + aev->x[i], aev->y[i], aev->pi[i], aev->pha[i], + aev->dx[i], aev->dy[i], aev->time[i]); + } + +Support for struct of arrays in the +FunTableRowPut() +call is handled analogously. + + +See the evread example code +and +evmerge example code +for working examples of how +FunColumnSelect() is used. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funcombine.pod b/funtools/doc/pod/funcombine.pod new file mode 100644 index 0000000..946d304 --- /dev/null +++ b/funtools/doc/pod/funcombine.pod @@ -0,0 +1,157 @@ +=pod + +=head1 NAME + + + +B<FunCombine: Combining Region and Table Filters> + + + +=head1 SYNOPSIS + + + + + +This document discusses the conventions for combining region and table +filters, especially with regards to the comma operator. + + + + +=head1 DESCRIPTION + + + +B<Comma Conventions> + +Filter specifications consist of a series of boolean expressions, +separated by commas. These expressions can be table filters, +spatial region filters, or combinations thereof. Unfortunately, +common usage requires that the comma operator must act differently +in different situations. Therefore, while its use is intuitive in +most cases, commas can be a source of confusion. + + +According to long-standing usage in IRAF, when a comma separates two +table filters, it takes on the meaning of a boolean B<and>. Thus: + + foo.fits[pha==1,pi==2] + +is equivalent to: + + foo.fits[pha==1 && pi==2] + + +When a comma separates two spatial region filters, however, it has +traditionally taken on the meaning of a boolean B<or>. Thus: + + foo.fits[circle(10,10,3),ellipse(20,20,8,5)] + +is equivalent to: + + foo.fits[circle(10,10,3) || ellipse(20,20,8,5)] + +(except that in the former case, each region is given a unique id +in programs such as funcnts). + + +Region and table filters can be combined: + + foo.fits[circle(10,10,3),pi=1:5] + +or even: + + foo.fits[pha==1&&circle(10,10,3),pi==2&&ellipse(20,20,8,5)] + +In these cases, it is not obvious whether the command should utilize an +B<or> or B<and> operator. We therefore arbitrarily chose to +implement the following rule: + + +=over 4 + + + + +=item * + +if both expressions contain a region, the operator used is B<or>. + + +=item * + +if one (or both) expression(s) does not contain a region, the operator +used is B<and>. + + +=back + + +This rule handles the cases of pure regions and pure column filters properly. +It unambiguously assigns the boolean B<and> to all mixed cases. Thus: + + foo.fits[circle(10,10,3),pi=1:5] + +and + + foo.fits[pi=1:5,circle(10,10,3)] + +both are equivalent to: + + foo.fits[circle(10,10,3) && pi=1:5] + + + +[NB: This arbitrary rule B<replaces the previous arbitrary rule> +(pre-funtools 1.2.3) which stated: + + +=over 4 + + + + +=item * + +if the 2nd expression contains a region, the operator used is B<or>. + + +=item * + +if the 2nd expression does not contain a region, the operator +used is B<and>. + + +=back + + +In that scenario, the B<or> operator was implied by: + + pha==4,circle 5 5 1 + +while the B<and> operator was implied by + + circle 5 5 1,pha==4 + +Experience showed that this non-commutative treatment of the comma +operator was confusing and led to unexpected results.] + + +The comma rule must be considered provisional: comments and complaints +are welcome to help clarify the matter. Better still, we recommend +that the comma operator be avoided in such cases in favor of an +explicit boolean operator. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funcone.pod b/funtools/doc/pod/funcone.pod new file mode 100644 index 0000000..a766ca6 --- /dev/null +++ b/funtools/doc/pod/funcone.pod @@ -0,0 +1,191 @@ +=pod + +=head1 NAME + + + +B<funcone - cone search of a binary table containing RA, Dec columns> + + + +=head1 SYNOPSIS + + + + + +funcone <switches> <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns] + + + + + +=head1 OPTIONS + + + + + + -d deccol:[hdr] # Dec column name, units (def: DEC:d) + -j # join columns from list file + -J # join columns from list file, output all rows + -l listfile # read centers and radii from a list + -L listfile # read centers and radii from a list, output list rows + -n # don't use cone limits as a filter + -r racol:[hdr] # RA column name, units (def: RA:h) + -x # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols + -X # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols, output all rows + + + + +=head1 DESCRIPTION + + + + +Funcone performs a cone search on the RA and Dec columns of a FITS +binary table. The distance from the center RA, Dec position to the RA, +Dec in each row in the table is calculated. Rows whose distance is +less than the specified radius are output. + + +The first argument to the program specifies the FITS file, raw event +file, or raw array file. If "stdin" is specified, data are read from +the standard input. Use Funtools Bracket +Notation to specify FITS extensions, and filters. The second +argument is the output FITS file. If "stdout" is specified, the FITS +binary table is written to the standard output. + + +The third and fourth required arguments are the RA and Dec center +position. By default, RA is specified in hours while Dec is specified +in degrees. You can change the units of either of these by appending +the character "d" (degrees), "h" (hours) or "r" (radians). Sexagesimal +notation is supported, with colons or spaces separating hms and dms. +(When using spaces, please ensure that the entire string is quoted.) + + +The fifth required argument is the radius of the cone search. By default, +the radius value is given in degrees. The units can be changed by appending +the character "d" (degrees), "r" (radians), "'" (arc minutes) or +'"' (arc seconds). + + +By default, all +columns of the input file are copied to the output file. Selected +columns can be output using an optional sixth argument in the form: + + "column1 column1 ... columnN" + +A seventh argument allows you to output selected columns from the list +file when B<-j> switch is used. Note that the RA and Dec columns +used in the cone calculation must not be de-selected. + + +Also by default, the RA and Dec column names are named "RA" and "Dec", +and are given in units of hours and degrees respectively. You can +change both the name and the units using the -r [RA] and/or -d [Dec] +switches. Once again, one of "h", "d", or "r" is appended to the +column name to specify units but in this case, there must be a colon ":" +between the name and the unit specification. + + +If the B<-l [listfile]> switch is used, then one or more of the +center RA, center Dec, and radius can be taken from a list file (which +can be a FITS table or an ASCII column text file). In this case, the +third (center RA), fourth (center Dec), and fifth (radius) command +line arguments can either be a column name in the list file (if that +parameter varies) or else a numeric value (if that parameter is +static). When a column name is specified for the RA, Dec, or radius, +you can append a colon followed by "h", "d", or "r" to specify units +(also ' and " for radius). The cone search algorithm is run once for +each row in the list, taking RA, Dec, and radius values from the +specified columns or from static numeric values specified on the +command line. + + +When using a list, all valid rows from each iteration are written to a +single output file. Use the B<-x> switch to help delineate which +line of the list file was used to produce the given output row(s). +This switch causes the values for the center RA, Dec, radius, and row +number to be appended to the output file, in columns called RA_CEN, +DEC_CEN, RAD_CEN and CONE_KEY, respectively. Alternatively, the +B<-j> (join) switch will append all columns from the list row to +the output row (essentially a join of the list row and input row), +along with the CONE_KEY row number. These two switches are mutually +exclusive. + + +The B<-X> and B<-J> switches write out the same data as their +lower case counterparts for each row satisfying a cone search. In +addition, these switches also write out rows from the event file that +do not satisfy any cone search. In such cases, that CONE_KEY column +will be given a value of -1 and the center and list position information +will be set to zero for the given row. Thus, all rows of the input +event file are guaranteed to be output, with rows satisfying at least +one cone search having additional search information. + + +The B<-L> switch acts similarly to the B<-l> switch in that it +takes centers from a list file. However, it also implicitly sets the +-j switch, so that output rows are the join of the input event row and +the center position row. In addition, this switch also writes out all +center position rows for which no event satisfies the cone search +criteria of that row. The CONE_KEY column will be given a value of -2 +for center rows that were not close to any data row and the event +columns will be zeroed out for such rows. In this way, all centers +rows are guaranteed to be output at least once. + + +If any of "all row" switches (B<-X>, B<-J>, or B<-L>) are +specified, then a new column named JSTAT is added to the output table. +The positive values in this column indicate the center position row number +(starting from 1) in the list file that this data row successful matched +in a cone search. A value of -1 means that the data row did not match +any center position. A value of -2 means that the center position was +not matched by any data row. + + +Given a center position and radius, the cone search algorithm +calculates limit parameters for a box enclosing the specified cone, +and only tests rows whose positions values lie within those limits. +For small files, the overhead associated with this cone limit +filtering can cause the program to run more slowly than if all events +were tested. You can turn off cone limit filtering using the B<-n> +switch to see if this speeds up the processing (especially useful when +processing a large list of positions). + + +For example, the default cone search uses columns "RA" and "Dec" in hours +and degrees (respectively) and RA position in hours, Dec and radius in degrees: + + funone in.fits out.fits 23.45 34.56 0.01 + +To specify the RA position in degrees: + + funcone in.fits out.fits 23.45d 34.56 0.01 + +To get RA and Dec from a list but use a static value for radius (and +also write identifying info for each row in the list): + + funcone -x -l list.txt in.fits out.fits MYRA MYDec 0.01 + +User specified columns in degrees, RA position in hours (sexagesimal +notation), Dec position in degrees (sexagesimal notation) and radius +in arc minutes: + + funcone -r myRa:d -d myDec in.fits out.fits 12:30:15.5 30:12 15' + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/fundisp.pod b/funtools/doc/pod/fundisp.pod new file mode 100644 index 0000000..8f36d86 --- /dev/null +++ b/funtools/doc/pod/fundisp.pod @@ -0,0 +1,484 @@ +=pod + +=head1 NAME + + + +B<fundisp - display data in a Funtools data file> + + + +=head1 SYNOPSIS + + + + + +fundisp [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n] + + + + + +=head1 OPTIONS + + + + + + -f # format string for display + -l # display image as a list containing the columns X, Y, VAL + -n # don't output header + -F [c] # use specified character as column separator (def: space) + -T # output in rdb/starbase format (tab separators) + + + + +=head1 DESCRIPTION + + + + +B<fundisp> displays the data in the specified +FITS Extension +and/or +Image Section +of a FITS file, or in a +Section +of a non-FITS array or raw event file. + +The first argument to the program specifies the FITS input image, array, or +raw event file to display. If "stdin" is specified, data are read from +the standard input. Use Funtools Bracket +Notation to specify FITS extensions, image sections, and filters. + + +If the data being displayed are columns (either in a FITS binary table +or a raw event file), the individual rows are listed. Filters can be +added using bracket notation. Thus: + + [sh] fundisp "test.ev[time-(int)time>.15]" + X Y PHA PI TIME DX DY + ------- ------- ------- --------- ---------------- ---------- ---------- + 10 8 10 8 17.1600 8.50 10.50 + 9 9 9 9 17.1600 9.50 9.50 + 10 9 10 9 18.1600 9.50 10.50 + 10 9 10 9 18.1700 9.50 10.50 + 8 10 8 10 17.1600 10.50 8.50 + 9 10 9 10 18.1600 10.50 9.50 + 9 10 9 10 18.1700 10.50 9.50 + 10 10 10 10 19.1600 10.50 10.50 + 10 10 10 10 19.1700 10.50 10.50 + 10 10 10 10 19.1800 10.50 10.50 + +[NB: The FITS binary table test file test.ev, as well as the FITS +image test.fits, are contained in the funtools funtest directory.] + + +When a table is being displayed using B<fundisp>, a second optional +argument can be used to specify the columns to display. For example: + + [sh] fundisp "test.ev[time-(int)time>=.99]" "x y time" + X Y TIME + -------- -------- --------------------- + 5 -6 40.99000000 + 4 -5 59.99000000 + -1 0 154.99000000 + -2 1 168.99000000 + -3 2 183.99000000 + -4 3 199.99000000 + -5 4 216.99000000 + -6 5 234.99000000 + -7 6 253.99000000 + + + +The special column B<$REGION> can be specified to display the +region id of each row: + + [sh $] fundisp "test.ev[time-(int)time>=.99&&annulus(0 0 0 10 n=3)]" 'x y time $REGION' + X Y TIME REGION + -------- -------- --------------------- ---------- + 5 -6 40.99000000 3 + 4 -5 59.99000000 2 + -1 0 154.99000000 1 + -2 1 168.99000000 1 + -3 2 183.99000000 2 + -4 3 199.99000000 2 + -5 4 216.99000000 2 + -6 5 234.99000000 3 + -7 6 253.99000000 3 + + +Here only rows with the proper fractional time and whose position also is +within one of the three annuli are displayed. + +Columns can be excluded from display using a minus sign before the +column: + + [sh $] fundisp "test.ev[time-(int)time>=.99]" "-time" + X Y PHA PI DX DY + -------- -------- -------- ---------- ----------- ----------- + 5 -6 5 -6 5.50 -6.50 + 4 -5 4 -5 4.50 -5.50 + -1 0 -1 0 -1.50 0.50 + -2 1 -2 1 -2.50 1.50 + -3 2 -3 2 -3.50 2.50 + -4 3 -4 3 -4.50 3.50 + -5 4 -5 4 -5.50 4.50 + -6 5 -6 5 -6.50 5.50 + -7 6 -7 6 -7.50 6.50 + +All columns except the time column are displayed. + +The special column B<$N> can be specified to display the +ordinal value of each row. Thus, continuing the previous example: + + fundisp "test.ev[time-(int)time>=.99]" '-time $n' + X Y PHA PI DX DY N + ------- -------- -------- ---------- ----------- ----------- ---------- + 5 -6 5 -6 5.50 -6.50 337 + 4 -5 4 -5 4.50 -5.50 356 + -1 0 -1 0 -1.50 0.50 451 + -2 1 -2 1 -2.50 1.50 465 + -3 2 -3 2 -3.50 2.50 480 + -4 3 -4 3 -4.50 3.50 496 + -5 4 -5 4 -5.50 4.50 513 + -6 5 -6 5 -6.50 5.50 531 + -7 6 -7 6 -7.50 6.50 550 + +Note that the column specification is enclosed in single quotes to protect +'$n' from begin expanded by the shell. + + +In general, the rules for activating and de-activating columns are: + + +=over 4 + + + + +=item * + +If only exclude columns are specified, then all columns but +the exclude columns will be activated. + + +=item * + +If only include columns are specified, then only the specified columns +are activated. + + +=item * + +If a mixture of include and exclude columns are specified, then +all but the exclude columns will be active; this last case +is ambiguous and the rule is arbitrary. + + +=back + + +In addition to specifying columns names explicitly, the special +symbols B<+> and B<-> can be used to activate and +de-activate B<all> columns. This is useful if you want to +activate the $REGION column along with all other columns. According +to the rules, the syntax "$REGION" only activates the region column +and de-activates the rest. Use "+ $REGION" to activate all +columns as well as the region column. + + +If the data being displayed are image data (either in a FITS primary +image, a FITS image extension, or an array file), an mxn pixel display +is produced, where m and n are the dimensions of the image. By +default, pixel values are displayed using the same data type as in the +file. However, for integer data where the BSCALE and BZERO header parameters +are present, the data is displayed as floats. In either case, the +display data type can be overridden using an optional second argument +of the form: + + bitpix=n + +where n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, +respectively. + + +Of course, running B<fundisp> on anything but the smallest image +usually results in a display whose size makes it unreadable. +Therefore, one can uses bracket notation (see below) +to apply section and/or blocking to the image before generating a +display. For example: + + [sh] fundisp "test.fits[2:6,2:7]" bitpix=-32 + 2 3 4 5 6 + ---------- ---------- ---------- ---------- ---------- + 2: 3.00 4.00 5.00 6.00 7.00 + 3: 4.00 5.00 6.00 7.00 8.00 + 4: 5.00 6.00 7.00 8.00 9.00 + 5: 6.00 7.00 8.00 9.00 10.00 + 6: 7.00 8.00 9.00 10.00 11.00 + 7: 8.00 9.00 10.00 11.00 12.00 + + + +Note that is is possible to display a FITS binary table as an image +simply by passing the table through B<funimage> first: + + [sh] ./funimage test.ev stdout | fundisp "stdin[2:6,2:7]" bitpix=8 + 2 3 4 5 6 + ------- ------- ------- ------- ------- + 2: 3 4 5 6 7 + 3: 4 5 6 7 8 + 4: 5 6 7 8 9 + 5: 6 7 8 9 10 + 6: 7 8 9 10 11 + 7: 8 9 10 11 12 + + +If the B<-l> (list) switch is used, then an image is displayed as a +list containing the columns: X, Y, VAL. For example: + + fundisp -l "test1.fits[2:6,2:7]" bitpix=-32 + X Y VAL + ---------- ---------- ----------- + 2 2 6.00 + 3 2 1.00 + 4 2 1.00 + 5 2 1.00 + 6 2 1.00 + 2 3 1.00 + 3 3 5.00 + 4 3 1.00 + 5 3 1.00 + 6 3 1.00 + 2 4 1.00 + 3 4 1.00 + 4 4 4.00 + 5 4 1.00 + 6 4 1.00 + 2 5 1.00 + 3 5 1.00 + 4 5 1.00 + 5 5 3.00 + 6 5 1.00 + 2 6 1.00 + 3 6 1.00 + 4 6 1.00 + 5 6 1.00 + 6 6 2.00 + 2 7 1.00 + 3 7 1.00 + 4 7 1.00 + 5 7 1.00 + 6 7 1.00 + + + +If the B<-n> (nohead) switch is used, then no header is output for +tables. This is useful, for example, when fundisp output is being +directed into gnuplot. + + +The B<fundisp> program uses a default set of display formats: + + datatype TFORM format + -------- ----- -------- + double D "%21.8f" + float E "%11.2f" + int J "%10d" + short I "%8d" + byte B "%6d" + string A "%12.12s" + bits X "%8x" + logical L "%1x" + +Thus, the default display of 1 double and 2 shorts gives: + + [sh] fundisp snr.ev "time x y" + + TIME X Y + --------------------- -------- -------- + 79494546.56818075 546 201 + 79488769.94469175 548 201 + ... + +You can change the display format for individual columns or for all +columns of a given data types by means of the -f switch. The format +string that accompanies -f is a space-delimited list of keyword=format +values. The keyword values can either be column names (in which case +the associated format pertains only to that column) or FITS table +TFORM specifiers (in which case the format pertains to all columns +having that data type). For example, you can change the double and +short formats for all columns like this: + + [sh] fundisp -f "D=%22.11f I=%3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- --- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ... + + + +Alternatively, you can change the format of the time and x columns like this: + + [sh] fundisp -f "time=%22.11f x=%3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- -------- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ... + +Note that there is a potential conflict if a column has the same name +as one of the TFORM specifiers. In the examples above, the the "X" +column in the table has the same name as the X (bit) datatype. To +resolve this conflict, the format string is processed such that +TFORM datatype specifiers are checked for first, using a +case-sensitive comparison. If the specified format value is not an +upper case TFORM value, then a case-insensitive check is made on the +column name. This means that, in the examples above, "X=%3d" will refer +to the X (bit) datatype, while "x=%3d" will refer to the X column: + + [sh] fundisp -f "X=%3d" snr.ev "x y" + + X Y + -------- -------- + 546 201 + 548 201 + ... + + [sh] fundisp -f "x=%3d" snr.ev "x y" + + X Y + --- -------- + 546 201 + 548 201 + ... + +As a rule, therefore, it is best always to specify the column name in +lower case and TFORM data types in upper case. + + +The B<-f [format]> will change the format for a single execution +of fundisp. You also can use the B<FUN_FORMAT> envronment variable +to change the format for all invocations of fundisp. The format of this +environment variable's value is identical to that used with +the B<-f> switch. This global value can be overridden in +individual cases by use of the B<-f [format]> switch. + + +Caveats: Please also note that it is the user's responsibility to +match the format specifier to the column data type correctly. Also +note that, in order to maintain visual alignment between names and +columns, the column name will be truncated (on the left) if the +format width is less than the length of the name. However, truncation +is not performed if the output is in RDB format (using the -T switch). + + +[An older-style format string is supported but deprecated. It +consists of space-delimited C format statements for all data types, +specified in the following order: + + double float int short byte string bit. + +This order of the list is based on the assumption that people generally +will want to change the float formats. + +If "-" is entered instead of a format statement for a given data type, the +default format is used. Also, the format string can be terminated without +specifying all formats, and defaults will be used for the rest of the +list. Note that you must supply a minimum field width, i.e., "%6d" and +"%-6d" are legal, "%d" is not legal. + +By using -f [format], you can change the double and short formats like this: + + [sh] fundisp -f "22.11f - - 3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- --- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ... + +NB: This format is deprecated and will be removed in a future release.] + + +The B<-F[c]> switch can be used to specify a (single-character) +column separator (where the default is a space). Note that column +formatting will almost certainly also add spaces to pad individual +columns to the required width. These can be removed with a program +such as sed, at the cost of generating unaligned columns. For example: + +fundisp -F',' snr.ev'[cir 512 512 .1]' + X, Y, PHA, PI, TIME, DX, DY +--------,--------,--------,--------,---------------------,--------,-------- + 512, 512, 6, 7, 79493997.45854475, 578, 574 + 512, 512, 8, 9, 79494575.58943175, 579, 573 + 512, 512, 5, 6, 79493631.03866175, 578, 575 + 512, 512, 5, 5, 79493290.86521725, 578, 575 + 512, 512, 8, 9, 79493432.00990875, 579, 573 + +fundisp -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g' + X,Y,PHA,PI,TIME,DX,DY +--------,--------,--------,--------,---------------------,--------,-------- + 512,512,6,7,79493997.45854475,578,574 + 512,512,8,9,79494575.58943175,579,573 + 512,512,5,6,79493631.03866175,578,575 + 512,512,5,5,79493290.86521725,578,575 + 512,512,8,9,79493432.00990875,579,573 + +fundisp -f "x=%3d y=%3d pi=%1d pha=%1d time=%20.11f dx=%3d dy=%3d" -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g' + X,Y,A,I,TIME,DX,DY +---,---,-,-,--------------------,---,--- +512,512,6,7,79493997.45854474604,578,574 +512,512,8,9,79494575.58943174779,579,573 +512,512,5,6,79493631.03866174817,578,575 +512,512,5,5,79493290.86521725357,578,575 +512,512,8,9,79493432.00990875065,579,573 + + + + +If the B<-T> (rdb table) switch is used, the output will conform +to starbase/rdb data base format: tabs will be inserted between +columns rather than spaces. This format is not available when +displaying image pixels (except in conjunction with the B<-l> +switch). + + +Finally, note that B<fundisp> can be used to create column filters from +the auxiliary tables in a FITS file. For example, the following shell code +will generate a good-time interval (GTI) filter for X-ray data files that +contain a standard GTI extension: + + #!/bin/sh + sed '1,/---- .*/d + /^$/,$d' | awk 'tot>0{printf "||"};{printf "time="$1":"$2; tot++}' + +If this script is placed in a file called "mkgti", it can be used in a +command such as: + + fundisp foo.fits"[GTI]" | mkgti > gti.filter + +The resulting filter file can then be used in various funtools programs: + + funcnts foo.fits"[@gti.filter]" ... + +to process only the events in the good-time intervals. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funds9.pod b/funtools/doc/pod/funds9.pod new file mode 100644 index 0000000..e1b5cc2 --- /dev/null +++ b/funtools/doc/pod/funds9.pod @@ -0,0 +1,113 @@ +=pod + +=head1 NAME + + + +B<FunDS9: Funtools and DS9 Image Display> + + + +=head1 SYNOPSIS + + + + +Describes how funtools can be integrated into the ds9 Analysis menu. + + + +=head1 DESCRIPTION + + + + + + +SAOImage/DS9 is an astronomical imaging and data visualization +application used by astronomers around the world. DS9 can display +standard astronomical FITS images and binary tables, but also has +support for displaying raw array files, shared memory files, and data +files automatically retrieved via FTP and HTTP. Standard functional +capabilities include multiple frame buffers, colormap and region +manipulation, and many data scaling algorithms. DS9's advanced +features include TrueColor visuals, deep frame buffers, true +PostScript printing, and display of image mosaics. The program's +support of image tiling, "blinking", arbitrary zoom, rotation, and pan +is unparalleled in astronomy. It also has innovative support for +automatic retrieval and display of standard image data such as the +Digital Sky Survey (using servers at SAO, StScI, or ESO). + + +DS9 can communicate with external programs such as Funtools using the +XPA +messaging system. In addition, programs can be integrated directly +into the DS9 GUI by means of a configurable Analysis menu. By +default, the DS9 Analysis menu contains algorithms deemed essential to +the core functions of DS9, e.g., display cross-cuts of data, +iso-intensity contours, and WCS grids. However, new programs can be +added to DS9 by creating a set-up file which can be loaded into DS9 +to reconfigure the Analysis menu. + + +The basic format of the analysis set-up file is: + + # + # Analysis command descriptions: + # menu label/description + # file templates for this command + # "menu" (add to menu) |"bind" (bind to key) + # analysis command line + + +For example, the funcnts program can be specified in this way: + + Funcnts (counts in source/bkgd regions; options: none) + * + menu + funcnts $filename $regions(source,,) $regions(background,,) | $text + +As shown above, DS9 supports a macro facility to provide information +as well as task support to command lines. For example, the $regions +macro is expanded by DS9 to provide the current source and/or +background region to the analysis command. The $text macro is expanded +to generate a text window display. It also is possible to query for +parameters using a $param macro, plot data using a $plot macro, +etc. See the DS9 documentation for further details. + + +A set-up file called funtools.ds9 will +load some useful Funtools applications (counts in regions, radial +profile, X-ray light curve and energy spectrum, 1D histogram) into the DS9 +Analysis menu (version 2.1 and above). The file resides in the bin +directory where Funtools programs are installed. It can be manually +loaded into DS9 from the B<Load Analysis Commands ...> option of +the B<Analysis> menu. Alternatively, you can tell DS9 to load +this file automatically at start-up time by adding the pathname to the +B<Edit>->B<Preferences>->B<Analysis Menu>->Analysis +File menu option. (NB: make sure you select +B<Edit>->B<Preferences>->B<Save Preferences> after setting +the pathname.) + + +The tasks in this setup file generally process the original disk-based +FITS file. Funcnts-based results (radial profile, counts in regions) +are presented in WCS units, if present in the FITS header. For +situations where a disk file is not available (e.g., image data +generated and sent to DS9's 'fits' XPA access point), versions of the +radial profile and counts in regions tasks also are also offered +utilizing DS9's internal image data. Results are presented in pixels. +Aside from the units, the results should be identical to the file-based +results. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funenv.pod b/funtools/doc/pod/funenv.pod new file mode 100644 index 0000000..0717a17 --- /dev/null +++ b/funtools/doc/pod/funenv.pod @@ -0,0 +1,349 @@ +=pod + +=head1 NAME + + + +B<FunEnv: Funtools Environment Variables> + + + +=head1 SYNOPSIS + + + + +Describes the environment variables which can be used to tailor the overall +Funtools environment. + + + +=head1 DESCRIPTION + + + + + +The following environment variables are supported by Funtools: + + +=over 4 + + + + + +=item * + +B<FITS_EXTNAME> + + +The B<FITS_EXTNAME> environment variable specifies the +default FITS extension name when FunOpen() is called on a file lacking +a primary image. Thus, + + setenv FITS_EXTNAME "NEWEV" + +will allow you to call FunOpen() on files without specifying NEWEV in +the +Funtools bracket specification. +If no FITS_EXTNAME variable is defined and the extension name also is +not passed in the bracket specification, then the default will be to +look for standard X-ray event table extension names "EVENTS" or +"STDEVT" (we are, after all, and X-ray astronomy group at heart!). + + + + +=item * + +B<FITS_EXTNUM> + + +The B<FITS_EXTNUM> environment variable specifies the +default FITS extension number when FunOpen() is called on a file lacking +a primary image. Thus, + + setenv FITS_EXTNUM 7 + +will allow you to call FunOpen() on files to open the seventh +extension without specifying the number in the +Funtools bracket specification. + + + + +=item * + +B<FITS_BINCOLS> and B<EVENTS_BINCOLS> + + +These environment variable specifies the default binning key for +FITS binary tables and raw event files, respectively. They can be +over-ridden using the B<bincols=[naxis1,naxis2]> keyword in a +Funtools bracket specification. +The value of each environment variable +is a pair of comma-delimited columns, enclosed in parentheses, to use +for binning. For example, if you want to bin on detx and dety by +default, then use: + + setenv FITS_BINCOLS "(detx,dety)" + +in preference to adding a bincols specification to each filename: + + foo.fits[bincols=(detx,dety)] + + + + + +=item * + +B<FITS_BITPIX> and B<EVENTS_BITPIX> + + +These environment variable specifies the default bitpix value for +binning FITS binary tables and raw event files, respectively. They can +be over-ridden using the B<bitpix=[value]> keyword in a +Funtools bracket specification. The value +of each environment variable is one of the standard FITS bitpix values +(8,16,32,-32,-64). For example, if you want binning routines to +create a floating array, then use: + + setenv FITS_BITPIX -32 + +in preference to adding a bitpix specification to each filename: + + foo.fits[bitpix=-32] + + + + + +=item * + +B<ARRAY> + + +The B<ARRAY> environment variable specifies the default +definition of an array file for Funtools. +It is used if there is no array specification passed in the +B<ARRAY()> directive in a +Non-FITS Array specification. +The value of the environment variable is a valid array specification such as: + + setenv ARRAY "s100.150" + foo.arr[ARRAY()] + +This can be defined in preference to adding the specification to each filename: + + foo.arr[ARRAY(s100.150)] + + + + + +=item * + +B<EVENTS> + + +The B<EVENTS> environment variable specifies the default +definition of an raw event file for Funtools. +It is used if there is no EVENTS specification passed in the +B<EVENTS()> directive in a +Non-FITS EVENTS specification. +The value of the environment variable is a valid EVENTS specification such as: + + setenv EVENTS "x:J:1024,y:J:1024,pi:I,pha:I,time:D,dx:E:1024,dx:E:1024" + foo.ev[EVENTS()] + +This can be defined in preference to adding the specification to each filename: + + foo.ev[EVENTS(x:J:1024,y:J:1024,pi:I,pha:I,time:D,dx:E:1024,dx:E:1024)] + + + +=back + + + +The following filter-related environment variables are supported by Funtools: + + +=over 4 + + + + + + +=item * + +B<FILTER_PTYPE> + + +The B<FILTER_PTYPE> environment variable specifies how to +build a filter. There are three possible methods: + + +=over 4 + + + + +=item * + +process or p + + +The filter is compiled and linked against the funtools library (which +must therefore be accessible in the original install directory) to produce +a slave program. This program is fed events or image data and returns +filter results. + + + +=item * + +dynamic or d (gcc only) + + +The filter is compiled and linked against the funtools library (which +must therefore be accessible in the original install directory) to produce +a dynamic shared object, which is loaded into the funtools program and +executed as a subroutine. (Extensive testing has shown that, contrary to +expectations, this method is no faster than using a slave process.) + + + +=item * + +contained or c + + +The filter and all supporting region code is compiled and linked +without reference to the funtools library to produce a slave program +(which is fed events or image data and returns filter results). This method +is slower than the other two, because of the time it takes to compile the +region filtering code. It is used by stand-alone programs such as ds9, +which do not have access to the funtools library. + + +=back + + + +By default, B<dynamic> is generally used for gcc compilers and +B<process> for other compilers. However the filter building algorithm +will check for required external files and will use B<contained> is +these are missing. + + + + +=item * + +B<FUN_MAXROW> + + +The B<FUN_MAXROW> environment variable is used by core +row-processing Funtools programs (funtable, fundisp, funcnts, funhist, +funmerge, and funcalc) to set the maximum number of rows read at once +(i.e. it sets the third argument to the FunTableRowGet() call). The +default is 8192. Note that this variable is a convention only: it will +not be a part of a non-core Funtools program unless code is explicitly +added, since each call to FunTableRowGet() specifies its own maximum +number of rows to read. NB: if you make this value very large, you +probably will need to increase B<FUN_MAXBUFSIZE> (see below) as well. + + + + +=item * + +B<FUN_MAXBUFSIZE> + + +The B<FUN_MAXBUFSIZE> environment variable is used to limit the +max buffer size that will be allocated to hold table row data. This +buffer size is calculated to be the row size of the table multiplied +by the maximum number of rows read at once (see above). Since the +row size is unlimited (and we have examples of it being larger than 5 +Mb), it is possible that the total buffer size will exceed the machine +capabilities. We therefore set a default value of 5Mb for the max buffer +size, and adjust maxrow so that the total size calculated is less than +this max buffer size. (If the row size is greater than this max buffer +size, then maxrow is set to 1.) This environment variable will change +the max buffer size allowed. + + + + +=item * + +B<FILTER_CC> + + +The B<FILTER_CC> environment variable specifies the compiler to +use for compiling a filter specification. You also can use the B<CC> +environment variable. If neither has been set, then gcc will be used +if available. Otherwise cc is used if available. + + + + +=item * + +B<FILTER_EXTRA> + + +The B<FILTER_EXTRA> environment variable specifies extra options +to add to a filter compile command line. In principle, you can add libraries, +include files, and compiler switches. This variable should be used with care. + + + + +=item * + +B<FILTER_TMPDIR> + + +The B<FILTER_TMPDIR> environment variable specifies the temporary +directory for filter compilation intermediate files. You also can use +the B<TMPDIR> and B<TMP> variables. By default, /tmp is used +as the temporary directory. + + + + +=item * + +B<FILTER_KEEP> + + +The B<FILTER_KEEP> environment variable specifies whether the +intermediate filter files (i.e. C source file and compile log file) +should be saved after a filter is built. The default is "false", so that +these intermediate files are deleted. This variable is useful for debugging, +but care should be taken to reset its value to false when debugging is +complete. + + + +=back + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funfiles.pod b/funtools/doc/pod/funfiles.pod new file mode 100644 index 0000000..7efb464 --- /dev/null +++ b/funtools/doc/pod/funfiles.pod @@ -0,0 +1,1019 @@ +=pod + +=head1 NAME + + + +B<FunFiles: Funtools Data Files> + + + +=head1 SYNOPSIS + + + + +This document describes the data file formats (FITS, array, raw +events) as well as the file types (gzip, socket, etc.) supported +by Funtools. + + + +=head1 DESCRIPTION + + + + + +Funtools supports FITS images and binary tables, and binary files +containing array (homogeneous) data or event (heterogeneous) data. +IRAF-style brackets are appended to the filename to specify various +kinds of information needed to characterize these data: + + file[ext|ind|ARRAY()|EVENTS(),section][filters] + or + file[ext|ind|ARRAY()|EVENTS(),section,filters] + +where: + + +=over 4 + + + + +=item * + +B<file> is the Funtools file name + + +=item * + +B<ext> is the FITS extension name + + +=item * + +B<ind> is the FITS extension number + + +=item * + +B<ARRAY()> is an array specification + + +=item * + +B<EVENTS()> is an event specification + + +=item * + +B<section> is the image section specification + + +=item * + +B<filters> are spatial region and table (row) filters + + +=back + + + +B<Supported Data Formats> + +Funtools programs (and the underlying libraries) support the +following data file formats: + + +=over 4 + + + + +=item * + +FITS images (and image extensions) + + +=item * + +FITS binary tables + + +=item * + +binary files containing an array of homogeneous data + + +=item * + +binary files containing events, i.e. records of heterogeneous data + + +=item * + +column-based text files, which are documented here + + +=item * + +non-disk files and lists of files + + +=back + + +Information needed to identify and characterize +the event or image data can be specified on the command line +using IRAF-style bracket notation appended to the filename: + + foo.fits # open FITS default extension + image.fits[3] # open FITS extension #3 + events.fits[EVENTS] # open EVENTS extension + array.file[ARRAY(s1024)] # open 1024x1024 short array + events.file[EVENTS(x:1024,y:1024...)] # open non-FITS event list + +Note that in many Unix shells (e.g., csh and tcsh), filenames must +be enclosed in quotes to protect the brackets from shell processing. + +B<FITS Images and Binary Tables> + +When FunOpen() opens a FITS file +without a bracket specifier, the default behavior is to look for a +valid image in the primary HDU. In the absence of a primary image, +Funtools will try to open an extension named either B<EVENTS> or +B<STDEVT>, if one of these exists. This default behavior supports +both FITS image processing and standard X-ray event list processing +(which, after all, is what we at SAO/HEAD do). + + +In order to open a FITS binary table or image extension explicitly, it +is necessary to specify either the extension name or the extension +number in brackets: + + foo.fits[1] # open extension #1: the primary HDU + foo.fits[3] # open extension #3 of a FITS file + foo.fits[GTI] # open GTI extension of a FITS file + +The ext argument specifies the name of the FITS extension (i.e. the +value of the EXTENSION header parameter in a FITS extension), while +the index specifies the value of the FITS EXTVER header parameter. +Following FITS conventions, extension numbers start at 1. + + +When a FITS data file is opened for reading using +FunOpen(), the specified extension +is automatically located and is used to initialize the Funtools internal +data structures. + +B<Non-FITS Raw Event Files> + +In addition to FITS tables, Funtools programs and libraries can operate +on non-FITS files containing heterogeneous event records. To specify +such an event file, use: + + + +=over 4 + + + + +=item * + +file[EVENTS(event-spec)] + + +=item * + +file[EVENTS()] + + +=back + + +where B<event-spec> is a string that specified the names, data +types, and optional image dimensions for each element of the event +record: + + +=over 4 + + + + +=item * + +[name]:[n][type]:[(lodim:)hidim] + + +=back + + + + +Data types follow standard conventions for FITS binary tables, but include +two extra unsigned types ('U' and 'V'): + + +=over 4 + + + + +=item * + +B<B> -- unsigned 8-bit char + + +=item * + +B<I> -- signed 16-bit int + + +=item * + +B<J> -- signed 32-bit int + + +=item * + +B<K> -- signed 64-bit int + + +=item * + +B<E> -- 32-bit float + + +=item * + +B<D> -- 64-bit float + + +=item * + +B<U> -- unsigned 16-bit int + + +=item * + +B<V> -- unsigned 32-bit int + + +=back + + +An optional integer value B<n> can be prefixed to the type to indicate +that the element is an array of n values. For example: + + foo.fits[EVENTS(x:I,y:I,status:4J)] + +defines x and y as 16-bit ints and status as an array of 4 32-bit ints. + + +Furthermore, image dimensions can be attached to the event specification +in order to tell Funtools how to bin the events into an image. They +follow the conventions for the FITS TLMIN/TLMAX keywords. If the low +image dimension is not specified, it defaults to 1. Thus: + + + +=over 4 + + + + +=item * + +RAWX:J:1:100 + + +=item * + +RAWX:J:100 + + +=back + + +both specify that the dimension of this column runs from 1 to 100. + + +NB: it is required that all padding be specified in the record +definition. Thus, when writing out whole C structs instead of +individual record elements, great care must be taken to include +the compiler-added padding in the event definition. + + +For example, suppose a FITS binary table has the following set of column +definitions: + + TTYPE1 = 'X ' / Label for field + TFORM1 = '1I ' / Data type for field + TLMIN1 = 1 / Min. axis value + TLMAX1 = 10 / Max. axis value + TTYPE2 = 'Y ' / Label for field + TFORM2 = '1I ' / Data type for field + TLMIN2 = 2 / Min. axis value + TLMAX2 = 11 / Max. axis value + TTYPE3 = 'PHA ' / Label for field + TFORM3 = '1I ' / Data type for field + TTYPE4 = 'PI ' / Label for field + TFORM4 = '1J ' / Data type for field + TTYPE5 = 'TIME ' / Label for field + TFORM5 = '1D ' / Data type for field + TTYPE6 = 'DX ' / Label for field + TFORM6 = '1E ' / Data type for field + TLMIN6 = 1 / Min. axis value + TLMAX6 = 10 / Max. axis value + TTYPE7 = 'DY ' / Label for field + TFORM7 = '1E ' / Data type for field + TLMIN7 = 3 / Min. axis value + TLMAX7 = 12 / Max. axis value + + +An raw event file containing these same data would have the event +specification: + + EVENTS(X:I:10,Y:I:2:11,PHA:I,PI:J,TIME:D,DX:E:10,DY:E:3:12) + + + +If no event specification string is included within the EVENTS() operator, +then the event specification is taken from the B<EVENTS> environment +variable: + + setenv EVENTS "X:I:10,Y:I:10,PHA:I,PI:J,TIME:D,DX:E:10,DY:E:10" + + + +In addition to knowing the data structure, it is necessary to know the +I<endian> ordering of the data, i.e., whether or not the data is +in I<bigendian> format, so that we can convert to the native +format for this platform. This issue does not arise for FITS Binary +Tables because all FITS files use big-endian ordering, regardless of +platform. But for non-FITS data, big-endian data produced on a Sun +workstation but read on a Linux PC needs to be byte-swapped, since PCs +use little-endian ordering. To specify an ordering, use the +I<bigendian=> or I<endian=> keywords on the command-line +or the EVENTS_BIGENDIAN or EVENTS_ENDIAN environment variables. The +value of the I<bigendian> variables should be "true" or "false", +while the value of the I<endian> variables should be "little" or +"big". + + +For example, a PC can access data produced by a Sun using: + + hrc.nepr[EVENTS(),bigendian=true] +or + hrc.nepr[EVENTS(),endian=big] +or + setenv EVENTS_BIGENDIAN true +or + setenv EVENTS_ENDIAN big + +If none of these are specified, the data are assumed to follow the +format for that platform and no byte-swapping is performed. + +B<Non-FITS Array Files> + +In addition to FITS images, Funtools programs and libraries can operate +on non-FITS files containing arrays of homogeneous data. To specify +an array file, use: + + +=over 4 + + + + +=item * + +file[ARRAY(array-spec)] + + +=item * + +file[ARRAY()] + + +=back + + + +where array-spec is of the form: + + +=over 4 + + + + +=item * + +[type][dim1][.dim2][:skip][endian] + + +=back + + + +and where [type] is: + + +=over 4 + + + + +=item * + +b (8-bit unsigned char) + + +=item * + +s (16-bit short int) + + +=item * + +u (16-bit unsigned short int) + + +=item * + +i (32-bit int) + + +=item * + +r,f (32-bit float) + + +=item * + +d (64-bit float) + + +=back + + + + +The dim1 specification is required, but dim2 is optional and defaults +to dim1. The skip specification is optional and defaults to 0. The +optional endian specification can be 'l' or 'b' and defaults to the +endian type for the current machine. + + +If no array specification is included within the ARRAY() operator, +then the array specification is taken from the B<ARRAY> environment +variable. For example: + + + foo.arr[ARRAY(r512)] # bitpix=-32 dim1=512 dim2=512 + foo.arr[ARRAY(r512.400)] # bitpix=-32 dim1=512 dim2=400 + foo.arr[ARRAY(r512.400]) # bitpix=-32 dim1=512 dim2=400 + foo.arr[ARRAY(r512.400:2880)] # bitpix=-32 dim1=512 dim2=400 skip=2880 + foo.arr[ARRAY(r512l)] # bitpix=-32 dim1=512 dim2=512 endian=little + setenv ARRAY "r512.400:2880" + foo.arr[ARRAY()] # bitpix=-32 dim1=512 dim2=400 skip=2880 + + +B<Specifying Image Sections> + +Once a data file (and possibly, a FITS extension) has been specified, +the next (optional) part of a bracket specification can be used to +select image B<section> information, i.e., to specify the x,y +limits of an image section, as well as the blocking factor to apply to +that section. This information can be added to any file specification but +only is used by Funtools image processing routines. + + +The format of the image section specification is one of the following: + + +=over 4 + + + + +=item * + +file[xy0:xy1,block] + + +=item * + +file[x0:x1,y0:y1,block] + + +=item * + +file[x0:x1,*,block] + + +=item * + +file[*,y0:y1,block] + + +=item * + +file[*,block] + + +=back + + +where the limit values can be ints or "*" for default. A single "*" +can be used instead of val:val, as shown. Note that blocking is +applied to the section after it is extracted. + + +In addition to image sections specified by the lo and hi x,y limits, image +sections using center positions can be specified: + + +=over 4 + + + + +=item * + +file[dim1@xcen,dim2@ycen] + + +=item * + +file[xdim2@xcen@ycen] + + +=item * + +file[dim1@xcen,dim2@ycen,block] + + +=item * + +file[dim@xcen@ycen,block] + + +=back + + +Note that the (float) values for dim, dim1, dim2, xcen, ycen must be +specified or else the expression does not make sense! + + +In all cases, block is optional and defaults to 1. An 's' or 'a' can +be appended to signify "sum" or "average" blocking (default is "sum"). +Section specifications are given in image coordinates by default. If you +wish to specify physical coordinates, add a 'p' as the last character +of the section specification, before the closing bracket. +For example: + + + +=over 4 + + + + +=item * + +file[-8:-7,-8:-7p] + + +=item * + +file[-8:-7,-8:-7,2p] + + +=back + + + +A section can be specified in any Funtools file name. If the operation +to be applied to that file is an imaging operation, then the +specification will be utilized. If the operation is purely a table +operation, then the section specification is ignored. + + +Do not be confused by: + + foo.fits[2] + foo.fits[*,2] + +The former specifies opening the second extension of the FITS file. +The latter specifies application of block 2 to the image section. + + +Note that the section specification must come after +any of FITS B<ext> name or B<ind> number, +but all sensible defaults are supported: + + +=over 4 + + + + +=item * + +file[ext] + + +=item * + +file[ext,index] + + +=item * + +file[index] + + +=item * + +file[ext,section] + + +=item * + +file[ext,index,section] + + +=item * + +file[index,section] + + +=item * + +file[section] + + +=back + + + +B<Binning FITS Binary Tables and Non-FITS Event Files> + +If a FITS binary table or a non-FITS raw event file is to be binned +into a 2D image (e.g., using the +funimage +program), it is necessary to specify the two columns to be used for the +binning, as well as the dimensions of the image. Funtools first looks +for a specifier of the form: + + bincols=([xnam[:tlmin[:tlmax:[binsiz]]]],[ynam[:tlmin[:tlmax[:binsiz]]]]) + +in bracket syntax, and uses the column names thus specified. The tlmin, tlmax, +and binsiz specifiers determine the image binning dimensions using: + + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) + +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters are present in the FITS binary +table header, respectively. If only one parameter is specified, it is +assumed to be tlmax, and tlmin defaults to 1. If two parameters are +specified, they are assumed to be tlmin and tlmax. + +For example, to bin an HRC event list columns "VPOS" and "UPOS", use: + + hrc.nepr[bincols=(VPOS,UPOS)] + +or + + hrc.nepr[bincols=(VPOS:49152,UPOS:4096)] + +Note that you can optionally specify the dimensions of these columns +to cover cases where neither TLMAX keywords are defined in +the header. If either dimension is specified, then both must be specified. + + +You can set the FITS_BINCOLS or EVENTS_BINCOLS environment variable as +an alternative to adding the "bincols=" specifier to each file name +for FITS binary tables and raw event files, respectively. If no +binning keywords or environment variables are specified, or if the +specified columns are not in the binary table, the Chandra parameters +CPREF (or PREFX) are searched for in the FITS binary table header. +Failing this, columns named "X" and "Y" are sought. If these are not +found, the code looks for columns containing the characters "X" and +"Y". Thus, you can bin on "DETX" and "DETX" columns without +specifying them, if these are the only column names containing the "X" +and "Y" characters. + + +Ordinarily, each event or row contributes one count to an image pixel +during the 2D binning process. Thus, if five events all have the same +(x,y) position, the image pixel value for that position will have a +value of five. It is possible to specify a variable contribution +for each event by using the vcol=[colname] filter spec: + + vcol=[colname] + +The vcol colname is a column containing a numeric value in each event row +that will be used as the contribution of the given event to its image +pixel. For example, consider an event file that has the following content: + + x:e:4 y:e:4 v:e + ------ ------ ---- + 1 1 1.0 + 2 2 2.0 + 3 3 3.0 + 4 4 0.0 + 1 1 1.0 + 2 2 2.0 + 3 3 3.0 + 4 4 4.0 + +There are two events with x,y value of (1,1) so ordinarily a 2D image will +have a value of 2 in the (1,1) pixel. If the v column is specified as the +value column: + + foo.fits'[vcol=v]' + +then each pixel will contain the additive sum of the associated (x,y) +column values from the v column. For example, image pixel (1,1) will +contain 1. + 1. = 2, image pixel (2,2) will contain (2 + 2) = 4, etc. + + +An important variation on the use of a value column to specify the +contribution an event makes to an image pixel is when the value column +contains the reciprocal of the event contribution. For this case, the +column name should be prefixed with a / (divide sign) thus: + + foo.fits'[vcol=/v]' + +Each image pixel value will then be the sum of the reciprocals of the value +column. A zero in the value column results in NaN (not a number). +Thus, in the above example, image pixel (1.1) will contain 1/1 + 1/1 = 2, +image pixel (2,2) will contain (1/2 + 1/2) = 1, etc. Image pixel (4,4) +will contain (1/0 + 1/4) = NaN. + + +You can set the FITS_VCOL or EVENTS_VCOL environment variable as +an alternative to adding the "vcol=" specifier to each file name +for FITS binary tables and raw event files, respectively. + + +Finally, when binning events, the data type of the resulting 2D image +must be specified. This can be done with the "bitpix=[n]" keyword in +the bracket specification. For example: + + events.fits[bincols=(VPOS,UPOS),bitpix=-32] + +will create a floating point image binned on columns VPOS and UPOS. +If no bitpix keyword is specified, bitpix=32 is assumed. As with +bincols values, you also can use the FITS_BITPIX and EVENTS_BITPIX +environment variables to set this value for FITS binary tables and +raw event files, respectively. + + +The B<funimage> program also allows you to create a 1D image projection +along any column of a table by using the B<bincols=[column]> +filter specification and specifying a single column. +For example, the following command projects a 1D image along +the chipx column of a table: + + funimage ev.fits'[bincols=chipx]' im.fits + +See funimage for more +information about creating 1D and 2D images. + + +Finally, please note that Funtools supports most FITS standards. +We will add missing support as required by the community. In general, +however, we do not support non-standard extensions. For example, we +sense the presence of the binary table 'variable length array' +proposed extension and we pass it along when copying and filtering +files, but we do not process it. We will add support for new standards +as they become official. + +B<Table and Spatial Region Filters> + +Note that, in addition extensions and image sections, Funtools bracket +notation can be used to specify table and spatial region filters. These +filters are always placed after the image section information. They +can be specified in the same bracket or in a separate bracket +immediately following: + + +=over 4 + + + + +=item * + +file[ext|ind|ARRAY()|EVENTS(),section][filters] + + +=item * + +file[ext|ind|ARRAY()|EVENTS(),section,filters] + + +=back + + +where: + + +=over 4 + + + + +=item * + +B<file> is the Funtools file name + + +=item * + +B<ARRAY()> is an array specification + + +=item * + +B<EVENTS()> is an event list specification + + +=item * + +B<ext> is the FITS extension name + + +=item * + +B<ind> is the FITS extension number + + +=item * + +B<section> is the image section to extract + + +=item * + +B<filters> are spatial region and table (row) filters to apply + + +=back + + + +The topics of table and region filtering are covered in detail in: + + +=over 4 + + + + +=item * + +Table Filtering + + +=item * + +Spatial Region Filtering + + +=back + + + +B<Disk Files and Other Supported File Types> + +The specified B<file> usually is an ordinary disk file. In +addition, gzip'ed files are supported in Funtools: gzip'ed input files +are automatically uncompressed as they are read, and gzip'ed output +files are compressed as they are written. NB: if a FITS binary table +is written in gzip format, the number of rows in the table will be set +to -1. Such a file will work with Funtools programs but will not work +with other FITS programs such as ds9. + + +The special keywords "stdin" and "stdout" designate Unix standard +input and standard output, respectively. The string "-" (hyphen) will +be taken to mean "stdin" if the file is opened for reading and +"stdout" if the file is opened for writing. + + +A file also can be an INET socket on the same or another machine using +the syntax: + + machine:port + +Thus, for example: + + karapet:1428 + +specifies that I/O should be performed to/from port 1428 on the +machine karapet. If no machine name is specified, the default is to +use the current machine: + + :1428 + +This means to open port 1428 on the current machine. Socket support +allows you to generate a distributed pipe: + + on karapet: funtask1 in.fits bynars:1428 + on bynars: funtask2 :1428 out.fits + +The socket mechanism thus supports simple parallel processing using +B<process decomposition>. Note that parallel processing using +B<data decomposition> is supported via the B<section> specifier (see +below), and the B<row#> specifier, which is part of +Table Filtering. + + +A file also can be a pointer to shared memory using the syntax: + + shm:[id|@key][:size] + +A shared memory segment is specified with a B<shm:> prefix, +followed by either the shared memory id or the shared memory key +(where the latter is prefixed by the '@' character). The size (in +bytes) of the shared memory segment can then be appended (preceded by +the ':' character). If the size specification is absent, the code will +attempt to determine the length automatically. + +If the open mode contains the string "w+", then the memory segment will be +created if it does not exist. (It also will be released and deleted when the +file is closed.) In the case where a memory segment is being created, the +length of the segment is required. + + +A file also can be Unix piped command (i.e. a program to run) using the syntax: + + "pipe: command arg1 ... argn" + +The output from the command must be a valid FITS file. It is important +to use quotes to protect spaces so that command arguments are passed +correctly. A silly example is: + + fundisp "pipe: funtable 'foo.fits[cir 512 512 .1]' stdout" + +This seemed like a good idea at the time ... + +B<Lists of Files> + + +Funtools also will process a list of files as a single file using the +syntax: + + "list: file1 file2 ... filen" + +The files in the list are separated by whitespace. Any of the +above file types can be used. For example, if two files, foo1.fits and +foo2.fits, are part of the same observation, they can be processed as +a single file (using their own filters): + + fundisp "list: foo1.fits[cir(512,512,10)] foo2.fits[cir(511,511,10)]" + X Y PHA PI TIME DX DY + -------- -------- -------- -------- --------------------- -------- -------- + 512 512 6 7 79493997.45854475 578 574 + 512 512 8 9 79494575.58943175 579 573 + 512 512 5 6 79493631.03866175 578 575 + 512 512 5 5 79493290.86521725 578 575 + 512 512 8 9 79493432.00990875 579 573 + 511 511 5 5 79488631.09462625 580 575 + 511 511 10 11 79488780.60006675 580 573 + 511 511 4 4 79494562.35474326 580 575 + 511 511 6 6 79488203.01561825 580 575 + 511 511 6 6 79488017.99730176 580 575 + 511 511 4 4 79494332.45355175 580 575 + 511 511 9 10 79492685.94014275 581 574 + 511 511 5 5 79487708.71298325 580 575 + 511 511 8 9 79493719.00160225 581 573 + +Again, note that it is important to avoid spaces in the filters +because the list separator also is whitespace. To protect whitespace +in a filter, enclose the file specification in quotes: + + fundisp "list: 'foo1.fits[cir 512 512 .1]' foo2.fits[cir(511,511,.1)]" + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funfilters.pod b/funtools/doc/pod/funfilters.pod new file mode 100644 index 0000000..b9b6d83 --- /dev/null +++ b/funtools/doc/pod/funfilters.pod @@ -0,0 +1,521 @@ +=pod + +=head1 NAME + + + +B<Funfilters: Filtering Rows in a Table> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the user interface for +filtering rows in binary tables. + + + +=head1 DESCRIPTION + + + + + +Table filtering allows a program to select rows from an table (e.g., +X-ray event list) by checking each row against one or more expressions +involving the columns in the table. When a table is filtered, only +valid rows satisfying these expressions are passed through for processing. + + +A filter expression is specified using bracket notation appended to +the filename of the data being processed: + + foo.fits[pha==1&&pi==2] + +It is also possible to put region specification inside a file and +then pass the filename in bracket notation: + + foo.fits[@my.reg] + +Filters must be placed after the extension and image section +information, when such information is present. The correct order is: + + +=over 4 + + + + +=item * + +file[fileinfo,sectioninfo][filters] + + +=item * + +file[fileinfo,sectioninfo,filters] + + +=back + + +where: + + +=over 4 + + + + +=item * + +B<file> is the Funtools file name + + +=item * + +B<fileinfo> is an ARRAY, EVENT, FITS extension, or FITS index + + +=item * + +B<sectioninfo> is the image section to extract + + +=item * + +B<filters> are spatial region and table (row) filters to apply + + +=back + + +See Funtools Files for more information +on file and image section specifications. + +B<Filter Expressions> + + +Table filtering can be performed on columns of data in a FITS +binary table or a raw event file. Table filtering is accomplished by +means of B<table filter specifications>. An table filter +specification consists of one or more B<filter expressions> Filter +specifications also can contain comments and local/global processing +directives. + + +More specifically, a filter specification consist of one or more lines +containing: + + # comment until end of line + # include the following file in the table descriptor + @file + # each row expression can contain filters separated by operators + [filter_expression] BOOLOP [filter_expression2], ... + # each row expression can contain filters separated by the comma operator + [filter_expression1], [filter_expression2], ... + # the special row# keyword allows a range of rows to be processed + row#=m:n + # or a single row + row#=m + # regions are supported -- but are described elsewhere + [spatial_region_expression] + + + +A single filter expression consists of an arithmetic, logical, or +other operations involving one or more column values from a +table. Columns can be compared to other columns, to header values, +or to numeric constants. Standard math functions can be applied to +columns. Separate filter expressions can be combined using boolean operators. +Standard C semantics can be used when constructing expressions, with +the usual precedence and associativity rules holding sway: + + Operator Associativity + -------- ------------- + () left to right + !! (logical not) right to left + ! (bitwise not) - (unary minus) right to left + * / left to right + + - left to right + < <= > >= left to right + == != left to right + & (bitwise and) left to right + ^ (bitwise exclusive or) left to right + | (bitwise inclusive or) left to right + && (logical and) left to right + || (logical or) left to right + = right to left + +For example, if energy and pha are columns in a table, +then the following are valid expressions: + + pha>1 + energy == pha + (pha>1) && (energy<=2) + max(pha,energy)>=2.5 + + + +Comparison values can be integers or floats. Integer comparison values can be +specified in decimal, octal (using '0' as prefix), hex (using '0x' as prefix) +or binary (using '0b' as prefix). Thus, the following all specify the same +comparison test of a status mask: + + (status & 15) == 8 # decimal + (status & 017) == 010 # octal + (status & 0xf) == 0x8 # hex + (status & 0b1111) == 0b1000 # binary + + +The special keyword row# allows you to process a range of rows. +When row# is specified, the filter code skips to the designated +row and only processes the specified number of rows. The +"*" character can be utilized as the high limit value to denote +processing of the remaining rows. Thus: + + row#=100:109 + +processes 10 rows, starting with row 100 (counting from 1), +while: + + row#=100:* + +specifies that all but the first 99 rows are to be processed. + + +Spatial region filtering allows a program to select regions of an +image or rows of a table (e.g., X-ray events) using simple geometric +shapes and boolean combinations of shapes. For a complete description +of regions, see Spatial Region Filtering. + +B<Separators Also Are Operators> + +As mentioned previously, multiple filter expressions can be specified +in a filter descriptor, separated by commas or new-lines. +When such a comma or new-line separator is used, the boolean AND operator +is automatically generated in its place. Thus and expression such as: + + pha==1,pi=2:4 + +is equivalent to: + + (pha==1) && (pi>=2&&pi<=4) + + +[Note that the behavior of separators is different for filter expressions +and spatial region expressions. The former uses AND as the operator, while +the latter user OR. See +Combining Region and Table Filters +for more information about these conventions and how they are treated +when combined.] + +B<Range Lists> + +Aside from the standard C syntax, filter expressions can make use of +IRAF-style B<range lists> which specify a range of values. The +syntax requires that the column name be followed by an '=' sign, which +is followed by one or more comma-delimited range expressions of the form: + + col = vv # col == vv in range + col = :vv # col <= vv in range + col = vv: # col >= vv in range + col = vv1:vv2 # vv1 <= col <= vv2 in range + +The vv's above must be numeric constants; the right hand side of a +range list cannot contain a column name or header value. + +Note that, unlike an ordinary comma separator, the comma separator used +between two or more range expressions denotes OR. Thus, when two or +more range expressions are combined with a comma separator, the resulting +expression is a shortcut for more complicated boolean logic. For example: + + col = :3,6:8,10: + +is equivalent to: + + (col=6 && col =10) + +Note also that the single-valued rangelist: + + col = val + +is equivalent to the C-based filter expression: + + col == val + +assuming, of course, that val is a numeric constant. + +B<Math Operations and Functions> + +It is permissible to specify C math functions as part of the filter syntax. +When the filter parser recognizes a function call, it automatically +includes the math.h and links in the C math library. Thus, it is +possible to filter rows by expressions such as these: + + +=over 4 + + + + +=item * + +(pi+pha)>(2+log(pi)-pha) + + +=item * + +min(pi,pha)*14>x + + +=item * + +max(pi,pha)==(pi+1) + + +=item * + +feq(pi,pha) + + +=item * + +div(pi,pha)>0 + + +=back + + +The function feq(a,b) returns true (1) if the difference between a and b +(taken as double precision values) is less than approximately 10E-15. +The function div(a,b) divides a by b, but returns NaN (not a number) +if b is 0. It is a safe way to avoid floating point errors when +dividing one column by another. + +B<Include Files> + +The special B<@filename> directive specifies an include file +containing filter expressions. This file is processed as part of +the overall filter descriptor: + + foo.fits[pha==1,@foo] + + +B<Header Parameters> + +The filter syntax supports comparison between a column value and a +header parameter value of a FITS binary tables (raw event files have no +such header). The header parameters can be taken from the binary +table header or the primary header. For example, assuming there is a +header value MEAN_PHA in one of these headers, you can select photons +having exactly this value using: + + + +=over 4 + + + + +=item * + +pha==MEAN_PHA + + +=back + + + + + +Table filtering is more easily described by means of examples. +Consider data containing the following table structure: + + +=over 4 + + + + +=item * + +double TIME + + +=item * + +int X + + +=item * + +int Y + + +=item * + +short PI + + +=item * + +short PHA + + +=item * + +int DX + + +=item * + +int DY + + +=back + + + + +Tables can be filtered on these columns using IRAF/QPOE range syntax or +any valid C syntax. The following examples illustrate the possibilities: + + +=over 4 + + + + + + +=item * + +pha=10 + + +=item * + +pha==10 + + +select rows whose pha value is exactly 10 + + + + +=item * + +pha=10:50 + + +select rows whose pha value is in the range of 10 to 50 + + + + +=item * + +pha=10:50,100 + + +select rows whose pha value is in the range of 10 to 50 or is +equal to 100 + + + + +=item * + +pha>=10 && pha<=50 + + +select rows whose pha value is in the range of 10 to 50 + + + + +=item * + +pi=1,2&&pha>3 + + +select rows whose pha value is 1 or 2 and whose pi value is 3 + + + + +=item * + +pi=1,2 || pha>3 + + +select rows whose pha value is 1 or 2 or whose pi value is 3 + + + + +=item * + +pha==pi+1 + + +select rows whose pha value is 1 less than the pi value + + + + +=item * + +(pha==pi+1) && (time>50000.0) + + +select rows whose pha value is 1 less than the pi value +and whose time value is greater than 50000 + + + + +=item * + +(pi+pha)>20 + + +select rows in which the sum of the pi and pha values is greater +than 20 + + + + +=item * + +pi%2==1 + + +select rows in which the pi value is odd + + +=back + + + + +Currently, integer range list limits cannot be specified in binary +notation (use decimal, hex, or octal instead). Please contact us if +this is a problem. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funflush.pod b/funtools/doc/pod/funflush.pod new file mode 100644 index 0000000..b525ad2 --- /dev/null +++ b/funtools/doc/pod/funflush.pod @@ -0,0 +1,107 @@ +=pod + +=head1 NAME + + + +B<FunFlush - flush data to output file> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void FunFlush(Fun fun, char *plist) + + + + + +=head1 DESCRIPTION + + + + +The B<FunFlush> routine will flush data to a FITS output file. In +particular, it can be called after all rows have been written (using +the FunTableRowPut() routine) +in order to add the null padding that is required to complete a FITS +block. It also should be called after completely writing an image using +FunImagePut() or after writing +the final row of an image using +FunTableRowPut(). + + +The B<plist> (i.e., parameter list) argument is a string +containing one or more comma-delimited B<keyword=value> +parameters. If the plist string contains the parameter +"copy=remainder" and the file was opened with a reference file, which, +in turn, was opened for extension copying (i.e. the input +FunOpen() mode also was "c" or "C"), +then FunFlush also will copy the remainder of the FITS extensions from +the input reference file to the output file. This normally would be +done only at the end of processing. + + +Note that FunFlush() is called +with "copy=remainder" in the mode string by FunClose(). This means +that if you close the output file before the reference input file, it +is not necessary to call +FunFlush() explicitly, unless you +are writing more than one extension. See the +evmerge example code. However, it is safe to +call FunFlush() more than once +without fear of re-writing either the padding or the copied +extensions. + + +In addition, if FunFlush() is +called on an output file with the plist set to "copy=reference" and if +the file was opened with a reference file, the reference extension is +written to the output file. This mechanism provides a simple way to +copy input extensions to an output file without processing the former. +For example, in the code fragment below, an input extension is set to +be the reference file for a newly opened output extension. If that +reference extension is not a binary table, it is written to the output +file: + + /* process each input extension in turn */ + for(ext=0; ;ext++){ + /* get new extension name */ + sprintf(tbuf, "%s[%d]", argv[1], ext); + /* open input extension -- if we cannot open it, we are done */ + if( !(ifun=FunOpen(tbuf, "r", NULL)) ) + break; + /* make the new extension the reference handle for the output file */ + FunInfoPut(ofun, FUN_IFUN, &ifun, 0); + /* if its not a binary table, just write it out */ + if( !(s=FunParamGets(ifun, "XTENSION", 0, NULL, &got)) || + strcmp(s, "BINTABLE")){ + if( s ) free(s); + FunFlush(ofun, "copy=reference"); + FunClose(ifun); + continue; + } + else{ + /* process binary table */ + .... + } + } + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funhead.pod b/funtools/doc/pod/funhead.pod new file mode 100644 index 0000000..ee224cc --- /dev/null +++ b/funtools/doc/pod/funhead.pod @@ -0,0 +1,218 @@ +=pod + +=head1 NAME + + + +B<funhead - display a header in a Funtools file> + + + +=head1 SYNOPSIS + + + + + +funhead [-a] [-s] [-t] [-L] <iname> [oname ename] + + + + + +=head1 OPTIONS + + + + + + -a # display all extension headers + -s # display 79 chars instead of 80 before the new-line + -t # prepend data type char to each line of output + -L # output in rdb/starbase list format + + + + +=head1 DESCRIPTION + + + + +B<funhead> displays the FITS header parameters in the specified +FITS Extension. + +The first argument to the program specifies the Funtools input file +to display. If "stdin" is specified, data are read from +the standard input. Funtools Bracket +Notation is used to specify particular FITS extension to process. +Normally, the full 80 characters of each header card is output, +followed by a new-line. + + +If the B<-a> switch is specified, the header from each FITS +extensions in the file is displayed. Note, however, that the B<-a> +switch does not work with FITS files input via stdin. We hope to +remove this restriction in a future release. + + +If the B<-s> switch is specified, only 79 characters are output +before the new-line. This helps the display on 80 character terminals. + + +If the B<-t> switch is specified, the data type of the parameter +is output as a one character prefix, followed by 77 characters of the +param. The parameter data types are defined as: FUN_PAR_UNKNOWN +('u'), FUN_PAR_COMMENT ('c'), FUN_PAR_LOGICAL ('l'), FUN_PAR_INTEGER +('i'), FUN_PAR_STRING ('s'), FUN_PAR_REAL ('r'), FUN_PAR_COMPLEX ('x'). + + +If the B<-L> (rdb table) switch is used, the output will conform +to starbase/rdb data base list format. + + +For example to display the EVENTS extension (binary table): + + [sh] funhead "foo.fits[EVENTS]" + XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE + BITPIX = 8 / Binary data + NAXIS = 2 / Table is a matrix + NAXIS1 = 20 / Width of table in bytes + NAXIS2 = 30760 / Number of entries in table + PCOUNT = 0 / Random parameter count + GCOUNT = 1 / Group count + TFIELDS = 7 / Number of fields in each row + EXTNAME = 'EVENTS ' / Table name + EXTVER = 1 / Version number of table + TFORM1 = '1I ' / Data type for field + TTYPE1 = 'X ' / Label for field + TUNIT1 = ' ' / Physical units for field + TFORM2 = '1I ' / Data type for field + etc. ... + END + + + +To display the third header: + + [sh] funhead "foo.fits[3]" + XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE + BITPIX = 8 / Binary data + NAXIS = 2 / Table is a matrix + NAXIS1 = 32 / Width of table in bytes + NAXIS2 = 40 / Number of entries in table + PCOUNT = 0 / Random parameter count + GCOUNT = 1 / Group count + TFIELDS = 7 / Number of fields in each row + EXTNAME = 'TGR ' / Table name + EXTVER = 1 / Version number of table + TFORM1 = '1D ' / Data type for field + etc. ... + END + + + +To display the primary header (i.e., extension 0): + + sh> funhead "coma.fits[0]" + SIMPLE = T /STANDARD FITS FORMAT + BITPIX = 16 /2-BYTE TWOS-COMPL INTEGER + NAXIS = 2 /NUMBER OF AXES + NAXIS1 = 800 / + NAXIS2 = 800 / + DATATYPE= 'INTEGER*2' /SHORT INTEGER + END + + + +The funhead program also can edit (i.e. add, delete, or modify) or +display individual headers parameters. Edit mode is signalled by the +presence of two additional command-line arguments: output file and +edit command file, in that order. Edit mode acts as a filter: the +output file will contain the entire input FITS file, including other +extensions. The edit command file can be "stdin", in which case edit +command are read from the standard input. + + +The edit command file contains parameter comments (having '#' in the +first column) and delete and assignment(modify or add) operations. A +delete operation is specified by preceding the parameter name with a +minus sign "-". A display operation (very useful in interactive +sessions, i.e., where the edit commands are taken from stdin) is +specified by preceding the parameter name with a question mark "?". In +either case, a parameter value need not be specified. An assignment +operation is specified in the same two ways that a parameter is +specified in a text header (but without the comment character that +precedes header params), i.e.: + + + +=over 4 + + + + +=item * + +FITS-style comments have an equal sign "=" between the keyword and +value and an optional slash "/" to signify a comment. The strict FITS +rules on column positions are not enforced. + + + +=item * + +Free-form comments can have an optional colon separator between the +keyword and value. In the absence of quote, all tokens after the +keyword are part of the value, i.e. no comment is allowed. + + +=back + + + + +For example, the following interactive session checks for the +existence of parameters, adds new parameters, modifies them, and +modifies and deletes existing parameters: + + sh$ ./funhead snr.ev foo.fits - + # look for FOO1 + ? FOO1 + WARNING: FOO1 not found + # add new foo1 + FOO1 = 100 + # add foo2 + FOO2 = 200 + # reset foo1 to a different value + FOO1 -1 + # delete foo2 + -FOO2 + # change existing value + EXTVER 2 + ? XS-SORT + XS-SORT = 'EOF ' / type of event sort + # delete existing value + -XS-SORT + # exit + ^D + + + +See Column-based Text Files +for more information about header parameter format. + + + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funhist.pod b/funtools/doc/pod/funhist.pod new file mode 100644 index 0000000..fc1885d --- /dev/null +++ b/funtools/doc/pod/funhist.pod @@ -0,0 +1,252 @@ +=pod + +=head1 NAME + + + +B<funhist - create a 1D histogram of a column (from a FITS binary table or raw event file) or an image> + + + +=head1 SYNOPSIS + + + + + +funhist [-n|-w|-T] <iname> [column] [[lo:hi:]bins] + + + + + +=head1 OPTIONS + + + + + + -n # normalize bin value by the width of each bin + -w # specify bin width instead of number of bins in arg3 + -T # output in rdb/starbase format (tab separators) + + + + +=head1 DESCRIPTION + + + + +B<funhist> creates a one-dimensional histogram from the specified +columns of a FITS Extension +binary table of a FITS file (or from a non-FITS raw event file), or +from a FITS image or array, and writes that histogram as an ASCII +table. Alternatively, the program can perform a 1D projection of one +of the image axes. + + +The first argument to the program is required, and specifies the +Funtools file: FITS table or image, raw event file, or array. If +"stdin" is specified, data are read from the standard input. Use +Funtools Bracket Notation to specify FITS +extensions, and filters. + + +For a table, the second argument also is required. It specifies the +column to use in generating the histogram. If the data file is of +type image (or array), the column is optional: if "x" (or "X"), "y" +(or "Y") is specified, then a projection is performed over the x +(dim1) or y (dim2) axes, respectively. (That is, this projection will +give the same results as a histogram performed on a table containing +the equivalent x,y event rows.) If no column name is specified or +"xy" (or "XY") is specified for the image, then a histogram is +performed on the values contained in the image pixels. + + +The argument that follows is optional and specifies the number of bins +to use in creating the histogram and, if desired, the range of bin +values. For image and table histograms, the range should specify the +min and max data values. For image histograms on the x and y axes, +the range should specify the min and max image bin values. If this +argument is omitted, the number of output bins for a table is +calculated either from the TLMIN/TLMAX headers values (if these exist +in the table FITS header for the specified column) or by going through +the data to calculate the min and max value. For an image, the number +of output bins is calculated either from the DATAMIN/DATAMAX header +values, or by going through the data to calculate min and max value. +(Note that this latter calculation might fail if the image cannot be +fit in memory.) If the data are floating point (table or image) and +the number of bins is not specified, an arbitrary default of 128 is +used. + + +For binary table processing, the B<-w> (bin width) switch can be used +to specify the width of each bin rather than the number of bins. Thus: + + funhist test.ev pha 1:100:5 + +means that 5 bins of width 20 are used in the histogram, while: + + funhist -w test.ev pha 1:100:5 + +means that 20 bins of width 5 are used in the histogram. + + +The data are divvied up into the specified number of bins and the +resulting 1D histogram (or projection) is output in ASCII table +format. For a table, the output displays the low_edge (inclusive) and +hi_edge (exclusive) values for the data. For example, a 15-row table +containing a "pha" column whose values range from -7.5 to 7.5 +can be processed thus: + + + [sh] funhist test.ev pha + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: -7.5 7.5 15 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 22 -7.50000000 -6.50000000 + 2 21 -6.50000000 -5.50000000 + 3 20 -5.50000000 -4.50000000 + 4 19 -4.50000000 -3.50000000 + 5 18 -3.50000000 -2.50000000 + 6 17 -2.50000000 -1.50000000 + 7 16 -1.50000000 -0.50000000 + 8 30 -0.50000000 0.50000000 + 9 16 0.50000000 1.50000000 + 10 17 1.50000000 2.50000000 + 11 18 2.50000000 3.50000000 + 12 19 3.50000000 4.50000000 + 13 20 4.50000000 5.50000000 + 14 21 5.50000000 6.50000000 + 15 22 6.50000000 7.50000000 + + [sh] funhist test.ev pha 1:6 + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: 0.5 6.5 6 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 16 0.50000000 1.50000000 + 2 17 1.50000000 2.50000000 + 3 18 2.50000000 3.50000000 + 4 19 3.50000000 4.50000000 + 5 20 4.50000000 5.50000000 + 6 21 5.50000000 6.50000000 + + [sh] funhist test.ev pha 1:6:3 + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: 0.5 6.5 3 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 33 0.50000000 2.50000000 + 2 37 2.50000000 4.50000000 + 3 41 4.50000000 6.50000000 + + + +For a table histogram, the B<-n>(normalize) switch can be used to +normalize the bin value by the width of the bin (i.e., hi_edge-lo_edge): + + [sh] funhist -n test.ev pha 1:6:3 + # data file: test.ev + # column: pha + # min,max,bins: 0.5 6.5 3 + # width normalization (val/(hi_edge-lo_edge)) is applied + + bin value lo_edge hi_edge + ------ --------------------- --------------------- --------------------- + 1 16.50000000 0.50000000 2.50000000 + 2 6.16666667 2.50000000 4.50000000 + 3 4.10000000 4.50000000 6.50000000 + +This could used, for example, to produce a light curve with values +having units of counts/second instead of counts. + + +For an image histogram, the output displays the low and high image +values (both inclusive) used to generate the histogram. For example, +in the following example, 184 pixels had a value of 1, 31 had a value +of 2, while only 2 had a value of 3,4,5,6, or 7: + + [sh] funhist test.fits + # data file: /home/eric/data/test.fits + # min,max,bins: 1 7 7 + + bin value lo_val hi_val + ------ --------------------- --------------------- --------------------- + 1 184.00000000 1.00000000 1.00000000 + 2 31.00000000 2.00000000 2.00000000 + 3 2.00000000 3.00000000 3.00000000 + 4 2.00000000 4.00000000 4.00000000 + 5 2.00000000 5.00000000 5.00000000 + 6 2.00000000 6.00000000 6.00000000 + 7 2.00000000 7.00000000 7.00000000 + + + +For the axis projection of an image, the output displays the low and +high image bins (both inclusive) used to generate the projection. For +example, in the following example, 21 counts had their X bin value of +2, etc.: + + [sh] funhist test.fits x 2:7 + # data file: /home/eric/data/test.fits + # column: X + # min,max,bins: 2 7 6 + + bin value lo_bin hi_bin + ------ --------------------- --------------------- --------------------- + 1 21.00000000 2.00000000 2.00000000 + 2 20.00000000 3.00000000 3.00000000 + 3 19.00000000 4.00000000 4.00000000 + 4 18.00000000 5.00000000 5.00000000 + 5 17.00000000 6.00000000 6.00000000 + 6 16.00000000 7.00000000 7.00000000 + + [sh] funhist test.fits x 2:7:2 + # data file: /home/eric/data/test.fits + # column: X + # min,max,bins: 2 7 2 + + bin value lo_bin hi_bin + ------ --------------------- --------------------- --------------------- + 1 60.00000000 2.00000000 4.00000000 + 2 51.00000000 5.00000000 7.00000000 + + + +You can use gnuplot or other plotting programs to graph the +results, using a script such as: + + #!/bin/sh + sed -e '1,/---- .*/d + /^$/,$d' | \ + awk '\ + BEGIN{print "set nokey; set title \"funhist\"; set xlabel \"bin\"; set ylabel \"counts\"; plot \"-\" with boxes"} \ + {print $3, $2, $4-$3}' | \ + gnuplot -persist - 1>/dev/null 2>&1 + + +Similar plot commands are supplied in the script B<funhist.plot>: + + funhist test.ev pha ... | funhist.plot gnuplot + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funidx.pod b/funtools/doc/pod/funidx.pod new file mode 100644 index 0000000..37bb712 --- /dev/null +++ b/funtools/doc/pod/funidx.pod @@ -0,0 +1,224 @@ +=pod + +=head1 NAME + + + +B<Funidx: Using Indexes to Filter Rows in a Table> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the user interface for +filtering rows in binary tables with indexes. + + + +=head1 DESCRIPTION + + + + + +Funtools Table Filtering allows rows in a +table to be selected based on the values of one or more columns in the +row. Because the actual filter code is compiled on the fly, it is very +efficient. However, for very large files (hundreds of Mb or larger), +evaluating the filter expression on each row can take a long time. Therefore, +funtools supports index files for columns, which are used automatically during +filtering to reduce dramatically the number of row evaluations performed. +The speed increase for indexed filtering can be an order of magnitude or +more, depending on the size of the file. + + +The funindex program creates an +index on one or more columns in a binary table. For example, to create an index +for the column pi in the file huge.fits, use: + + funindex huge.fits pi + +This will create an index named huge_pi.idx. + + +When a filter expression is initialized for row evaluation, funtools +looks for an index file for each column in the filter expression. If +found, and if the file modification date of the index file is later +than that of the data file, then the index will be used to reduce the +number of rows that are evaluated in the filter. When +Spatial Region Filtering is part of the +expression, the columns associated with the region are checked for index +files. + + +If an index file is not available for a given column, then in general, +all rows must be checked when that column is part of a filter +expression. This is not true, however, when a non-indexed column is +part of an AND expression. In this case, only the rows that pass the +other part of the AND expression need to be checked. Thus, in some cases, +filtering speed can increase significantly even if all columns are not +indexed. + + +Also note that certain types of filter expression syntax cannot make +use of indices. For example, calling functions with column names as +arguments implies that all rows must be checked against the function +value. Once again, however, if this function is part of an AND +expression, then a significant improvement in speed still is possible +if the other part of the AND expression is indexed. + + +For example, note below the dramatic speedup in searching a 1 Gb +file using an AND filter, even when one of the columns (pha) has no +index: + + + time fundisp \ + huge.fits'[idx_activate=0,idx_debug=1,pha=2348&&cir 4000 4000 1]' \ + "x y pha" + x y pha + ---------- ----------- ---------- + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 42.36u 13.07s 6:42.89 13.7% + + time fundisp \ + huge.fits'[idx_activate=1,idx_debug=1,pha=2348&&cir 4000 4000 1]' \ + "x y pha" + x y pha + ---------- ----------- ---------- + idxeq: [INDEF] + idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] + idxand(1): INDEF [IDX_OR_SORT] + idxall(1): [IDX_OR_SORT] + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 1.55u 0.37s 1:19.80 2.4% + + +When all columns are indexed, the increase in speed can be even more dramatic: + + time fundisp \ + huge.fits'[idx_activate=0,idx_debug=1,pi=770&&cir 4000 4000 1]' \ + "x y pi" + x y pi + ---------- ----------- ---------- + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 42.60u 12.63s 7:28.63 12.3% + + time fundisp \ + huge.fits'[idx_activate=1,idx_debug=1,pi=770&&cir 4000 4000 1]' \ + "x y pi" + x y pi + ---------- ----------- ---------- + idxeq: pi start=9473025,stop=9492240 => pi[ROW 9473025:9492240] + idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] + idxor sort/merge: pi[ROW 9473025:9492240] [IDX_OR_SORT] + idxmerge(5): [IDX_OR_SORT] pi[ROW] + idxall(1): [IDX_OR_SORT] + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 1.67u 0.30s 0:24.76 7.9% + + + +The miracle of indexed filtering (and indeed, of any indexing) is the +speed of the binary search on the index, which is of order log2(n) +instead of n. (The funtools binary search method is taken from +http://www.tbray.org/ongoing/When/200x/2003/03/22/Binary, to whom +grateful acknowledgement is made.) This means that the larger the +file, the better the performance. Conversely, it also means that for +small files, using an index (and the overhead involved) can slow +filtering down somewhat. Our tests indicate that on a file containing +a few tens of thousands of rows, indexed filtering can be 10 to 20 +percent slower than non-indexed filtering. Of course, your mileage +will vary with conditions (disk access speed, amount of available +memory, process load, etc.) + + +Any problem encountered during index processing will result in +indexing being turned off, and replaced by filtering all rows. You can turn +filtering off manually by setting the idx_activate variable to 0 (in a filter +expression) or the FILTER_IDX_ACTIVATE environment variable to 0 (in the global +environment). Debugging output showing how the indexes are being processed can +be displayed to stderr by setting the idx_debug variable to 1 (in a filter +expression) or the FILTER_IDX_DEBUG environment variable to 1 (in the global +environment). + + +Currently, indexed filtering only works with FITS binary tables and raw +event files. It does not work with text files. This restriction might be +removed in a future release. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funimage.pod b/funtools/doc/pod/funimage.pod new file mode 100644 index 0000000..7ec3f93 --- /dev/null +++ b/funtools/doc/pod/funimage.pod @@ -0,0 +1,314 @@ +=pod + +=head1 NAME + + + +B<funimage - create a FITS image from a Funtools data file> + + + +=head1 SYNOPSIS + + + + + +funimage [-a] <iname> <oname> [bitpix=n] +funimage [-l] <iname> <oname> <xcol:xdims> <ycol:ydims> <vcol> [bitpix=n] +funimage [-p x|y] <iname> <oname> [bitpix=n] + + + + + +=head1 OPTIONS + + + + + + -a # append to existing output file as an image extension + -l # input is a list file containing xcol, ycol, value + -p [x|y] # project along x or y axis to create a 1D image + + + + +=head1 DESCRIPTION + + + + +B<funimage> creates a primary FITS image from the specified +FITS Extension +and/or +Image Section +of a FITS file, or from an +Image Section +of a non-FITS array, or from a raw event file. + +The first argument to the program specifies the FITS input image, +array, or raw event file to process. If "stdin" is specified, data are +read from the standard input. Use Funtools +Bracket Notation to specify FITS extensions, image sections, and +filters. The second argument is the output FITS file. If "stdout" is +specified, the FITS image is written to the standard output. By +default, the output pixel values are of the same data type as those of the +input file (or type "int" when binning a table), but this can be +overridden using an optional third argument of the form: + + bitpix=n + +where n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, +respectively. + + +If the input data are of type image, the appropriate section is +extracted and blocked (based on how the +Image Section is specified), and +the result is written to the FITS primary image. When an integer +image containing the BSCALE and BZERO keywords is converted to float, +the pixel values are scaled and the scaling keywords are deleted from the +output header. When converting integer scaled data to integer +(possibly of a different size), the pixels are not scaled and the +scaling keywords are retained. + + +If the input data is a binary table or raw event file, these are +binned into an image, from which a section is extracted and blocked, +and written to a primary FITS image. In this case, it is necessary to +specify the two columns that will be used in the 2D binning. This can +be done on the command line using the B<bincols=(x,y)> keyword: + + funcnts "foo.ev[EVENTS,bincols=(detx,dety)]" + +The full form of the B<bincols=> specifier is: + + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]) + +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: + + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) + +Using this syntax, it is possible to bin any two columns of a binary +table at any bin size. Note that the tlmin, tlmax, and binsiz +specifiers can be omitted if TLMIN, TLMAX, and TDBIN header parameters +(respectively) are present in the FITS binary table header for the +column in question. Note also that if only one parameter is specified, +it is assumed to be tlmax, and tlmin defaults to 1. If two parameters +are specified, they are assumed to be tlmin and tlmax. +See Binning FITS Binary Tables and Non-FITS +Event Files for more information about binning parameters. + + +By default, a new 2D FITS image file is created and the image is written +to the primary HDU. If the B<-a> (append) switch is specified, +the image is appended to an existing FITS file as an IMAGE extension. +(If the output file does not exist, the switch is effectively ignored +and the image is written to the primary HDU.) This can be useful in a +shell programming environment when processing multiple FITS images +that you want to combine into a single final FITS file. + + +B<funimage> also can take input from a table containing columns of +x, y, and value (e.g., the output from B<fundisp -l> which +displays each image x and y and the number of counts at that +position.) When the B<-l> (list) switch is used, the input file is +taken to be a FITS or ASCII table containing (at least) three columns +that specify the x and y image coordinates and the value of that +image pixel. In this case, B<funimage> requires four extra +arguments: xcolumn:xdims, ycolumn:ydims, vcolumn and bitpix=n. The x +and y col:dim information takes the form: + + name:dim # values range from 1 to dim + name:min:max # values range from min to max + name:min:max:binsiz # dimensions scaled by binsize + +In particular, the min value should be used whenever the +minimum coordinate value is something other than one. For example: + + funimage -l foo.lst foo.fits xcol:0:512 ycol:0:512 value bitpix=-32 + + + +The list feature also can be used to read unnamed columns from standard +input: simply replace the column name with a null string. Note +that the dimension information is still required: + + funimage -l stdin foo.fits "":0:512 "":0:512 "" bitpix=-32 + 240 250 1 + 255 256 2 + ... + ^D + + + +The list feature provides a simple way to generate a blank image. +If you pass a Column-based Text File +to funimage in which the text header contains the required image +information, then funimage will correctly make a blank image. For +example, consider the following text file (called foo.txt): + + x:I:1:10 y:I:1:10 + ------ ------ + 0 0 + +This text file defines two columns, x and y, each of data type 32-bit int and +image dimension 10. The command: + + funimage foo.txt foo.fits bitpix=8 + +will create an empty FITS image called foo.fits containing a 10x10 +image of unsigned char: + + fundisp foo.fits + 1 2 3 4 5 6 7 8 9 10 + ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ + 10: 0 0 0 0 0 0 0 0 0 0 + 9: 0 0 0 0 0 0 0 0 0 0 + 8: 0 0 0 0 0 0 0 0 0 0 + 7: 0 0 0 0 0 0 0 0 0 0 + 6: 0 0 0 0 0 0 0 0 0 0 + 5: 0 0 0 0 0 0 0 0 0 0 + 4: 0 0 0 0 0 0 0 0 0 0 + 3: 0 0 0 0 0 0 0 0 0 0 + 2: 0 0 0 0 0 0 0 0 0 0 + 1: 1 0 0 0 0 0 0 0 0 0 + + + +Note that the text file must contain at least +one row of data. However, in the present example, event position 0,0 is +outside the limits of the image and will be ignored. (You can, of course, +use real x,y values to seed the image with data.) + + +Furthermore, you can use the TEXT filter specification to obviate the need for +an input text file altogether. The following command will create the same +10x10 char image without an actual input file: + + funimage stdin'[TEXT(x:I:10,y:I:10)]' foo.fits bitpix=8 < /dev/null +or + funimage /dev/null'[TEXT(x:I:10,y:I:10)]' foo.fits bitpix=8 + + + +You also can use either of these methods to generate a region mask simply +by appending a region inside the filter brackets and specfying B<mask=all> +along with the bitpix. For example, the following command will generate a +10x10 char mask using 3 regions: + + funimage stdin'[TEXT(x:I:10,y:I:10),cir(5,5,4),point(10,1),-cir(5,5,2)]' \ + foo.fits bitpix=8,mask=all < /dev/null + +The resulting mask looks like this: + + fundisp foo.fits + 1 2 3 4 5 6 7 8 9 10 + ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ + 10: 0 0 0 0 0 0 0 0 0 0 + 9: 0 0 0 0 0 0 0 0 0 0 + 8: 0 0 1 1 1 1 1 0 0 0 + 7: 0 1 1 1 1 1 1 1 0 0 + 6: 0 1 1 0 0 0 1 1 0 0 + 5: 0 1 1 0 0 0 1 1 0 0 + 4: 0 1 1 0 0 0 1 1 0 0 + 3: 0 1 1 1 1 1 1 1 0 0 + 2: 0 0 1 1 1 1 1 0 0 0 + 1: 0 0 0 0 0 0 0 0 0 2 + + + +You can use B<funimage> to create 1D image projections along the x +or y axis using the B<-p [x|y]> switch. This capability works for +both images and tables. For example consider a FITS table named ev.fits +containing the following rows: + + X Y + -------- -------- + 1 1 + 1 2 + 1 3 + 1 4 + 1 5 + 2 2 + 2 3 + 2 4 + 2 5 + 3 3 + 3 4 + 3 5 + 4 4 + 4 5 + 5 5 + + +A corresponding 5x5 image, called dim2.fits, would therefore contain: + + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 5: 1 1 1 1 1 + 4: 1 1 1 1 0 + 3: 1 1 1 0 0 + 2: 1 1 0 0 0 + 1: 1 0 0 0 0 + +A projection along the y axis can be performed on either the table or +the image: + + funimage -p y ev.fits stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 + + funimage -p y dim2.fits stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 + + + +Furthermore, you can create a 1D image projection along any column of +a table by using the B<bincols=[column]> filter specification and +specifying a single column. For example, the following command +projects the same 1D image along the y axis of a table as use of +the B<-p y> switch: + + funimage ev.fits'[bincols=y]' stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 + + + +Examples: + +Create a FITS image from a FITS binary table: + + [sh] funimage test.ev test.fits + + + +Display the FITS image generated from a blocked section of FITS binary table: + + [sh] funimage "test.ev[2:8,3:7,2]" stdout | fundisp stdin + 1 2 3 + --------- --------- --------- + 1: 20 28 36 + 2: 28 36 44 + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funimageget.pod b/funtools/doc/pod/funimageget.pod new file mode 100644 index 0000000..a569bba --- /dev/null +++ b/funtools/doc/pod/funimageget.pod @@ -0,0 +1,237 @@ +=pod + +=head1 NAME + + + +B<FunImageGet - get an image or image section> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void *FunImageGet(Fun fun, void *buf, char *plist) + + + + + +=head1 DESCRIPTION + + + + +The B<FunImageGet()> routine returns an binned image array of the +specified section of a Funtools data file. If the input data are +already of type image, the array is generated by extracting the +specified image section and then binning it according to the specified +bin factor. If the input data are contained in a binary table or raw +event file, the rows are binned on the columns specified by the +B<bincols=> keyword (using appropriate default columns as +necessary), after which the image section and bin factors are +applied. In both cases, the data is automatically converted from FITS +to native format, if necessary. + +The first argument is the Funtools handle returned by +FunOpen(). The second B<buf> +argument is a pointer to a data buffer to fill. If NULL is specified, +FunImageGet will allocate a buffer of the appropriate size. Generally +speaking, you always want Funtools to allocate the buffer because +the image dimensions will be determined by +Funtools image sectioning +on the command line. + +The third B<plist> (i.e., parameter list) argument is a string +containing one or more comma-delimited B<keyword=value> +parameters. It can be used to specify the return data type using the +B<bitpix=> keyword. If no such keyword is specified in the plist +string, the data type of the returned image is the same as the data type +of the original input file, or is of type int for FITS binary tables. + + +If the B<bitpix=> keyword is supplied in the plist string, the data +type of the returned image will be one of the supported FITS image +data types: + + +=over 4 + + + + +=item * + +8 unsigned char + + +=item * + +16 short + + +=item * + +32 int + + +=item * + +-32 float + + +=item * + +-64 double + + +=back + + +For example: + + void *buf; + /* extract data section into an image buffer */ + if( !(buf = FunImageGet(fun, NULL, NULL)) ) + gerror(stderr, "could not FunImageGet: %s\n", iname); + +will allocate buf and retrieve the image in the file data format. In +this case, you will have to determine the data type (using the +FUN_SECT_BITPIX value in the +FunInfoGet() +routine) +and then use a switch statement to process each data type: + + int bitpix; + void *buf; + unsigned char *cbuf; + short *sbuf; + int *ibuf; + ... + buf = FunImageGet(fun, NULL, NULL); + FunInfoGet(fun, FUN_SECT_BITPIX, &bitpix, 0); + /* set appropriate data type buffer to point to image buffer */ + switch(bitpix){ + case 8: + cbuf = (unsigned char *)buf; break; + case 16: + sbuf = (short *)buf; break; + case 32: + ibuf = (int *)buf; break; + ... + +See the +imblank example code +for more details on how to process an image when the data type is not +specified beforehand. + + +It often is easier to specify the data type directly: + + double *buf; + /* extract data section into a double image buffer */ + if( !(buf = FunImageGet(fun, NULL, "bitpix=-64")) ) + gerror(stderr, "could not FunImageGet: %s\n", iname); + +will extract the image while converting to type double. + + +On success, a pointer to the image buffer is returned. (This will be +the same as the second argument, if NULL is not passed to the latter.) +On error, NULL is returned. + + +In summary, to retrieve image or row data into a binned image, you simply +call FunOpen() followed by +FunImageGet(). Generally, you +then will want to call +FunInfoGet() +to retrieve the +axis dimensions (and data type) of the section you are processing +(so as to take account of sectioning and blocking of the original data): + + double *buf; + int i, j; + int dim1, dim2; + ... other declarations, etc. + + /* open the input FITS file */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + + /* extract and bin the data section into a double float image buffer */ + if( !(buf = FunImageGet(fun, NULL, "bitpix=-64")) ) + gerror(stderr, "could not FunImageGet: %s\n", argv[1]); + + /* get dimension information from funtools structure */ + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + + /* loop through pixels and reset values below limit to value */ + for(i=0; i<dim1*dim2; i++){ + if( buf[i] <= blimit ) buf[i] = bvalue; + } + + + +Another useful plist string value is "mask=all", which returns an +image populated with regions id values. Image pixels within a region +will contain the associated region id (region values start at 1), and +otherwise will contain a 0 value. Thus, the returned image is a +region mask which can be used to process the image data (which +presumably is retrieved by a separate call to FunImageGet) pixel by +pixel. + + +If a FITS binary table or a non-FITS raw event file is being binned +into an image, it is necessary to specify the two columns that will be +used in the 2D binning. This usually is done on the command line +using the B<bincols=(x,y)> keyword: + + funcnts "foo.ev[EVENTS,bincols=(detx,dety)]" + + + +The full form of the B<bincols=> specifier is: + + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]) + +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: + + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) + +These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, +TLMAX, and TDBIN header parameters (respectively) are present in the +FITS binary table header for the column in question. Note that if +only one parameter is specified, it is assumed to be tlmax, and tlmin +defaults to 1. If two parameters are specified, they are assumed to be +tlmin and tlmax. + + +If B<bincols> is not specified on the command line, Funtools tries +to use appropriate defaults: it looks for the environment variable +FITS_BINCOLS (or FITS_BINKEY). Then it looks for the Chandra +parameters CPREF (or PREFX) in the FITS binary table header. Failing +this, it looks for columns named "X" and "Y" and if these are not +found, it looks for columns containing the characters "X" and "Y". + +See Binning FITS Binary Tables and +Non-FITS Event Files for more information. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funimageput.pod b/funtools/doc/pod/funimageput.pod new file mode 100644 index 0000000..60eea7d --- /dev/null +++ b/funtools/doc/pod/funimageput.pod @@ -0,0 +1,145 @@ +=pod + +=head1 NAME + + + +B<FunImagePut - put an image to a Funtools file> + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunImagePut(Fun fun, void *buf, int dim1, int dim2, int bitpix, + char *plist) + + + + + +=head1 DESCRIPTION + + + +The B<FunImagePut()> routine outputs an image array to a FITS +file. The image is written either as a primary header/data unit or as +an image extension, depending on whether other data have already been +written to the file. That is, if the current file position is at the +beginning of the file, a primary HDU is written. Otherwise, an +image extension is written. + + +The first argument is the Funtools handle returned by +FunOpen(). The second B<buf> +argument is a pointer to a data buffer to write. The B<dim1>and +B<dim2> arguments that follow specify the dimensions of the image, +where dim1 corresponds to naxis1 and dim2 corresponds to naxis2. The +B<bitpix> argument specifies the data type of the image and can +have the following FITS-standard values: + + +=over 4 + + + + +=item * + +8 unsigned char + + +=item * + +16 short + + +=item * + +32 int + + +=item * + +-32 float + + +=item * + +-64 double + + +=back + + + + +When FunTableRowPut() is first +called for a given image, Funtools checks to see if the primary header +has already been written (by having previously written an image or a +binary table.) If not, this image is written to the primary HDU. +Otherwise, it is written to an image extension. + +Thus, a simple program to generate a FITS image might look like this: + + int i; + int dim1=512, dim2=512; + double *dbuf; + Fun fun; + dbuf = malloc(dim1*dim2*sizeof(double)); + /* open the output FITS image, preparing to copy input params */ + if( !(fun = FunOpen(argv[1], "w", NULL)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[1]); + for(i=0; i<(dim1*dim2); i++){ + ... fill dbuf ... + } + /* put the image (header will be generated automatically */ + if( !FunImagePut(fun, buf, dim1, dim2, -64, NULL) ) + gerror(stderr, "could not FunImagePut: %s\n", argv[1]); + FunClose(fun); + free(dbuf); + + + +In addition, if a +Funtools reference handle +was specified when this table was opened, the +parameters from this +Funtools reference handle +are merged into the new image +header. Furthermore, if a reference image was specified during +FunOpen(), the values of +B<dim1>, B<dim2>, and B<bitpix> in the calling sequence +can all be set to 0. In this case, default values are taken from the +reference image section. This is useful if you are reading an image +section in its native data format, processing it, and then writing +that section to a new FITS file. See the +imblank example code. + + +The data are assumed to be in the native machine format and will +automatically be swapped to FITS big-endian format if necessary. This +behavior can be over-ridden with the B<convert=[true|false]> +keyword in the B<plist> param list string. + + +When you are finished writing the image, you should call +FunFlush() to write out the FITS +image padding. However, this is not necessary if you subsequently call +FunClose() without doing any other I/O to the FITS file. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funimagerowget.pod b/funtools/doc/pod/funimagerowget.pod new file mode 100644 index 0000000..91c776a --- /dev/null +++ b/funtools/doc/pod/funimagerowget.pod @@ -0,0 +1,137 @@ +=pod + +=head1 NAME + + + +B<FunImageRowGet - get row(s) of an image> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void *FunImageRowGet(Fun fun, void *buf, int rstart, int rstop, + char *plist) + + + + + +=head1 DESCRIPTION + + + + +The B<FunImageRowGet()> routine returns one or more image rows +from the specified section of a Funtools data file. If the input data +are of type image, the array is generated by extracting the specified +image rows and then binning them according to the specified bin +factor. If the input data are contained in a binary table or raw +event file, the rows are binned on the columns specified by the +B<bincols=> keyword (using appropriate default columns as needed), +after which the image section and bin factors are applied. + + +The first argument is the Funtools handle returned by +FunOpen(). The second B<buf> +argument is a pointer to a data buffer to fill. If NULL is specified, +FunImageGet() will allocate a buffer of the appropriate size. + + +The third and fourth arguments specify the first and last row to +retrieve. Rows are counted starting from 1, up to the value of +FUN_YMAX(fun). The final B<plist> (i.e., parameter list) argument +is a string containing one or more comma-delimited +B<keyword=value> parameters. It can be used to specify the return +data type using the B<bitpix=> keyword. If no such keyword is +specified in the plist string, the data type of the image is the same +as the data type of the original input file, or is of type int for +FITS binary tables. + + +If the B<bitpix=>value is supplied in the plist string, the data +type of the returned image will be one of the supported FITS image +data types: + + +=over 4 + + + + +=item * + +8 unsigned char + + +=item * + +16 short + + +=item * + +32 int + + +=item * + +-32 float + + +=item * + +-64 double + + +=back + + + + +For example: + + double *drow; + Fun fun; + ... open files ... + /* get section dimensions */ + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + /* allocate one line's worth */ + drow = malloc(dim1*sizeof(double)); + /* retrieve and process each input row (starting at 1) */ + for(i=1; i <= dim2; i++){ + if( !FunImageRowGet(fun, drow, i, i, "bitpix=-64") ) + gerror(stderr, "can't FunImageRowGet: %d %s\n", i, iname); + /* reverse the line */ + for(j=1; j<=dim1; j++){ + ... process drow[j-1] ... + } + } + ... + + + +On success, a pointer to the image buffer is returned. (This will be +the same as the second argument, if NULL is not passed to the latter.) +On error, NULL is returned. Note that the considerations described +above for specifying binning columns in +FunImageGet() also apply to +B<FunImageRowGet()>. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funimagerowput.pod b/funtools/doc/pod/funimagerowput.pod new file mode 100644 index 0000000..2ff05cc --- /dev/null +++ b/funtools/doc/pod/funimagerowput.pod @@ -0,0 +1,122 @@ +=pod + +=head1 NAME + + + +B<FunImageRowPut - put row(s) of an image> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void *FunImageRowPut(Fun fun, void *buf, int rstart, int rstop, + int dim1, int dim2, int bitpix, char *plist) + + + + + +=head1 DESCRIPTION + + + + +The B<FunImageRowPut()> routine writes one or more image rows to +the specified FITS image file. The first argument is the Funtools +handle returned by FunOpen(). +The second B<buf> argument is a pointer to the row data buffer, +while the third and fourth arguments specify the starting and ending +rows to write. Valid rows values range from 1 to dim2, i.e., row is +one-valued. + + +The B<dim1>and B<dim2> arguments that follow specify the +dimensions, where dim1 corresponds to naxis1 and dim2 corresponds to +naxis2. The B<bitpix> argument data type of the image and can +have the following FITS-standard values: + + +=over 4 + + + + +=item * + +8 unsigned char + + +=item * + +16 short + + +=item * + +32 int + + +=item * + +-32 float + + +=item * + +-64 double + + +=back + + + +For example: + + double *drow; + Fun fun, fun2; + ... open files ... + /* get section dimensions */ + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + /* allocate one line's worth */ + drow = malloc(dim1*sizeof(double)); + /* retrieve and process each input row (starting at 1) */ + for(i=1; i <= dim2; i++){ + if( !FunImageRowGet(fun, drow, i, i, "bitpix=-64") ) + gerror(stderr, "can't FunImageRowGet: %d %s\n", i, iname); + ... process drow ... + if( !FunImageRowPut(fun2, drow, i, i, 64, NULL) ) + gerror(stderr, "can't FunImageRowPut: %d %s\n", i, oname); + } + ... + + + +The data are assumed to be in the native machine format and will +automatically be swapped to big-endian FITS format if necessary. This +behavior can be over-ridden with the B<convert=[true|false]> +keyword in the B<plist> param list string. + + +When you are finished writing the image, you should call +FunFlush() to write out the FITS +image padding. However, this is not necessary if you subsequently call +FunClose() without doing any other I/O to the FITS file. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funindex.pod b/funtools/doc/pod/funindex.pod new file mode 100644 index 0000000..cdc2fc7 --- /dev/null +++ b/funtools/doc/pod/funindex.pod @@ -0,0 +1,83 @@ +=pod + +=head1 NAME + + + +B<funindex - create an index for a column of a FITS binary table> + + + +=head1 SYNOPSIS + + + + + +funindex <switches> <iname> <key> [oname] + + + + + +=head1 OPTIONS + + + + + + NB: these options are not compatible with Funtools processing. Please + use the defaults instead. + -c # compress output using gzip" + -a # ASCII output, ignore -c (default: FITS table)" + -f # FITS table output (default: FITS table)" + -l # long output, i.e. with key value(s) (default: long)" + -s # short output, i.e. no key value(s) (default: long)" + + + + +=head1 DESCRIPTION + + + + +The funindex script creates an index for the specified column (key) by +running funtable -s (sort) and then saving the column value and the +record number for each sorted row. This index will be used automatically + by funtools filtering of that column, provided the index file's modification +date is later than that of the data file. + + +The first required argument is the name of the FITS binary table +to index. Please note that text files cannot be indexed at this time. +The second required argument is the column (key) name to index. While +multiple keys can be specified in principle, the funtools index processing +assume a single key and will not recognize files containing multiple keys. + + +By default, the output index file name is [root]_[key].idx, where [root] +is the root of the input file. Funtools looks for this specific file name +when deciding whether to use an index for faster filtering. Therefore, the +optional third argument (output file name) should not be used for funtools +processing. + + +For example, to create an index on column Y for a given FITS file, use: + + funindex foo.fits Y + +This will generate an index named foo_y.idx, which will be used by funtools +for filters involving the Y column. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funindexes.pod b/funtools/doc/pod/funindexes.pod new file mode 100644 index 0000000..e4cabaa --- /dev/null +++ b/funtools/doc/pod/funindexes.pod @@ -0,0 +1,215 @@ +=pod + +=head1 NAME + + + +B<Funindexes: Using Indexes to Filtering Rows in a Table> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the user interface for +filtering rows in binary tables with indexes. + + + +=head1 DESCRIPTION + + + + + +Funtools Table Filtering allows rows in a +table to be selected based on the values of one or more columns in the +row. Because the actual filter code is compiled on the fly, it is very +efficient. For very large files (hundreds of Mb or larger), however, +evaluating the filter expression on each row can take a long time. Therefore, +funtools supports index files for columns, which are used automatically during +filtering to reduce dramatically the number of row evaluations performed. +The speed increase for indexed filtering can be an order of magnitude or +more, depending on the size of the file. + + +The funindex program creates a +index on column in a binary table. For example, to create an index +for the column pi in the file huge.fits, use: + + funindex huge.fits pi + +This will create an index named huge_pi.idx. + + +When a filter expression is initialized for row evaluation, funtools +looks for an index file for each column in the filter expression. If +found, and if the file modification date of the index file is later +than that of the data file, then the index will be used to reduce the +number of rows that are evaluated in the filter. When <A +HREF="./regions.html">Spatial Region Filtering is part of the +expression, the columns associated with the region checked for index +files. + + +If an index file is not available for a given column, then in general, +all rows must be checked when that column is part of a filter +expression. This is not true, however, when a non-indexed column is +part of an AND expression. In this case, only the rows that pass the +other part of the AND expression need to be checked. Thus, in some cases, +filtering speed can increase significantly even if all columns are not +indexed. + + +Also note that certain types of filter expression syntax cannot make +use of indices. For example, calling functions with column names as +arguments implies that all rows must be checked against the function +value. Once again, however, if this function is part of an AND +expression, then a significant improvement in speed still is possible +if the other part of the AND expression is indexed. + + +As an example, note below the dramatic speedup in searching a 1 Gb +file using an AND filter, even when one of the columns (pha) has no +index: + + +bynars-16: time fundisp huge.fits'[idx_use=0,idx_debug=1,pha=2348&&cir 4000 4000 1]' "x y pha" + x y pha +----------- ----------- ---------- + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 +42.36u 13.07s 6:42.89 13.7% + +bynars-17: time fundisp huge.fits'[idx_use=1,idx_debug=1,pha=2348&&cir 4000 4000 1]' "x y pha" + x y pha +----------- ----------- ---------- +idxeq: [INDEF] +idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] +idxand(1): INDEF [IDX_OR_SORT] +idxall(1): [IDX_OR_SORT] + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 + 3999.48 4000.47 2348 +1.55u 0.37s 1:19.80 2.4% + + +When all columns are indexed, the increase in speed can be even more dramatic: + +bynars-20: time fundisp huge.fits'[idx_use=0,idx_debug=1,pi=770&&cir 4000 4000 1]' "x y pi" + x y pi +----------- ----------- ---------- + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 +42.60u 12.63s 7:28.63 12.3% + +bynars-21: time fundisp huge.fits'[idx_use=1,idx_debug=1,pi=770&&cir 4000 4000 1]' "x y pi" + x y pi +----------- ----------- ---------- +idxeq: pi start=9473025,stop=9492240 => pi[ROW 9473025:9492240] +idxand sort: x[ROW 8037025:8070128] y[ROW 5757665:5792352] +idxor sort/merge: pi[ROW 9473025:9492240] [IDX_OR_SORT] +idxmerge(5): [IDX_OR_SORT] pi[ROW] +idxall(1): [IDX_OR_SORT] + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 + 3999.48 4000.47 770 +1.67u 0.30s 0:24.76 7.9% + + + +The miracle of indexed filtering (and indeed, of any indexing) is due +to the speed of the binary search on the index, which is of order +log2(n) instead of n. (The funtools binary search method is taken from +http://www.tbray.org/ongoing/When/200x/2003/03/22/Binary, to whom +grateful acknowledgement is made.) This means that the larger the +file, the better the performance. Conversely, it also means that +for small files, using an index (and the overhead involved) can slow +filtering down somewhat. Our tests indicate that on a file containing +a few tens of thousands of rows, indexed filtering can be 10-20 +percent slower. Of course, your mileage will vary with conditions +(disk access speed, amount of available memory, process load, etc.) + + +Any problem encountered during index processing is supposed to result in +indexing being turned off, replaced by filtering all rows. You can turn +filtering off manually by setting the idx_use variable to 0 (in a filter +expression) or the FILTER_IDX_USE environment variable to 0 (in the global +environment). Debugging output showing how the indexes are being processed can +be displayed to stderr by setting the idx_debug variable to 1 (in a filter +expression) or the FILTER_IDX_DEBUG environment variable to 1 (in the global +environment). + + +Currently, indexed filtering only works with FITS binary tables and raw +event files. It does not work with text files. This restriction might be +removed in a future release. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funinfoget.pod b/funtools/doc/pod/funinfoget.pod new file mode 100644 index 0000000..456ac0c --- /dev/null +++ b/funtools/doc/pod/funinfoget.pod @@ -0,0 +1,226 @@ +=pod + +=head1 NAME + + + +B<FunInfoGet - get information from Funtools struct> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunInfoGet(Fun fun, int type, char *addr, ...) + + + + + +=head1 DESCRIPTION + + + + +The B<FunInfoGet()> routine returns information culled from the +Funtools structure. The first argument is the Fun handle from which +information is to be retrieved. This first required argument is followed +by a variable length list of pairs of arguments. Each pair consists +of an integer representing the type of information to retrieve and the +address where the information is to be stored. The list is terminated by a 0. +The routine returns the number of get actions performed. + + +The full list of available information is described below. Please note +that only a few of these will be useful to most application developers. +For imaging applications, the most important types are: + + FUN_SECT_DIM1 int /* dim1 for section */ + FUN_SECT_DIM2 int /* dim2 for section */ + FUN_SECT_BITPIX int /* bitpix for section */ + +These would be used to determine the dimensions and data type of image +data retrieved using the +FunImageGet() routine. For +example: + + /* extract and bin the data section into an image buffer */ + buf = FunImageGet(fun, NULL, NULL); + /* get required information from funtools structure. + this should come after the FunImageGet() call, in case the call + changed sect_bitpix */ + FunInfoGet(fun, + FUN_SECT_BITPIX, &bitpix, + FUN_SECT_DIM1, &dim1, + FUN_SECT_DIM2, &dim2, + 0); + /* loop through pixels and reset values below limit to value */ + for(i=0; i<dim1*dim2; i++){ + switch(bitpix){ + case 8: + if( cbuf[i] <= blimit ) cbuf[i] = bvalue; + ... + } + +It is important to bear in mind that the call to +FunImageGet() +can change the value of FUN_SECT_BITPIX (e.g. if "bitpix=n" is passed +in the param list). Therefore, a call to +FunInfoGet() +should be made B<after> the call to +FunImageGet(), +in order to retrieve the updated bitpix value. +See the imblank example code for more +details. + + +It also can be useful to retrieve the World Coordinate System +information from the Funtools structure. Funtools uses the the WCS +Library developed by Doug Mink at SAO, which is available +here. +(More information about the WCSTools project in general can be found +here.) +The FunOpen() routine initializes +two WCS structures that can be used with this WCS Library. +Applications can retrieve either of these two WCS structures using +B<FunInfoGet()>: + + FUN_WCS struct WorldCoor * /* wcs structure, for image coordinates*/ + FUN_WCS0 struct WorldCoor * /* wcs structure, for physical coordinates */ + +The structure retrieved by FUN_WCS is a WCS library handle containing +parameters suitable for use with image coordinates, regardless of whether the +data are images or tables. For this structure, the WCS reference point +(CRPIX) has been converted to image coordinates if the underlying file +is a table (and therefore in physical coordinates). You therefore must +ensure that the positions being passed to a routine like pix2wcs are in +image coordinates. The FUN_WCS0 structure has not had its WCS +reference point converted to image coordinates. It therefore is useful +when passing processing physical coordinates from a table. + + +Once a WCS structure has been retrieved, it can be used as the first +argument to the WCS library routines. (If the structure is NULL, no +WCS information was contained in the file.) The two important WCS routines +that Funtools uses are: + + #include <wcs.h> + void pix2wcs (wcs,xpix,ypix,xpos,ypos) + struct WorldCoor *wcs; /* World coordinate system structure */ + double xpix,ypix; /* x and y coordinates in pixels */ + double *xpos,*ypos; /* RA and Dec in degrees (returned) */ + + +which converts pixel coordinates to sky coordinates, and: + + + void wcs2pix (wcs, xpos, ypos, xpix, ypix, offscl) + struct WorldCoor *wcs; /* World coordinate system structure */ + double xpos,ypos; /* World coordinates in degrees */ + double *xpix,*ypix; /* coordinates in pixels */ + int *offscl; /* 0 if within bounds, else off scale */ + +which converts sky coordinates to pixel coordinates. Again, please note +that the wcs structure returned by FUN_WCS assumes that image coordinates +are passed to the pix2wcs routine, while FUN_WCS0 assumes that physical +coordinates are passed. + + +Note that funtools.h file automatically includes wcs.h. An example +program that utilizes these WCS structure to call WCS Library routines +is twcs.c. + + +The following is the complete list of information that can be returned: + + name type comment + --------- -------- --------------------------------------------- + FUN_FNAME char * /* file name */ + FUN_GIO GIO /* gio handle */ + FUN_HEADER FITSHead /* fitsy header struct */ + FUN_TYPE int /* TY_TABLE,TY_IMAGE,TY_EVENTS,TY_ARRAY */ + FUN_BITPIX int /* bits/pixel in file */ + FUN_MIN1 int /* tlmin of axis1 -- tables */ + FUN_MAX1 int /* tlmax of axis1 -- tables */ + FUN_MIN2 int /* tlmin of axis2 -- tables */ + FUN_MAX2 int /* tlmax of axis2 -- tables */ + FUN_DIM1 int /* dimension of axis1 */ + FUN_DIM2 int /* dimension of axis2 */ + FUN_ENDIAN int /* 0=little, 1=big endian */ + FUN_FILTER char * /* supplied filter */ + FUN_IFUN FITSHead /* pointer to reference header */ + FUN_IFUN0 FITSHead /* same as above, but no reset performed */ + /* image information */ + FUN_DTYPE int /* data type for images */ + FUN_DLEN int /* length of image in bytes */ + FUN_DPAD int /* padding to end of extension */ + FUN_DOBLANK int /* was blank keyword defined? */ + FUN_BLANK int /* value for blank */ + FUN_SCALED int /* was bscale/bzero defined? */ + FUN_BSCALE double /* bscale value */ + FUN_BZERO double /* bzero value */ + /* table information */ + FUN_NROWS int /* number of rows in file (naxis2) */ + FUN_ROWSIZE int /* size of user row struct */ + FUN_BINCOLS char * /* specified binning columns */ + FUN_OVERFLOW int /* overflow detected during binning? */ + /* array information */ + FUN_SKIP int /* bytes to skip in array header */ + /* section information */ + FUN_SECT_X0 int /* low dim1 value of section */ + FUN_SECT_X1 int /* hi dim1 value of section */ + FUN_SECT_Y0 int /* low dim2 value of section */ + FUN_SECT_Y1 int /* hi dim2 value of section */ + FUN_SECT_BLOCK int /* section block factor */ + FUN_SECT_BTYPE int /* 's' (sum), 'a' (average) for binning */ + FUN_SECT_DIM1 int /* dim1 for section */ + FUN_SECT_DIM2 int /* dim2 for section */ + FUN_SECT_BITPIX int /* bitpix for section */ + FUN_SECT_DTYPE int /* data type for section */ + FUN_RAWBUF char * /* pointer to raw row buffer */ + FUN_RAWSIZE int /* byte size of raw row records */ + /* column information */ + FUN_NCOL int /* number of row columns defined */ + FUN_COLS FunCol /* array of row columns */ + /* WCS information */ + FUN_WCS struct WorldCoor * /* wcs structure, converted for images*/ + FUN_WCS0 struct WorldCoor * /* wcs structure, not converted */ + + + +Row applications would not normally need any of this information. +An example of how these values can be used in more complex programs is +the evnext example code. In this program, the +time value for each row is changed to be the value of the succeeding +row. The program thus reads the time values for a batch of rows, +changes the time values to be the value for the succeeding row, and +then merges these changed time values back with the other columns to +the output file. It then reads the next batch, etc. + + +This does not work for the last row read in each batch, since there +is no succeeding row until the next batch is read. Therefore, the +program saves that last row until it has read the next batch, then +processes the former before starting on the new batch. In order to +merge the last row successfully, the code uses FUN_RAWBUF to save +and restore the raw input data associated with each batch of +rows. Clearly, this requires some information about how funtools +works internally. We are happy to help you write such programs as the +need arises. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funinfoput.pod b/funtools/doc/pod/funinfoput.pod new file mode 100644 index 0000000..5124dfc --- /dev/null +++ b/funtools/doc/pod/funinfoput.pod @@ -0,0 +1,180 @@ +=pod + +=head1 NAME + + + +B<FunInfoPut - put information into a Funtools struct> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunInfoPut(Fun fun, int type, char *addr, ...) + + + + + +=head1 DESCRIPTION + + + + +The B<FunInfoPut()> routine puts information into a Funtools +structure. The first argument is the Fun handle from which +information is to be retrieved. After this first required argument +comes a variable length list of pairs of arguments. Each pair consists +of an integer representing the type of information to store and the +address of the new information to store in the struct. The variable +list is terminated by a 0. The routine returns the number of put +actions performed. + + +The full list of available information is described above with the +FunInfoPut() routine. Although +use of this routine is expected to be uncommon, there is one +important situation in which it plays an essential part: writing +multiple extensions to a single output file. + + +For input, multiple extensions are handled by calling +FunOpen() for each extension to be +processed. When opening multiple inputs, it sometimes is the case that +you will want to process them and then write them (including their +header parameters) to a single output file. To accomplish this, you +open successive input extensions using +FunOpen() and then call +B<FunInfoPut()> to set the +Funtools reference handle +of the output file to that of the newly opened input extension: + + /* open a new input extension */ + ifun=FunOpen(tbuf, "r", NULL)) ) + /* make the new extension the reference handle for the output file */ + FunInfoPut(ofun, FUN_IFUN, &ifun, 0); + + +Resetting FUN_IFUN has same effect as when a funtools handle is passed +as the final argument to +FunOpen(). The state of the output +file is reset so that a new extension is ready to be written. +Thus, the next I/O call on the output extension will output the +header, as expected. + + +For example, in a binary table, after resetting FUN_IFUN you can then +call FunColumnSelect() to +select the columns for output. When you then call +FunImagePut() or <A +HREF="./library.html#funtablerowput">FunTableRowPut(), a new +extension will be written that contains the header parameters from the +reference extension. Remember to call +FunFlush() to complete output of a +given extension. + + +A complete example of this capability is given +in the evcol example code. +The central algorithm is: + + +=over 4 + + + + +=item * + +open the output file without a reference handle + + +=item * + +loop: open each input extension in turn + + +=over 4 + + + + +=item * + +set the reference handle for output to the newly opened input extension + + +=item * + +read the input rows or image and perform processing + + +=item * + +write new rows or image to the output file + + +=item * + +flush the output + + +=item * + +close input extension + + +=back + + + + +=item * + +close output file + + +=back + + +Note that FunFlush() is called +after processing each input extension in order to ensure that the +proper padding is written to the output file. A call to +FunFlush() also ensures that the +extension header is written to the output file in the case where there +are no rows to output. + + +If you wish to output a new extension without using a +Funtools reference handle, you can +call FunInfoPut() to reset the FUN_OPS value directly. For a binary +table, you would then call FunColumnSelect() to set up the columns for +this new extension. + + /* reset the operations performed on this handle */ + int ops=0; + FunInfoPut(ofun, FUN_OPS, &ops, 0); + FunColumnSelect(fun, sizeof(EvRec), NULL, + "MYCOL", "J", "w", FUN_OFFSET(Ev, mycol), + NULL); + +Once the FUN_OPS variable has been reset, the next I/O call on the +output extension will output the header, as expected. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funjoin.pod b/funtools/doc/pod/funjoin.pod new file mode 100644 index 0000000..d7830f9 --- /dev/null +++ b/funtools/doc/pod/funjoin.pod @@ -0,0 +1,233 @@ +=pod + +=head1 NAME + + + +B<funjoin - join two or more FITS binary tables on specified columns> + + + +=head1 SYNOPSIS + + + + + +funjoin [switches] <ifile1> <ifile2> ... <ifilen> <ofile> + + + + + +=head1 OPTIONS + + + + + + -a cols # columns to activate in all files + -a1 cols ... an cols # columns to activate in each file + -b 'c1:bvl,c2:bv2' # blank values for common columns in all files + -bn 'c1:bv1,c2:bv2' # blank values for columns in specific files + -j col # column to join in all files + -j1 col ... jn col # column to join in each file + -m min # min matches to output a row + -M max # max matches to output a row + -s # add 'jfiles' status column + -S col # add col as status column + -t tol # tolerance for joining numeric cols [2 files only] + + + + +=head1 DESCRIPTION + + + +B<funjoin> joins rows from two or more (up to 32) +FITS Binary Table files, based on the values +of specified join columns in each file. NB: the join columns must have +an index file associated with it. These files are generated using the +B<funindex> program. + + +The first argument to the program specifies the first input FITS table +or raw event file. If "stdin" is specified, data are read from the +standard input. Subsequent arguments specify additional event files +and tables to join. The last argument is the output FITS file. + + +NB: Do B<not> use Funtools Bracket +Notation to specify FITS extensions and row filters when running +funjoin or you will get wrong results. Rows are accessed and joined +using the index files directly, and this bypasses all filtering. + + +The join columns are specified using the B<-j col> switch (which +specifies a column name to use for all files) or with B<-j1 col1>, +B<-j2 col2>, ... B<-jn coln> switches (which specify a column +name to use for each file). A join column must be specified for each file. +If both B<-j col> and B<-jn coln> are specified for a given +file, then the latter is used. Join columns must either be of type +string or type numeric; it is illegal to mix numeric and string +columns in a given join. For example, to join three files using the +same key column for each file, use: + + funjoin -j key in1.fits in2.fits in3.fits out.fits + +A different key can be specified for the third file in this way: + + funjoin -j key -j3 otherkey in1.fits in2.fits in3.fits out.fits + + + +The B<-a "cols"> switch (and B<-a1 "col1">, +B<-a2 "cols2"> counterparts) can be used to specify columns to +activate (i.e. write to the output file) for each input file. By +default, all columns are output. + + +If two or more columns from separate files have the same name, the +second (and subsequent) columns are renamed to have an underscore +and a numeric value appended. + + +The B<-m min> and B<-M max> switches specify the minimum +and maximum number of joins required to write out a row. The default +minimum is 0 joins (i.e. all rows are written out) and the default maximum +is 63 (the maximum number of possible joins with a limit of 32 input files). +For example, to write out only those rows in which exactly two files +have columns that match (i.e. one join): + + funjoin -j key -m 1 -M 1 in1.fits in2.fits in3.fits ... out.fits + + + +A given row can have the requisite number of joins without all of the +files being joined (e.g. three files are being joined but only two +have a given join key value). In this case, all of the columns of the +non-joined file are written out, by default, using blanks (zeros or NULLs). +The B<-b c1:bv1,c2:bv2> and +B<-b1 'c1:bv1,c2:bv2' -b2 'c1:bv1,c2:bv2' ...> +switches can be used to set the blank value for columns common to all +files and/or columns in a specified file, respectively. Each blank value +string contains a comma-separated list of column:blank_val specifiers. +For floating point values (single or double), a case-insensitive string +value of "nan" means that the IEEE NaN (not-a-number) should be +used. Thus, for example: + + funjoin -b "AKEY:???" -b1 "A:-1" -b3 "G:NaN,E:-1,F:-100" ... + +means that a non-joined AKEY column in any file will contain the +string "???", the non-joined A column of file 1 will contain a value +of -1, the non-joined G column of file 3 will contain IEEE NaNs, while +the non-joined E and F columns of the same file will contain values -1 +and -100, respectively. Of course, where common and specific blank values +are specified for the same column, the specific blank value is used. + + +To distinguish which files are non-blank components of a given row, +the B<-s> (status) switch can be used to add a bitmask column named +"JFILES" to the output file. In this column, a bit is set for each +non-blank file composing the given row, with bit 0 corresponds to the +first file, bit 1 to the second file, and so on. The file names +themselves are stored in the FITS header as parameters named JFILE1, +JFILE2, etc. The B<-S col> switch allows you to change the name +of the status column from the default "JFILES". + + +A join between rows is the Cartesian product of all rows in one file +having a given join column value with all rows in a second file having +the same value for its join column and so on. Thus, if file1 has 2 +rows with join column value 100, file2 has 3 rows with the same value, +and file3 has 4 rows, then the join results in 2*3*4=24 rows being output. + + +The join algorithm directly processes the index file associated with +the join column of each file. The smallest value of all the current +columns is selected as a base, and this value is used to join +equal-valued columns in the other files. In this way, the index files +are traversed exactly once. + + +The B<-t tol> switch specifies a tolerance value for numeric +columns. At present, a tolerance value can join only two files at a +time. (A completely different algorithm is required to join more than +two files using a tolerance, somethng we might consider implementing +in the future.) + + +The following example shows many of the features of funjoin. The input files +t1.fits, t2.fits, and t3.fits contain the following columns: + + [sh] fundisp t1.fits + AKEY KEY A B + ----------- ------ ------ ------ + aaa 0 0 1 + bbb 1 3 4 + ccc 2 6 7 + ddd 3 9 10 + eee 4 12 13 + fff 5 15 16 + ggg 6 18 19 + hhh 7 21 22 + +fundisp t2.fits + AKEY KEY C D + ----------- ------ ------ ------ + iii 8 24 25 + ggg 6 18 19 + eee 4 12 13 + ccc 2 6 7 + aaa 0 0 1 + +fundisp t3.fits + AKEY KEY E F G +------------ ------ -------- -------- ----------- + ggg 6 18 19 100.10 + jjj 9 27 28 200.20 + aaa 0 0 1 300.30 + ddd 3 9 10 400.40 + + + +Given these input files, the following funjoin command: + + + funjoin -s -a1 "-B" -a2 "-D" -a3 "-E" -b \ + "AKEY:???" -b1 "AKEY:XXX,A:255" -b3 "G:NaN,E:-1,F:-100" \ + -j key t1.fits t2.fits t3.fits foo.fits + +will join the files on the KEY column, outputting all columns except B +(in t1.fits), D (in t2.fits) and E (in t3.fits), and setting blank +values for AKEY (globally, but overridden for t1.fits) and A (in file +1) and G, E, and F (in file 3). A JFILES column will be output to +flag which files were used in each row: + + + AKEY KEY A AKEY_2 KEY_2 C AKEY_3 KEY_3 F G JFILES + ------------ ------ ------ ------------ ------ ------ ------------ ------ -------- ----------- -------- + aaa 0 0 aaa 0 0 aaa 0 1 300.30 7 + bbb 1 3 ??? 0 0 ??? 0 -100 nan 1 + ccc 2 6 ccc 2 6 ??? 0 -100 nan 3 + ddd 3 9 ??? 0 0 ddd 3 10 400.40 5 + eee 4 12 eee 4 12 ??? 0 -100 nan 3 + fff 5 15 ??? 0 0 ??? 0 -100 nan 1 + ggg 6 18 ggg 6 18 ggg 6 19 100.10 7 + hhh 7 21 ??? 0 0 ??? 0 -100 nan 1 + XXX 0 255 iii 8 24 ??? 0 -100 nan 2 + XXX 0 255 ??? 0 0 jjj 9 28 200.20 4 + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funlib.pod b/funtools/doc/pod/funlib.pod new file mode 100644 index 0000000..354e217 --- /dev/null +++ b/funtools/doc/pod/funlib.pod @@ -0,0 +1,542 @@ +=pod + +=head1 NAME + + + +B<FunLib: the Funtools Programming Interface> + + + +=head1 SYNOPSIS + + + + +A description of the Funtools library. + + + +=head1 DESCRIPTION + + + +B<Introduction to the Funtools Programming Interface> + +To create a Funtools application, you need to include +the funtools.h definitions file in your code: + + #include <funtools.h> + + +You then call Funtools subroutines and functions to access Funtools data. +The most important routines are: + + +=over 4 + + + + + + +=item * + +FunOpen: open a Funtools file + + +=item * + +FunInfoGet: get info about an image or table + + +=item * + +FunImageGet: retrieve image data + + +=item * + +FunImageRowGet: retrieve image data by row + + +=item * + +FunImagePut: output image data + + +=item * + +FunImageRowPut: output image data by row + + +=item * + +FunColumnSelect: select columns in a table for access + + +=item * + +FunTableRowGet: retrieve rows from a table + + +=item * + +FunTableRowPut: output rows to a table + + +=item * + +FunClose: close a Funtools file + + +=back + + + +Your program must be linked against the libfuntools.a library, +along with the math library. The following libraries also might be required +on your system: + + +=over 4 + + + + +=item * + +-lsocket -lnsl for socket support + + +=item * + +-ldl for dynamic loading + + +=back + + + +For example, on a Solaris system using gcc, use the following link line: + + gcc -o foo foo.c -lfuntools -lsocket -lnsl -ldl -lm + +On a Solaris system using Solaris cc, use the following link line: + + gcc -o foo foo.c -lfuntools -lsocket -lnsl -lm + +On a Linux system using gcc, use the following link line: + + gcc -o foo foo.c -lfuntools -ldl -lm + +Once configure has built a Makefile on your platform, the required +"extra" libraries (aside from -lm, which always is required) are +specified in that file's EXTRA_LIBS variable. For example, under +Linux you will find: + + grep EXTRA_LIBS Makefile + EXTRA_LIBS = -ldl + ... + + + +The Funtools library contains both the zlib library +(http://www.gzip.org/zlib/) and Doug Mink's WCS library +(http://tdc-www.harvard.edu/software/wcstools/). It is not necessary +to put these libraries on a Funtools link line. Include files +necessary for using these libraries are installed in the Funtools +include directory. + +B<Funtools Programming Tutorial> + +The +FunOpen() +function is used to open a FITS file, an array, or a raw event file: + + /* open the input FITS file for reading */ + ifun = FunOpen(iname, "r", NULL); + /* open the output FITS file for writing, and connect it to the input file */ + ofun = FunOpen(iname, "w", ifun); + +A new output file can inherit header parameters automatically from +existing input file by passing the input Funtools handle as the last +argument to the new file's +FunOpen() +call as shown above. + + +For image data, you then can call +FunImageGet() +to read an image into memory. + + float buf=NULL; + /* extract and bin the data section into an image buffer */ + buf = FunImageGet(fun, NULL, "bitpix=-32"); + +If the (second) buf argument to this call is NULL, buffer space is allocated +automatically. The (third) plist argument can be used to specify the +return data type of the array. If NULL is specified, the data type of +the input file is used. + + +To process an image buffer, you would generally make a call to +FunInfoGet() to determine the +dimensions of the image (which may have been changed from the original +file dimensions due to Funtools image +sectioning on the command line). In a FITS image, the index along +the dim1 axis varies most rapidly, followed by the dim2 axis, etc. +Thus, to access each pixel in an 2D image, use a double loop such as: + + buf = FunImageGet(fun, NULL, "bitpix=-32"); + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + for(i=1; i<=dim2; i++){ + for(j=1; j<=dim1; j++){ + ... process buf[((i-1)*dim1)+(j-1)] ... + } + } + +or: + + buf = FunImageGet(fun, NULL, "bitpix=-32"); + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, 0); + for(i=0; i<(dim1*dim2); i++){ + ... process buf[i] ... + } + +Finally, you can write the resulting image to disk using +FunImagePut(): + + FunImagePut(fun2, buf, dim1, dim2, -32, NULL); + +Note that Funtools automatically takes care of book-keeping tasks such as +reading and writing FITS headers (although you can, of course, write +your own header or add your own parameters to a header). + + +For binary tables and raw event files, a call to +FunOpen() +will be followed by a call to the +FunColumnSelect() +routine to select columns to be read from the input file and/or +written to the output file: + + + typedef struct evstruct{ + double time; + int time2; + } *Ev, EvRec; + FunColumnSelect(fun, sizeof(EvRec), NULL, + "time", "D", "rw", FUN_OFFSET(Ev, time), + "time2", "J", "w", FUN_OFFSET(Ev, time2), + NULL); + +Columns whose (third) mode argument contains an "r" are "readable", +i.e., columns will be read from the input file and converted into the +data type specified in the call's second argument. These columns +values then are stored in the specified offset of the user record +structure. Columns whose mode argument contains a "w" are +"writable", i.e., these values will be written to the output file. +The +FunColumnSelect() +routine also offers the option of automatically merging user +columns with the original input columns when writing the output +rows. + + +Once a set of columns has been specified, you can retrieve rows using +FunTableRowGet(), +and write the rows using +FunTableRowPut(): + + Ev ebuf, ev; + /* get rows -- let routine allocate the array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + /* time2 is generated here */ + ev->time2 = (int)(ev->time+.5); + /* change the input time as well */ + ev->time = -(ev->time/10.0); + } + /* write out this batch of rows with the new column */ + FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL); + /* free row data */ + if( ebuf ) free(ebuf); + } + +The input rows are retrieved into an array of user structs, which +can be accessed serially as shown above. Once again, Funtools +automatically takes care of book-keeping tasks such as reading and writing +FITS headers (although you can, of course, write your own header or +add your own parameters to a header). + + +When all processing is done, you can call +FunClose() +to close the file(s): + + FunClose(fun2); + FunClose(fun); + + + +These are the basics of processing FITS files (and arrays or raw event +data) using Funtools. The routines in these examples are described in +more detail below, along with a few other routines that support +parameter access, data flushing, etc. + +B<Compiling and Linking> + +To create a Funtools application, a software developer will include +the funtools.h definitions file in Funtools code: + + #include <funtools.h> + +The program is linked against the libfuntools.a library, along with the +math library (and the dynamic load library, if the latter is available +on your system): + + gcc -o foo foo.c -lfuntools -ldl -lm + + +If gcc is used, Funtools filtering can be performed using dynamically +loaded shared objects that are built at run-time. Otherwise, filtering +is performed using a slave process. + +Funtools has been built on the following systems: + + +=over 4 + + + + +=item * + +Sun/Solaris 5.X + + +=item * + +Linux/RedHat Linux 5.X,6.X,7.X + + +=item * + +Dec Alpha/OSF1 V4.X + + +=item * + +WindowsNT/Cygwin 1.0 + + +=item * + +SGI/IRIX64 6.5 + + +=back + + + +B<A Short Digression on Subroutine Order> + +There is a natural order for all I/O access libraries. You would not +think of reading a file without first opening it, or writing a file +after closing it. A large part of the experiment in funtools is to use +the idea of "natural order" as a means of making programming +easier. We do this by maintaining the state of processing for a given +funtools file, so that we can do things like write headers and flush +extension padding at the right time, without you having to do it. + + +For example, if you open a new funtools file for writing using +FunOpen(), +then generate an array of image data and call +FunImagePut(), +funtools knows to write the image header automatically. +There is no need to think about writing a standard header. +Of course, you can add parameters to the file first by +calling one of the +FunParamPut() +routines, and these parameters will automatically be added +to the header when it is written out. There still is no +need to write the header explicitly. + + +Maintaining state in this way means that there are certain rules of +order which should be maintained in any funtools program. In particular, +we strongly recommend the following ordering rules be adhered to: + + + +=over 4 + + + + +=item * + +When specifying that input extensions be copied to an output file +via a reference handle, open the output file B<before> reading the +input file. (Otherwise the initial copy will not occur). + + + +=item * + +Always write parameters to an output file using one of the +FunParamPut() calls +B<before> writing any data. (This is a good idea for all FITS +libraries, to avoid having to recopy data is the FITS header needs +to be extended by adding a single parameter.) + + + +=item * + +If you retrieve an image, and need to know the data +type, use the FUN_SECT_BITPIX option of +FunInfoGet(), +B<after> calling +FunImageGet(), since +it is possible to change the value of BITPIX from the latter. + + + +=item * + +When specifying that input extensions be copied to an output file +via a reference handle, close the output file B<before> closing +input file, or else use +FunFlush() +explicitly on the output file +B<before> closing the input file. (Otherwise the final copy will +not occur). + + +=back + + + + +We believe that these are the natural rules that are implied in most +FITS programming tasks. However, we recognize that making explicit use +of "natural order" to decide what automatic action to take on behalf +of the programmer is experimental. Therefore, if you find that your +needs are not compatible with our preferred order, please let us know +-- it will be most illuminating for us as we evaluate this experiment. + +B<Funtools Programming Examples> + +The following complete coding examples are provided to illustrate the +simplicity of Funtools applications. They can be found in the funtest +subdirectory of the Funtools distribution. In many cases, you should +be able to modify one of these programs to generate your own Funtools +program: + + +=over 4 + + + + +=item * + +evread.c: read and write binary tables + + +=item * + +evcols.c: add column and rows to binary tables + + +=item * + +evmerge.c: merge new columns with existing columns + + +=item * + +evnext.c: manipulate raw data pointers + + +=item * + +imblank.c: blank out image values below a threshold + + +=item * + +asc2fits.c: convert a specific ASCII table to FITS binary table + + +=back + + + +B<The Funtools Programming Reference Manual> + + +#include <funtools.h> + +Fun FunOpen(char *name, char *mode, Fun ref) + +void *FunImageGet(Fun fun, void *buf, char *plist) + +int FunImagePut(Fun fun, void *buf, int dim1, int dim2, int bitpix, char *plist) + +void * FunImageRowGet(Fun fun, void *buf, int rstart, int rstop, char *plist) + +void * FunImageRowPut(Fun fun, void *buf, int rstart, int rstop, int dim1, int dim2, int bitpix, char *plist) + +int FunColumnSelect(Fun fun, int size, char *plist, ...) + +void FunColumnActivate(Fun fun, char *s, char *plist) + +int FunColumnLookup(Fun fun, char *s, int which, char **name, int *type, int *mode, int *offset, int *n, int *width) + +void *FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, int *nrow) + +int FunTableRowPut(Fun fun, void *rows, int nev, int idx, char *plist) + +int FunParamGetb(Fun fun, char *name, int n, int defval, int *got) + +int FunParamGeti(Fun fun, char *name, int n, int defval, int *got) + +double FunParamGetd(Fun fun, char *name, int n, double defval, int *got) + +char *FunParamGets(Fun fun, char *name, int n, char *defval, int *got) + +int FunParamPutb(Fun fun, char *name, int n, int value, char *comm, int append) + +int FunParamPuti(Fun fun, char *name, int n, int value, char *comm, int append) + +int FunParamPutd(Fun fun, char *name, int n, double value, int prec, char *comm, int append) + +int FunParamPuts(Fun fun, char *name, int n, char *value, char *comm, int append) + +int FunInfoGet(Fun fun, int type, ...) + +int FunInfoPut(Fun fun, int type, ...) + +void FunFlush(Fun fun, char *plist) + +void FunClose(Fun fun) + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funmerge.pod b/funtools/doc/pod/funmerge.pod new file mode 100644 index 0000000..0cb9216 --- /dev/null +++ b/funtools/doc/pod/funmerge.pod @@ -0,0 +1,122 @@ +=pod + +=head1 NAME + + + +B<funmerge - merge one or more Funtools table files> + + + +=head1 SYNOPSIS + + + + + +funmerge [-w|-x] -f [colname] <iname1> <iname2> ... <oname> + + + + + +=head1 OPTIONS + + + + + + -f # output a column specifying file from which this event came + -w # adjust position values using WCS info + -x # adjust position values using WCS info and save old values + + + + +=head1 DESCRIPTION + + + + +B<funmerge> merges FITS data from one or more +FITS Binary Table files +or raw event files. + +The first argument to the program specifies the first input FITS table +or raw event file. If "stdin" is specified, data are read from the +standard input. Use Funtools Bracket +Notation to specify FITS extensions and row filters. Subsequent +arguments specify additional event files and tables to merge. (NB: Stdin +cannot not be used for any of these additional input file arguments.) +The last argument is the output FITS file. The columns in each input table +must be identical. + + +If an input file begins with the '@' character, it is processed as an +include file, i.e., as a text file containing event file names (as +well as blank lines and/or comment lines starting with the '#' sign). +If standard input is specified as an include file ('@stdin'), then +file names are read from the standard input until EOF (^D). Event +files and include files can be mixed on a command line. + + +Rows from each table are written sequentially to the output +file. If the switch B<-f [colname]> is specified on the command +line, an additional column is added to each row containing the number +of the file from which that row was taken (starting from one). In +this case, the corresponding file names are stored in the header +parameters having the prefix B<FUNFIL>, i.e., FUNFIL01, +FUNFIL02, etc. + + +Using the B<-w> switch (or B<-x> switch as described +below), B<funmerge> also can adjust the position column values +using the WCS information in each file. (By position columns, we mean +the columns that the table is binned on, i.e., those columns defined +by the B<bincols=> switch, or (X,Y) by default.) To perform WCS +alignment, the WCS of the first file is taken as the base WCS. Each +position in subsequent files is adjusted by first converting it to the +sky coordinate in its own WCS coordinate system, then by converting +this sky position to the sky position of the base WCS, and finally +converting back to a pixel position in the base system. Note that in +order to perform WCS alignment, the appropriate WCS and TLMIN/TLMAX +keywords must already exist in each FITS file. + +When performing WCS alignment, you can save the original positions in +the output file by using the B<-x> (for "xtra") switch instead +of the B<-w> switch (i.e., using this switch also implies using +B<-w>) The old positions are saved in columns having the same +name as the original positional columns, with the added prefix "OLD_". + +Examples: + + +Merge two tables, and preserve the originating file number for +each row in the column called "FILE" (along with the corresponding +file name in the header): + + [sh] funmerge -f "FILE" test.ev test2.ev merge.ev + + + +Merge two tables with WCS alignment, saving the old position values in +2 additional columns: + + [sh] funmerge -x test.ev test2.ev merge.ev + + + +This program only works on raw event files and binary tables. We have +not yet implemented image and array merging. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funopen.pod b/funtools/doc/pod/funopen.pod new file mode 100644 index 0000000..3d6ab3e --- /dev/null +++ b/funtools/doc/pod/funopen.pod @@ -0,0 +1,166 @@ +=pod + +=head1 NAME + + + +B<FunOpen - open a Funtools data file> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + Fun FunOpen(char *name, char *mode, Fun ref); + + + + + +=head1 DESCRIPTION + + + + +The B<FunOpen()> routine opens a Funtools data file for reading or +appending, or creates a new FITS file for writing. The B<name> +argument specifies the name of the Funtools data file to open. You can +use IRAF-style bracket notation to specify +Funtools Files, Extensions, and Filters. +A separate call should be made each time a different FITS extension is +accessed: + + Fun fun; + char *iname; + ... + if( !(fun = FunOpen(iname, "r", NULL)) ){ + fprintf(stderr, "could not FunOpen input file: %s\n", iname); + exit(1); + } + + +If B<mode> is "r", the file is opened for reading, and processing +is set up to begin at the specified extension. For reading, +B<name> can be B<stdin>, in which case the standard input is read. + + +If B<mode> is "w", the file is created if it does not exist, or +opened and truncated for writing if it does exist. Processing starts +at the beginning of the file. The B<name> can be B<stdout>, +in which case the standard output is readied for processing. + + +If B<mode> is "a", the file is created if it does not exist, or +opened if it does exist. Processing starts at the end of the file. +The B<name> can be B<stdout>, in which case the standard +output is readied for processing. + + +When a Funtools file is opened for writing or appending, a previously +opened Funtools reference +handle can be specified as the third argument. This handle +typically is associated with the input Funtools file that will be used +to generate the data for the output data. When a reference file is +specified in this way, the output file will inherit the (extension) +header parameters from the input file: + + Fun fun, fun2; + ... + /* open input file */ + if( !(fun = FunOpen(argv[1], "r", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + /* open the output FITS image, inheriting params from input */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + +Thus, in the above example, the output FITS binary table file will +inherit all of the parameters associated with the input binary table +extension. + +A file opened for writing with a +Funtools reference handle also +inherits the selected columns (i.e. those columns chosen for +processing using the +FunColumnSelect() routine) +from the reference file as its default columns. This makes it easy to +open an output file in such a way that the columns written to the +output file are the same as the columns read in the input file. Of +course, column selection can easily be tailored using the +FunColumnSelect() routine. +In particular, it is easy to merge user-defined columns with the input +columns to generate a new file. See the +evmerge for a complete example. + + +In addition, when a +Funtools reference handle +is supplied in a FunOpen() call, +it is possible also to specify that all other extensions from the +reference file (other than the input extension being processed) should +be copied from the reference file to the output file. This is useful, +for example, in a case where you are processing a FITS binary table +or image and you want to copy all of the other extensions to +the output file as well. Copy of other extensions is controlled by +adding a "C" or "c" to the mode string of the +FunOpen() call of the input +reference file. If "C" is specified, then other extensions are +B<always> copied (i.e., copy is forced by the application). If +"c" is used, then other extensions are copied if the user requests +copying by adding a plus sign "+" to the extension name in the bracket +specification. For example, the B<funtable> program utilizes +"c" mode, giving users the option of copying all other extensions: + + /* open input file -- allow user copy of other extensions */ + if( !(fun = FunOpen(argv[1], "rc", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + /* open the output FITS image, inheriting params from input */ + if( !(fun2 = FunOpen(argv[2], "w", fun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + + +Thus, B<funtable> supports either of these command lines: + + # copy only the EVENTS extension + csh> funtable "test.ev[EVENTS,circle(512,512,10)]" foo.ev + # copy ALL extensions + csh> funtable "test.ev[EVENTS+,circle(512,512,10)]" foo.ev + + + +Use of a Funtools reference +handle implies that the input file is opened before the output +file. However, it is important to note that if copy mode ("c" or "C") +is specified for the input file, the actual input file open is delayed +until just after the output file is opened, since the copy of prior +extensions to the output file takes place while Funtools is seeking to +the specified input extension. This implies that the output file +should be opened before any I/O is done on the input file or else the +copy will fail. Note also that the copy of subsequent extension will +be handled automatically by +FunClose() +if the output file is +closed before the input file. Alternatively, it can be done explicitly +by FunFlush(), but again, this +assumes that the input file still is open. + + +Upon success FunOpen() returns a +Fun handle that is used in subsequent Funtools calls. On error, NULL +is returned. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funparamget.pod b/funtools/doc/pod/funparamget.pod new file mode 100644 index 0000000..6d06d10 --- /dev/null +++ b/funtools/doc/pod/funparamget.pod @@ -0,0 +1,153 @@ +=pod + +=head1 NAME + + + +B<FunParamGet - get a Funtools param value> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunParamGetb(Fun fun, char *name, int n, int defval, int *got) + + int FunParamGeti(Fun fun, char *name, int n, int defval, int *got) + + double FunParamGetd(Fun fun, char *name, int n, double defval, int *got) + + char *FunParamGets(Fun fun, char *name, int n, char *defval, int *got) + + + + + +=head1 DESCRIPTION + + + + +The four routines B<FunParamGetb()>, B<FunParamGeti()>, +B<FunParamGetd()>, and B<FunParamGets()>, return the value of +a FITS header parameter as a boolean, int, double, and string, +respectively. The string returned by B<FunParamGets()> is a malloc'ed +copy of the header value and should be freed when no longer needed. + + +The first argument is the Fun handle associated with the FITS header +being accessed. Normally, the header is associated with the FITS +extension that you opened with B<FunOpen()>. However, you can use +FunInfoPut() to specify access of the primary header. In particular, +if you set the FUN_PRIMARYHEADER parameter to 1, then the primary +header is used for all parameter access until the value is reset to +0. For example: + + int val; + FunParamGeti(fun, "NAXIS", 1, 0, &got); # current header + val=1; + FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0); # switch to ... + FunParamGeti(fun, "NAXIS", 1, 0, &got); # ... primary header + FunParamGeti(fun, "NAXIS", 2, 0, &got); # ... primary header + val=0; + FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0); # switch back to ... + FunParamGeti(fun, "NAXIS", 2, 0, &got); # current header + + + +Alternatively, you can use the FUN_PRIMARY macro to access parameters +from the primary header on a per-parameter basis: + + FunParamGeti(fun, "NAXIS1", 0, 0, &got); # current header + FunParamGeti(FUN_PRIMARY(fun), "NAXIS1", 0, 0, &got); # primary header + +B<NB: FUN_PRIMARY is deprecated.> +It makes use of a global parameter and therefore will not not +appropriate for threaded applications, when we make funtools +thread-safe. We recommend use of FunInfoPut() to switch between the +extension header and the primary header. + + +For output data, access to the primary header is only possible until +the header is written out, which usually takes place when the first +data are written. + + +The second argument is the name of the parameter to access. The third +B<n> argument, if non-zero, is an integer that will be added as a +suffix to the parameter name. This makes it easy to use a simple loop +to process parameters having the same root name. For example, to +gather up all values of TLMIN and TLMAX for each column in a binary +table, you can use: + + for(i=0, got=1; got; i++){ + fun->cols[i]->tlmin = (int)FunParamGeti(fun, "TLMIN", i+1, 0.0, &got); + fun->cols[i]->tlmax = (int)FunParamGeti(fun, "TLMAX", i+1, 0.0, &got); + } + + + +The fourth B<defval> argument is the default value to return if +the parameter does not exist. Note that the data type of this +parameter is different for each specific FunParamGet() call. The final +B<got> argument will be 0 if no param was found. Otherwise the +data type of the parameter is returned as follows: FUN_PAR_UNKNOWN +('u'), FUN_PAR_COMMENT ('c'), FUN_PAR_LOGICAL ('l'), FUN_PAR_INTEGER +('i'), FUN_PAR_STRING ('s'), FUN_PAR_REAL ('r'), FUN_PAR_COMPLEX ('x'). + + +These routines return the value of the header parameter, or the +specified default value if the header parameter does not exist. The +returned value is a malloc'ed string and should be freed when no +longer needed. + + +By default, B<FunParamGets()> returns the string value of the +named parameter. However, you can use FunInfoPut() to retrieve the +raw 80-character FITS card instead. In particular, if you set the +FUN_RAWPARAM parameter to 1, then card images will be returned by +FunParamGets() until the value is reset to 0. + + +Alternatively, if the FUN_RAW macro is applied to the name, then the +80-character raw FITS card is returned instead. +B<NB: FUN_RAW is deprecated.> +It makes use of a global parameter and therefore will not not +appropriate for threaded applications, when we make funtools +thread-safe. We recommend use of FunInfoPut() to switch between the +extension header and the primary header. + + +Note that in addition to the behaviors described above, the +routine B<FunParamGets()> will return the 80 raw characters of the +B<nth> FITS card (including the comment) if B<name> is specified as +NULL and B<n> is positive. For example, to loop through all FITS +header cards in a given extension and print out the raw card, use: + + for(i=1; ;i++){ + if( (s = FunParamGets(fun, NULL, i, NULL, &got)) ){ + fprintf(stdout, "%.80s\n", s); + free(s); + } + else{ + break; + } + } + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funparamput.pod b/funtools/doc/pod/funparamput.pod new file mode 100644 index 0000000..65d1b55 --- /dev/null +++ b/funtools/doc/pod/funparamput.pod @@ -0,0 +1,147 @@ +=pod + +=head1 NAME + + + +B<FunParamPut - put a Funtools param value> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + int FunParamPutb(Fun fun, char *name, int n, int value, char *comm, + int append) + + int FunParamPuti(Fun fun, char *name, int n, int value, char *comm, + int append) + + int FunParamPutd(Fun fun, char *name, int n, double value, int prec, + char *comm, int append) + + int FunParamPuts(Fun fun, char *name, int n, char *value, char *comm, + int append) + + + + + +=head1 DESCRIPTION + + + + +The four routines B<FunParamPutb()>, B<FunParamPuti()>, +B<FunParamPutd()>, and B<FunParamPuts()>, will set the value +of a FITS header parameter as a boolean, int, double, and string, +respectively. + + +The first argument is the Fun handle associated with the FITS header +being accessed. Normally, the header is associated with the FITS +extension that you opened with B<FunOpen()>. +However, you can use FunInfoPut() to specify that use of the primary +header. In particular, if you set the FUN_PRIMARYHEADER parameter to +1, then the primary header is used for all parameter access until the +value is reset to 0. For example: + + int val; + FunParamPuti(fun, "NAXIS1", 0, 10, NULL, 1); # current header + val=1; + FunInfoPut(fun, FUN_PRIMARYHEADER, &val, 0); # switch to ... + FunParamPuti(fun, "NAXIS1", 0, 10, NULL, 1); # primary header + +(You also can use the deprecated FUN_PRIMARY macro, to access +parameters from the primary header.) + + +The second argument is the B<name> of the parameter. ( +In accordance with FITS standards, the special names B<COMMENT> +and B<HISTORY>, as well as blank names, are output without the "= " +value indicator in columns 9 and 10. + + +The third B<n> argument, if non-zero, is an integer that will be +added as a suffix to the parameter name. This makes it easy to use a +simple loop to process parameters having the same root name. For +example, to set the values of TLMIN and TLMAX for each column in a +binary table, you can use: + + for(i=0; i<got; i++){ + FunParamPutd(fun, "TLMIN", i+1, tlmin[i], 7, "min column val", 1); + FunParamPutd(fun, "TLMAX", i+1, tlmax[i], 7, "max column val", 1); + } + + + +The fourth B<defval> argument is the value to set. Note that the +data type of this argument is different for each specific +FunParamPut() call. The B<comm> argument is the comment +string to add to this header parameter. Its value can be NULL. The +final B<append> argument determines whether the parameter is added +to the header if it does not exist. If set to a non-zero value, the +header parameter will be appended to the header if it does not exist. +If set to 0, the value will only be used to change an existing parameter. + + +Note that the double precision routine FunParamPutd() supports an +extra B<prec> argument after the B<value> argument, in order +to specify the precision when converting the double value to ASCII. In +general a 20.[prec] format is used (since 20 characters are alloted to +a floating point number in FITS) as follows: if the double value being +put to the header is less than 0.1 or greater than or equal to +10**(20-2-[prec]), then %20.[prec]e format is used (i.e., scientific +notation); otherwise %20.[prec]f format is used (i.e., numeric +notation). + + +As a rule, parameters should be set before writing the table or image. +It is, however, possible to update the value of an B<existing> +parameter after writing an image or table (but not to add a new +one). Such updating only works if the parameter already exists and if +the output file is seekable, i.e. if it is a disk file or is stdout +being redirected to a disk file. + + +It is possible to add a new parameter to a header after the data has +been written, but only if space has previously been reserved. To reserve +space, add a blank parameter whose value is the name of the parameter you +eventually will update. Then, when writing the new parameter, specify a +value of 2 for the append flag. The parameter writing routine will +first look to update an existing parameter, as usual. If an existing +parameter is not found, an appropriately-valued blank parameter will be +searched for and replaced. For example: + + /* add blank card to be used as a place holder for IPAR1 update */ + FunParamPuts(fun, NULL, 0, "IPAR1", "INTEGER Param", 0); + ... + /* write header and data */ + FunTableRowPut(fun, events, got, 0, NULL); + ... + /* update param in file after writing data -- note append = 2 here */ + FunParamPuti(fun, "IPAR", 1, 400, "INTEGER Param", 2); + + + +The parameter routines return a 1 if the routine was successful and a 0 on +failure. In general, the major reason for failure is that you did not +set the append argument to a non-zero value and the parameter did not +already exist in the file. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funref.pod b/funtools/doc/pod/funref.pod new file mode 100644 index 0000000..13b158f --- /dev/null +++ b/funtools/doc/pod/funref.pod @@ -0,0 +1,170 @@ +=pod + +=head1 NAME + + + +B<FunRef: the Funtools Reference Handle> + + + +=head1 SYNOPSIS + + + + +A description of how to use a Funtools reference handle to connect a +Funtools input file to an output file. + + + +=head1 DESCRIPTION + + + + + +The Funtools reference handle connects a Funtools input file to a +Funtools output file so that parameters (or even whole extensions) can +be copied from the one to the other. To make the connection, the Funtools +handle of the input file is passed to the +final argument of the +FunOpen() call for the output file: + + if( !(ifun = FunOpen(argv[1], "r", NULL)) ) + gerror(stderr, "could not FunOpen input file: %s\n", argv[1]); + if( !(ofun = FunOpen(argv[2], "w", ifun)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + +It does not matter what type of input or output file (or extension) is +opened, or whether they are the same type. When the output image or +binary table is written using +FunImagePut() +or +FunTableRowPut() +an appropriate header will be written first, with parameters copied +from the input extension. Of course, invalid parameters will be +removed first, e.g., if the input is a binary table and the output is +an image, then binary table parameters such as TFORM, TUNIT, +etc. parameters will not be copied to the output. + + +Use of a reference handle also allows default values to be passed +to +FunImagePut() in order to +write out an output image with the same dimensions and data type +as the input image. To use the defaults from the input, a value +of 0 is entered for dim1, dim2, and bitpix. For example: + + fun = FunOpen(argv[1], "r", NULL); + fun2 = FunOpen(argv[2], "w", fun); + buf = FunImageGet(fun, NULL, NULL); + ... process image data ... + FunImagePut(fun2, buf, 0, 0, 0, NULL); + +Of course, you often want to get information about the data type +and dimensions of the image for processing. The above code +is equivalent to the following: + + fun = FunOpen(argv[1], "r", NULL); + fun2 = FunOpen(argv[2], "w", fun); + buf = FunImageGet(fun, NULL, NULL); + FunInfoGet(fun, FUN_SECT_DIM1, &dim1, FUN_SECT_DIM2, &dim2, + FUN_SECT_BITPIX, &bitpix, 0); + ... process image data ... + FunImagePut(fun2, buf, dim1, dim2, bitpix, NULL); + + + +It is possible to change the reference handle for a given output Funtools +handle using the +FunInfoPut() routine: + + /* make the new extension the reference handle for the output file */ + FunInfoPut(fun2, FUN_IFUN, &fun, 0); + +When this is done, Funtools specially resets the output file to start +a new output extension, which is connected to the new input reference +handle. You can use this mechanism to process multiple input extensions +into a single output file, by successively opening the former and +setting the reference handle for the latter. For example: + + /* open a new output FITS file */ + if( !(fun2 = FunOpen(argv[2], "w", NULL)) ) + gerror(stderr, "could not FunOpen output file: %s\n", argv[2]); + /* process each input extension in turn */ + for(ext=0; ;ext++){ + /* get new extension name */ + sprintf(tbuf, "%s[%d]", argv[1], ext); + /* open it -- if we cannot open it, we are done */ + if( !(fun=FunOpen(tbuf, "r", NULL)) ) + break; + /* make the new extension the reference handle for the output file */ + FunInfoPut(fun2, FUN_IFUN, &fun, 0); + ... process ... + /* flush output extension (write padding, etc.) */ + FunFlush(fun2, NULL); + /* close the input extension */ + FunClose(fun); + } + +In this example, the output file is opened first. Then each successive +input extension is opened, and the output reference handle is set to +the newly opened input handle. After data processing is performed, the +output extension is flushed and the input extension is closed, in +preparation for the next input extension. + +Finally, a reference handle can be used to copy other extensions from +the input file to the output file. Copy of other extensions is +controlled by adding a "C" or "c" to the mode string of the +FunOpen() +call B<of the input reference file>. If "C" is specified, then +other extensions are B<always> copied (i.e., copy is forced by the +application). If "c" is used, then other extensions are copied if the +user requests copying by adding a plus sign "+" to the extension name +in the bracket specification. For example, the B<funtable> +program utilizes user-specified "c" mode so that the second example +below will copy all extensions: + + # copy only the EVENTS extension + csh> funtable "test.ev[EVENTS,circle(512,512,10)]" foo.ev + # copy ALL extensions + csh> funtable "test.ev[EVENTS+,circle(512,512,10)]" foo.ev + +When extension copy is specified in the input file, the call to +FunOpen() +on the input file delays the actual file open until the output file +also is opened (or until I/O is performed on the input file, which +ever happens first). Then, when the output file is opened, the input +file is also opened and input extensions are copied to the output +file, up to the specific extension being opened. Processing of input +and output extensions then proceed. + +When extension processing is complete, the remaining extensions need to +be copied from input to output. This can be done explicitly, using the +FunFlush() +call with the "copy=remaining" plist: + + FunFlush(fun, "copy=remaining"); + +Alternatively, this will happen automatically, if the output file +is closed B<before> the input file: + + /* we could explicitly flush remaining extensions that need copying */ + /* FunFlush(fun2, "copy=remaining"); */ + /* but if we close output before input, end flush is done automatically */ + FunClose(fun2); + FunClose(fun); + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funregions.pod b/funtools/doc/pod/funregions.pod new file mode 100644 index 0000000..7f975ad --- /dev/null +++ b/funtools/doc/pod/funregions.pod @@ -0,0 +1,571 @@ +=pod + +=head1 NAME + + + +B<Regions: Spatial Region Filtering> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the user interface for spatial +region filtering images and tables. + + + +=head1 DESCRIPTION + + + + + +Spatial region filtering allows a program to select regions of an +image or rows of a table (e.g., X-ray events) to process using +simple geometric shapes and boolean combinations of shapes. When an +image is filtered, only pixels found within these shapes are +processed. When a table is filtered, only rows found within these +shapes are processed. + + +Spatial region filtering for images and tables is accomplished by +means of B<region specifications>. A region specification +consists of one or more B<region expressions>, which are geometric +shapes,combined according to the rules of boolean algebra. Region +specifications also can contain comments and local/global processing +directives. + + +Typically, region specifications are specified using bracket notation +appended to the filename of the data being processed: + + foo.fits[circle(512,512,100)] + +It is also possible to put region specification inside a file and +then pass the filename in bracket notation: + + foo.fits[@my.reg] + + + +When region filters are passed in bracket notation in this manner, the +filtering is set up automatically when the file is opened and all +processing occurs through the filter. Programs also can use the filter +library API to open filters explicitly. + +B<Region Expressions> + +More specifically, region specifications consist of one or more lines +containing: + + # comment until end of line + global keyword=value keyword=value ... # set global value(s) + # include the following file in the region descriptor + @file + # use the FITS image as a mask (cannot be used with other regions) + @fitsimage + # each region expression contains shapes separated by operators + [region_expression1], [region_expression2], ... + [region_expression], [region_expression], ... + + + +A single region expression consists of: + + # parens and commas are optional, as is the + sign + [+-]shape(num , num , ...) OP1 shape num num num OP2 shape ... + +e.g.: + + ([+-]shape(num , num , ...) && shape num num || shape(num, num) + # a comment can come after a region -- reserved for local properties + [+-]shape(num , num , ...) # local properties go here, e.g. color=red + + + +Thus, a region descriptor consists of one or more region +expressions or B<regions>, separated by comas, new-lines, or +semi-colons. Each B<region> consists of one or more geometric +shapes combined using standard boolean operation. Several types +of shapes are supported, including: + + + shape: arguments: + ----- ---------------------------------------- + ANNULUS xcenter ycenter inner_radius outer_radius + BOX xcenter ycenter xwidth yheight (angle) + CIRCLE xcenter ycenter radius + ELLIPSE xcenter ycenter xwidth yheight (angle) + FIELD none + LINE x1 y1 x2 y2 + PIE xcenter ycenter angle1 angle2 + POINT x1 y1 + POLYGON x1 y1 x2 y2 ... xn yn + + + +In addition, the following regions accept B<accelerator> syntax: + + + shape arguments + ----- ------------------------------------------ + ANNULUS xcenter ycenter radius1 radius2 ... radiusn + ANNULUS xcenter ycenter inner_radius outer_radius n=[number] + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + BOX xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + ELLIPSE xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + PIE xcenter ycenter angle1 angle2 (angle3) (angle4) (angle5) ... + PIE xcenter ycenter angle1 angle2 (n=[number]) + POINT x1 y1 x2 y2 ... xn yn + +Note that the circle accelerators are simply aliases for the annulus +accelerators. See region geometry +for more information about accelerators. + + +Finally, the following are combinations of pie with different shapes +(called "panda" for "Pie AND Annulus") allow for easy specification of +radial sections: + + + shape: arguments: + ----- --------- + PANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + BPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # box + EPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # ellipse + + +The panda and cpanda specify combinations of annulus and circle with pie, +respectively and give identical results. The bpanda combines box and pie, +while epanda combines ellipse and pie. +See region geometry +for more information about pandas. + + +The following "shapes" are ignored by funtools (generated by ds9): + + shape: arguments: + ----- --------- + PROJECTION x1 y1 x2 y2 width # NB: ignored by funtools + RULER x1 y1 x2 y2 # NB: ignored by funtools + TEXT x y # NB: ignored by funtools + GRID # NB: ignored by funtools + TILE # NB: ignored by funtools + COMPASS # NB: ignored by funtools + + + +All arguments to regions are real values; integer values are +automatically converted to real where necessary. All angles are in +degrees and run from the positive image x-axis to the positive image +y-axis. If a rotation angle is part of the associated WCS header, that +angle is added implicitly as well. + + +Note that 3-letter abbreviations are supported for all shapes, so that +you can specify "circle" or "cir". + + +B<Columns Used in Region Filtering> + +By default, the x,y values in a region expression refer to the two +"image binning" columns, i.e. the columns that would be used to +bin the data into an image. For images, these are just the 2 dimensions +of the image. For tables, these usually default to x and y but +can be changed as required. For example, in Funtools, new binning +columns are specified using a bincols=(col1,col2) statement within +the bracket string on the command line. + +Alternate columns for region filtering can be specified by the syntax: + + (col1,col2)=region(...) + +e.g.: + + (X,Y)=annulus(x,y,ri,ro) + (PHA,PI)=circle(x,y,r) + (DX,DY)=ellipse(x,y,a,b[,angle]) + + + +B<Region Algebra> + +(See also Region Algebra for more complete +information.) + + +Region shapes can be combined together using Boolean operators: + + Symbol Operation Use + -------- --------- ----------------------------------- + ! not Exclude this shape from this region + & or && and Include only the overlap of these shapes + | or || inclusive or Include all of both shapes + ^ exclusive or Include both shapes except their overlap + +Note that the !region syntax must be combined with another region in order +that we be able to assign a region id properly. That is, + + !circle(512,512,10) + +is not a legal region because there is no valid region id to work with. +To get the full field without a circle, combine the above with field(), +as in: + + field() && !circle(512,512,10) + + +B< Region Separators Also Are Operators> + + +As mentioned previously, multiple region expressions can be specified +in a region descriptor, separated by commas, new-lines, or +semi-colons. When such a separator is used, the boolean OR operator +is automatically generated in its place but, unlike explicit use of +the OR operator, the region ID is incremented (starting from 1). + + +For example, the two shapes specified in this example are given the +same region value: + + foo.fits[circle(512,512,10)||circle(400,400,20)] + +On the other hand, the two shapes defined in the following example are +given different region values: + + foo.fits[circle(512,512,10),circle(400,400,20)] + + + +Of course these two examples will both mask the same table rows or +pixels. However, in programs that distinguish region id's (such as +funcnts ), they will act +differently. The explicit OR operator will result in one region +expression consisting of two shapes having the same region id and +funcnts will report a single region. The comma operator will cause +funcnts to report two region expressions, each with one shape, in +its output. + + +In general, commas are used to separate region expressions entered +in bracket notation on the command line: + + # regions are added to the filename in bracket notation + foo.fits[circle(512,512,100),circle(400,400,20)] + +New-lines are used to separate region +expressions in a file: + + # regions usually are separated by new-lines in a file + # use @filename to include this file on the command line + circle(512,512,100) + circle(400,400,20) + +Semi-colons are provided for backward compatibility with the original +IRAF/PROS implementation and can be used in either case. + + +If a pixel is covered by two different regions expressions, it is +given the mask value of the B<first> region that contains that +pixel. That is, successive regions B<do not> overwrite previous +regions in the mask, as was the case with the original PROS regions. +In this way, an individual pixel is covered by one and only one +region. This means that one must sometimes be careful about the order +in which regions are defined. If region N is fully contained within +region M, then N should be defined B<before> M, or else it will be +"covered up" by the latter. + +B<Region Exclusion> + +Shapes also can be globally excluded from all the region specifiers in +a region descriptor by using a minus sign before a region: + + + operator arguments: + -------- ----------- + - Globally exclude the region expression following '-' sign + from ALL regions specified in this file + +The global exclude region can be used by itself; in such a case, field() is +implied. + + +A global exclude differs from the local exclude (i.e. a shape prefixed +by the logical not "!" symbol) in that global excludes are logically +performed last, so that no region will contain pixels from a globally +excluded shape. A local exclude is used in a boolean expression with +an include shape, and only excludes pixels from that include shape. +Global excludes cannot be used in boolean expressions. + +B<Include Files> + + +The B<@filename> directive specifies an include file +containing region expressions. This file is processed as part of +the overall region descriptor: + + foo.fits[circle(512,512,10),@foo] + +A filter include file simply includes text without changing the state +of the filter. It therefore can be used in expression. That is, if the +file foo1 contains "pi==1" and foo2 contains "pha==2" then +the following expressions are equivalent: + + "[@foo1&&@foo2]" is equivalent to "[pi==1&&pha==2]" + "[pha==1||@foo2]" is equivalent to "[pi==1||pha==2]" + "[@foo1,@foo2]" is equivalent to "[pi==1,pha==2]" + +Be careful that you specify evaluation order properly using +parenthesis, especially if the include file contains multiple +filter statements. For example, consider a file containing two +regions such as: + + circle 512 512 10 + circle 520 520 10 + +If you want to include only events (or pixels) that are in these regions +and have a pi value of 4, then the correct syntax is: + + pi==4&&(@foo) + +since this is equivalent to: + + pi==4 && (circle 512 512 10 || circle 520 520 10) + +If you leave out the parenthesis, you are filtering this statement: + + pi==4 && circle 512 512 10 || circle 520 520 10) + +which is equivalent to: + + (pi==4 && circle 512 512 10) || circle 520 520 10) + +The latter syntax only applies the pi test to the first region. + + +For image-style filtering, the B<@filename> can specify an 8-bit +or 16-bit FITS image. In this case, the pixel values in the mask image +are used as the region mask. The valid pixels in the mask must have +positive values. Zero values are excluded from the mask and negative +values are not allowed. Moreover, the region id value is taken as +the image pixel value and the total number of regions is taken to be +the highest pixel value. The dimensions of the image mask must be less +than or equal to the image dimensions of the data. The mask will be +replicated as needed to match the size of the image. (Thus, best +results are obtained when the data dimensions are an even multiple of +the mask dimensions.) + + +An image mask can be used in any image filtering operation, regardless +of whether the data is of type image or table. For example, the +funcnts ) +program performs image filtering on images or tables, and so +FITS image masks are valid input for either type of data in this +program.. An image mask cannot be used in a program such as +fundisp ) +when the input data is a table, because fundisp displays +rows of a table and processes these rows using event-style filtering. + +B<Global and Local Properties of Regions> + + +The ds9 image display program describes a host of properties such as +color, font, fix/free state, etc. Such properties can be specified +globally (for all regions) or locally (for an individual region). +The B<global> keyword specifies properties and qualifiers for all +regions, while local properties are specified in comments on the same +line as the region: + + global color=red + circle(10,10,2) + circle(20,20,3) # color=blue + circle(30,30,4) + +The first and third circles will be red, which the second circle will +be blue. Note that funtools currently ignores region properties, as +they are used in display only. + +B< Coordinate Systems> + +For each region, it is important to specify the coordinate system +used to interpret the region, i.e., to set the context in which position and +size values are interpreted. For this purpose, the following keywords +are recognized: + + + name description + ---- ------------------------------------------ + PHYSICAL pixel coords of original file using LTM/LTV + IMAGE pixel coords of current file + FK4, B1950 sky coordinate systems + FK5, J2000 sky coordinate systems + GALACTIC sky coordinate systems + ECLIPTIC sky coordinate systems + ICRS currently same as J2000 + LINEAR linear wcs as defined in file + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV + + + +B<Specifying Positions, Sizes, and Angles> + +The arguments to region shapes can be floats or integers describing +positions and sizes. They can be specified as pure numbers or using +explicit formatting directives: + + + position arguments description + ------------------ ------------------------------ + [num] context-dependent (see below) + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + [num]:[num]:[num] hms for 'odd' position arguments + [num]:[num]:[num] dms for 'even' position arguments + [num]h[num]m[num]s explicit hms + [num]d[num]m[num]s explicit dms + + size arguments description + -------------- ----------- + [num] context-dependent (see below) + [num]" arc seconds + [num]' arc minutes + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + + + +When a "pure number" (i.e. one without a format directive such as 'd' +for 'degrees') is specified, its interpretation depends on the context +defined by the 'coordsys' keyword. In general, the rule is: + + +All pure numbers have implied units corresponding to the current +coordinate system. + + +If no such system is explicitly specified, the default system is +implicitly assumed to be PHYSICAL. + + +In practice this means that for IMAGE and PHYSICAL systems, pure +numbers are pixels. Otherwise, for all systems other than linear, +pure numbers are degrees. For LINEAR systems, pure numbers are in the +units of the linear system. This rule covers both positions and +sizes. + + +The input values to each shape can be specified in several coordinate +systems including: + + + name description + ---- ---------------------------- + IMAGE pixel coords of current file + LINEAR linear wcs as defined in file + FK4, B1950 various sky coordinate systems + FK5, J2000 + GALACTIC + ECLIPTIC + ICRS + PHYSICAL pixel coords of original file using LTM/LTV + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV + + + +If no coordinate system is specified, PHYSICAL is assumed. PHYSICAL or +a World Coordinate System such as J2000 is preferred and most general. +The coordinate system specifier should appear at the beginning of the +region description, on a separate line (in a file), or followed by a +new-line or semicolon; e.g., + + + global coordsys physical + circle 6500 9320 200 + + +The use of celestial input units automatically implies WORLD +coordinates of the reference image. Thus, if the world coordinate +system of the reference image is J2000, then + + + circle 10:10:0 20:22:0 3' + + +is equivalent to: + + + circle 10:10:0 20:22:0 3' # j2000 + + + +Note that by using units as described above, you may mix coordinate +systems within a region specifier; e.g., + + + circle 6500 9320 3' # physical + + + +Note that, for regions which accept a rotation angle: + + +ellipse (x, y, r1, r2, angle) +box(x, y, w, h, angle) + + +the angle is relative to the specified coordinate system. In +particular, if the region is specified in WCS coordinates, the angle +is related to the WCS system, not x/y image coordinate axis. For WCS +systems with no rotation, this obviously is not an issue. However, +some images do define an implicit rotation (e.g., by using a non-zero +CROTA value in the WCS parameters) and for these images, the angle +will be relative to the WCS axes. In such case, a region specification +such as: + + +fk4;ellipse(22:59:43.985, +58:45:26.92,320", 160", 30) + + +will not, in general, be the same region specified as: + + +physical;ellipse(465, 578, 40, 20, 30) + + +even when positions and sizes match. The angle is relative to WCS axes +in the first case, and relative to physical x,y axes in the second. + + + +More detailed descriptions are available for: +Region Geometry, +Region Algebra, +Region Coordinates, and +Region Boundaries. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funsky.pod b/funtools/doc/pod/funsky.pod new file mode 100644 index 0000000..421d5d7 --- /dev/null +++ b/funtools/doc/pod/funsky.pod @@ -0,0 +1,260 @@ +=pod + +=head1 NAME + + + +B<funsky - convert between image and sky coordinates> + + + +=head1 SYNOPSIS + + + + + + funsky iname[ext] # RA,Dec (deg) or image pix from stdin + funsky iname[ext] [lname] # RA, Dec (deg) or image pix from list + funsky iname[ext] [col1] [col2] # named cols:units from stdin + funsky iname[ext] [lname] [col1] [col2] # named cols:units from list + + + + + +=head1 OPTIONS + + + + + + -d # always use integer tlmin conversion (as ds9 does) + -r # convert x,y to RA,Dec (default: convert RA,Dec to x,y) + -o # include offset from the nominal target position (in arcsec) + -v # display input values also (default: display output only) + -T # output display in rdb format (w/header,tab delimiters) + + + + +=head1 DESCRIPTION + + + + +Funsky converts input sky coordinates (RA, Dec) to image coordinates (or vice +versa) using the WCS information contained in the specified FITS file. Several +calling sequences are supported in order to make it easy to specify +coordinate positions in different ways. + + +The first required argument is always the input FITS file (or +extension) containing the WCS information in an extension header. Note +that the data from this file is not used. By default, the program +converts input RA and Dec values to X and Y using this WCS +information. If the WCS is associated with a FITS image, then the X,Y +values are image values. If the WCS is associated with a binary table, +then the X, Y values are physical values. To convert X,Y to RA and +Dec, use the B<-r> (reverse) switch. + + +If no other command arguments are supplied, then the input positions +are read from the standard input. Each line is assumed to contain a +single coordinate position consisting of an RA in degrees (or X in +pixels) followed by a Dec in degrees (or Y in pixels). The usual +delimiters are supported (spaces, commas, tabs). For example: + + # read from stdin, default column names and units + [sh] funsky snr.ev + 22.982695 58.606523 # input RA (hrs), Dec(deg) + 510.00 510.00 + 22.982127 58.607634 # input + 512.00 510.50 + 22.981700 58.614301 # input + 513.50 513.50 + ^D # end of input + + + +If a second argument is supplied, this argument is assumed to be +a file containing RA (X) and Dec (Y) positions. The file can either be +an ASCII table or a FITS binary table. The order of columns is +unimportant, if the table has a column header. In this case, the +names of the columns must be one of "RA", "DEC", or "X", "Y" for sky +to image and image to sky conversions, respectively. If the table has +no header, then once again, RA (X) is assumed to first, followed +by DEC (Y). +For example: + + # read from file, default column names and units + [sh] cat hd.in + RA DEC + --------- --------- + 22.982695 58.606523 + 22.982127 58.607634 + 22.981700 58.614301 + + [sh] funsky snr.ev hd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + + +If three arguments are supplied, then the input positions again are +read from the standard input. Each line is assumed to contain a single +coordinate position consisting of an RA (or X in pixels) followed by a +Dec (or Y in pixels), with the usual delimiters supported. However, +the second and third arguments now specify the column names and/or +sky units using a colon-delimited syntax: + + [colname]:[h|d|r] + +If the colname is omitted, the names default to "RA", "DEC", "X", "Y", +"COL1", or "COL2" as above. If the units are omitted, the default is degrees +for both RA and Dec. When the -r switch is used (convert from image +to sky) the units are applied to the output instead of the input. The following +examples will serve to illustrate the options: + + # read from stdin, specifying column names (def. units: degrees) + [sh] cat hd.in + MYRA MYDEC + --------- --------- + 22.982695 58.606523 + 22.982127 58.607634 + 22.981700 58.614301 + + [sh] funsky snr.ev MYRA MYDEC < hd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read from stdin, specifying column names and units + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev MYRA:d MYDEC:d < dd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read stdin, convert image to sky, specifying output sky units + [sh] cat im.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] cat im.in | funsky -r snr.ev :d :d + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + + +Finally, four command arguments specify both and input file and column names +and/or units: + + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev dd.in MYRA:d MYDEC:d + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read file, convert image to sky, specifying output sky units + [sh] cat im.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] funsky -r snr.ev im.in :d :d + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + + +By default, the output of funsky consists only of the converted coordinate +position(s), one per output line. This makes parsing in shell scripts easy. +Use the B<-v> (verbose) switch to specify that the input +coordinates should be pre-pended to each line. For example: + + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev dd.in MYRA:d MYDEC:d + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] funsky -v snr.ev dd.in MYRA:d MYDEC:d + 344.740432 58.606523 510.00 510.00 + 344.731900 58.607634 512.00 510.50 + 344.725500 58.614301 513.50 513.50 + + + +In addition, a full starbase table can be output using the B<-T> +(table) switch. This switch can be used with or without the -v +switch. If the -T and -v are both specified, then a descriptive header +parameters are output before the table (mainly to remind you of the +sky units): + + # output table in non-verbose mode + [sh] funsky -T snr.ev dd.in MYRA:d MYDEC:d + X Y + ------------ ------------ + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # output table in verbose mode + [sh] funsky -T -v snr.ev dd.in MYRA:d MYDEC:d + # IFILE = /Users/eric/data/snr.ev + # ICOL1 = MYRA + # ICOL2 = MYDEC + # IUNITS1 = d + # IUNITS2 = d + # OCOL1 = X + # OCOL2 = Y + + MYRA MYDEC X Y + ------------ ------------ ------------ ------------ + 344.740432 58.606523 510.00 510.00 + 344.731900 58.607634 512.00 510.50 + 344.725500 58.614301 513.50 513.50 + + + +Finally, the B<-d> (ds9) switch mimicks ds9's use of integer TLMIN +and TLMAX values for all coordinate transformations. FITS conventions +seem to call for use of floating point TLMIN and TLMAX when the data are +floats. This convention is followed by funsky but results in a +small discrepancy with ds9's converted values for floating point +data. We will remedy this conflict in the future, maybe. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funtable.pod b/funtools/doc/pod/funtable.pod new file mode 100644 index 0000000..d4e8475 --- /dev/null +++ b/funtools/doc/pod/funtable.pod @@ -0,0 +1,296 @@ +=pod + +=head1 NAME + + + +B<funtable - copy selected rows from a Funtools file to a FITS binary table> + + + +=head1 SYNOPSIS + + + + + +funtable [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns] + + + + + +=head1 OPTIONS + + + + + + -a # append to existing output file as a table extension + -i # for image data, only generate X and Y columns + -m # for tables, write a separate file for each region + -s "col1 ..." # columns on which to sort + -z # for image data, output zero-valued pixels + + + + +=head1 DESCRIPTION + + + + +B<funtable> selects rows from the specified +FITS Extension +(binary table only) of a FITS file, or from a non-FITS raw event +file, and writes those rows to a FITS binary table file. It also +will create a FITS binary table from an image or a raw array file. + + +The first argument to the program specifies the FITS file, raw event +file, or raw array file. If "stdin" is specified, data are read from +the standard input. Use Funtools Bracket +Notation to specify FITS extensions, and filters. The second +argument is the output FITS file. If "stdout" is specified, the FITS +binary table is written to the standard output. By default, all +columns of the input file are copied to the output file. Selected +columns can be output using an optional third argument in the form: + + "column1 column1 ... columnN" + + + +The B<funtable> program generally is used to select rows from a +FITS binary table using +Table Filters +and/or +Spatial Region Filters. +For example, you can copy only selected rows (and output only selected +columns) by executing in a command such as: + + [sh] funtable "test.ev[pha==1&&pi==10]" stdout "x y pi pha" | fundisp stdin + X Y PHA PI + ------- ------- ------- --------- + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + + +The special column B<$REGION> can be specified to write the +region id of each row: + + [sh $] funtable "test.ev[time-(int)time>=.99&&annulus(0 0 0 10 n=3)]" stdout 'x y time $REGION' | fundisp stdin + X Y TIME REGION + -------- -------- --------------------- ---------- + 5 -6 40.99000000 3 + 4 -5 59.99000000 2 + -1 0 154.99000000 1 + -2 1 168.99000000 1 + -3 2 183.99000000 2 + -4 3 199.99000000 2 + -5 4 216.99000000 2 + -6 5 234.99000000 3 + -7 6 253.99000000 3 + + +Here only rows with the proper fractional time and whose position also is +within one of the three annuli are written. + +Columns can be excluded from display using a minus sign before the +column: + + [sh $] funtable "test.ev[time-(int)time>=.99]" stdout "-time" | fundisp stdin + X Y PHA PI DX DY + -------- -------- -------- ---------- ----------- ----------- + 5 -6 5 -6 5.50 -6.50 + 4 -5 4 -5 4.50 -5.50 + -1 0 -1 0 -1.50 0.50 + -2 1 -2 1 -2.50 1.50 + -3 2 -3 2 -3.50 2.50 + -4 3 -4 3 -4.50 3.50 + -5 4 -5 4 -5.50 4.50 + -6 5 -6 5 -6.50 5.50 + -7 6 -7 6 -7.50 6.50 + +All columns except the time column are written. + +In general, the rules for activating and de-activating columns are: + + +=over 4 + + + + +=item * + +If only exclude columns are specified, then all columns but +the exclude columns will be activated. + + +=item * + +If only include columns are specified, then only the specified columns +are activated. + + +=item * + +If a mixture of include and exclude columns are specified, then +all but the exclude columns will be active; this last case +is ambiguous and the rule is arbitrary. + + +=back + + +In addition to specifying columns names explicitly, the special +symbols I<+> and I<-> can be used to activate and +de-activate I<all> columns. This is useful if you want to +activate the $REGION column along with all other columns. According +to the rules, the syntax "$REGION" only activates the region column +and de-activates the rest. Use "+ $REGION" to activate all +columns as well as the region column. + + +Ordinarily, only the selected table is copied to the output file. In +a FITS binary table, it sometimes is desirable to copy all of the +other FITS extensions to the output file as well. This can be done by +appending a '+' sign to the name of the extension in the input file +name. For example, the first command below copies only the EVENT table, +while the second command copies other extensions as well: + + [sh] funtable "/proj/rd/data/snr.ev[EVENTS]" events.ev + [sh] funtable "/proj/rd/data/snr.ev[EVENTS+]" eventsandmore.ev + + + +If the input file is an image or a raw array file, then +B<funtable> will generate a FITS binary table from the pixel +values in the image. Note that it is not possible to specify the +columns to output (using command-line argument 3). Instead, there are +two ways to create such a binary table from an image. By default, a +3-column table is generated, where the columns are "X", "Y", and +"VALUE". For each pixel in the image, a single row (event) is +generated with the "X" and "Y" columns assigned the dim1 and dim2 +values of the image pixel, respectively and the "VALUE" column +assigned the value of the pixel. With sort of table, running +B<funhist> on the "VALUE" column will give the same results as +running B<funhist> on the original image. + + +If the B<-i> ("individual" rows) switch is specified, then only +the "X" and "Y" columns are generated. In this case, each positive +pixel value in the image generates n rows (events), where n is equal +to the integerized value of that pixel (plus 0.5, for floating point +data). In effect, B<-i> approximately recreates the rows of a +table that would have been binned into the input image. (Of course, +this is only approximately correct, since the resulting x,y positions +are integerized.) + + +If the B<-s [col1 col2 ... coln]> ("sort") switch is specified, +the output rows of a binary table will be sorted using the +specified columns as sort keys. The sort keys must be scalar columns +and also must be part of the output file (i.e. you cannot sort on a +column but not include it in the output). This facility uses the +B<_sort> program (included with funtools), which must be accessible +via your path. + + +For binary tables, the B<-m> ("multiple files") switch will +generate a separate file for each region in the filter specification +i.e. each file contains only the rows from that region. Rows +which pass the filter but are not in any region also are put in a +separate file. + + +The separate output file names generated by the B<-m> switch are +produced automatically from the root output file to contain the region id of +the associated region. (Note that region ids start at 1, so that the +file name associated with id 0 contains rows that pass the filter but +are not in any given region.) Output file names are generated as follows: + + + +=over 4 + + + + +=item * + +A $n specification can be used anywhere in the root file name (suitably +quoted to protect it from the shell) and will be expanded to be the id +number of the associated region. For example: + + funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' 'foo.goo_$n.fits' + +will generate files named foo.goo_0.fits (for rows not in any region but +still passing the filter), foo.goo_1.fits (rows in region id #1, the first +region), foo.goo_2.fits (rows in region id #2), etc. Note that single quotes +in the output root are required to protect the '$' from the shell. + + + +=item * + +If $n is not specified, then the region id will be placed before +the first dot (.) in the filename. Thus: + + funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' foo.evt.fits + +will generate files named foo0.evt.fits (for rows not in any region but +still passing the filter), foo1.evt.fits (rows in region id #1), +foo2.evt.fits (rows in region id #2), etc. + + + +=item * + +If no dot is specified in the root output file name, then +the region id will be appended to the filename. Thus: + + funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' 'foo_evt' + +will generate files named foo_evt0 (for rows not in any region but +still passing the filter), foo_evt1 (rows in region id #1), +foo_evt2 (rows in region id #2), etc. + + +=back + + +The multiple file mechanism provide a simple way to generate +individual source data files with a single pass through the data. + + +By default, a new FITS file is created and the binary table is written +to the first extension. If the B<-a> (append) switch is specified, +the table is appended to an existing FITS file as a BINTABLE extension. +Note that the output FITS file must already exist. + + +If the B<-z> ("zero" pixel values) switch is specified and +B<-i> is not specified, then pixels having a zero value will +be output with their "VALUE" column set to zero. Obviously, this +switch does not make sense when individual events are output. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funtablerowget.pod b/funtools/doc/pod/funtablerowget.pod new file mode 100644 index 0000000..86c1e66 --- /dev/null +++ b/funtools/doc/pod/funtablerowget.pod @@ -0,0 +1,111 @@ +=pod + +=head1 NAME + + + +B<FunTableRowGet - get Funtools rows> + + + +=head1 SYNOPSIS + + + + + + #include <funtools.h> + + void *FunTableRowGet(Fun fun, void *rows, int maxrow, char *plist, + int *nrow) + + + + + +=head1 DESCRIPTION + + + + +The B<FunTableRowGet()> routine retrieves rows from a Funtools +binary table or raw event file, and places the values of columns +selected by FunColumnSelect() +into an array of user structs. Selected column values are +automatically converted to the specified user data type (and to native +data format) as necessary. + + +The first argument is the Fun handle associated with this row data. +The second B<rows> argument is the array of user structs into +which the selected columns will be stored. If NULL is passed, the +routine will automatically allocate space for this array. (This +includes proper allocation of pointers within each struct, if the "@" +pointer type is used in the selection of columns. Note that if you +pass NULL in the second argument, you should free this space using the +standard free() system call when you are finished with the array of +rows.) The third B<maxrow> argument specifies the maximum number +of rows to be returned. Thus, if B<rows> is allocated by the +user, it should be at least of size maxrow*sizeof(evstruct). + + +The fourth B<plist> argument is a param list string. Currently, +the keyword/value pair "mask=transparent" is supported in the plist +argument. If this string is passed in the call's plist argument, then +all rows are passed back to the user (instead of just rows passing +the filter). This is only useful when +FunColumnSelect() also is +used to specify "$region" as a column to return for each row. In +such a case, rows found within a region have a returned region value +greater than 0 (corresponding to the region id of the region in which +they are located), rows passing the filter but not in a region have +region value of -1, and rows not passing any filter have region +value of 0. Thus, using "mask=transparent" and the returned region +value, a program can process all rows and decide on an action based +on whether a given row passed the filter or not. + + +The final argument is a pointer to an int variable that will return +the actual number of rows returned. The routine returns a pointer to +the array of stored rows, or NULL if there was an error. (This pointer +will be the same as the second argument, if the latter is non-NULL). + + /* get rows -- let routine allocate the row array */ + while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = buf+i; + /* rearrange some values. etc. */ + ev->energy = (ev->pi+ev->pha)/2.0; + ev->pha = -ev->pha; + ev->pi = -ev->pi; + } + /* write out this batch of rows */ + FunTableRowPut(fun2, buf, got, 0, NULL); + /* free row data */ + if( buf ) free(buf); + } + +As shown above, successive calls to +FunTableRowGet() will return the +next set of rows from the input file until all rows have been read, +i.e., the routine behaves like sequential Unix I/O calls such as +fread(). See evmerge example code for a +more complete example. + + +Note that FunTableRowGet() also can be called as FunEventsGet(), for +backward compatibility. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funtablerowput.pod b/funtools/doc/pod/funtablerowput.pod new file mode 100644 index 0000000..b063e43 --- /dev/null +++ b/funtools/doc/pod/funtablerowput.pod @@ -0,0 +1,200 @@ +=pod + +=head1 NAME + + + +B<FunTableRowPut - put Funtools rows> + + + +=head1 SYNOPSIS + + + + + +int FunTableRowPut(Fun fun, void *rows, int nev, int idx, char *plist) + + + + + +=head1 DESCRIPTION + + + +The B<FunTableRowPut()> routine writes rows to a FITS binary +table, taking its input from an array of user structs that contain +column values selected by a previous call to +FunColumnSelect(). Selected +column values are automatically converted from native data format to +FITS data format as necessary. + + +The first argument is the Fun handle associated with this row data. +The second B<rows> argument is the array of user structs to +output. The third B<nrow> argument specifies the number number of +rows to write. The routine will write B<nrow> records, starting +from the location specified by B<rows>. + + +The fourth B<idx> argument is the index of the first raw input +row to write, in the case where rows from the user buffer are +being merged with their raw input row counterparts (see below). Note +that this B<idx> value is has nothing to do with the +row buffer specified in argument 1. It merely matches the row +being written with its corresponding (hidden) raw row. Thus, if you +read a number of rows, process them, and then write them out all at +once starting from the first user row, the value of B<idx> +should be 0: + + Ev ebuf, ev; + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ... + } + /* write out this batch of rows, starting with the first */ + FunTableRowPut(fun2, (char *)ebuf, got, 0, NULL); + /* free row data */ + if( ebuf ) free(ebuf); + } + + + +On the other hand, if you write out the rows one at a time (possibly +skipping rows), then, when writing the i'th row from the input +array of rows, set B<idx> to the value of i: + + Ev ebuf, ev; + /* get rows -- let routine allocate the row array */ + while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = ebuf+i; + ... + /* write out the current (i.e., i'th) row */ + FunTableRowPut(fun2, (char *)ev, 1, i, NULL); + } + /* free row data */ + if( ebuf ) free(ebuf); + } + + + +The final argument is a param list string that is not currently used. +The routine returns the number of rows output. This should be equal +to the value passed in the third nrow</B argument. + + +When FunTableRowPut() is first +called for a given binary table, Funtools checks to see of the primary +header has already been written (either by writing a previous row +table or by writing an image.) If not, a dummy primary header is +written to the file specifying that an extension should be expected. +After this, a binary table header is automatically written containing +information about the columns that will populate this table. In +addition, if a +Funtools reference handle +was specified when this table was opened, the parameters from this +Funtools reference handle +are merged into the new binary table header. + + +In a typical Funtools row loop, you read rows using +FunTableRowGet()() and write +rows using FunTableRowPut(). The columns written by +FunTableRowPut()() are those defined as writable by a previous call to +FunColumnSelect(). If +that call to FunColumnSelect also specified +B<merge=[update|replace|append]>, then the entire corresponding +raw input row record will be merged with the output row according +to the B<merge> specification (see +FunColumnSelect() above). + + +A call to write rows can either be done once, after all rows in +the input batch have been processed, or it can be done (slightly less +efficiently) one row at a time (or anything in between). We do +recommend that you write all rows associated with a given batch of +input rows before reading new rows. This is B<required> if +you are merging the output rows with the raw input rows (since +the raw rows are destroyed with each successive call to get new rows). + +For example: + + Ev buf, ev; + ... + /* get rows -- let routine allocate the row array */ + while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* point to the i'th row */ + ev = buf + i; + .... process + } + /* write out this batch of rows */ + FunTableRowPut(fun2, buf, got, 0, NULL); + /* free row data */ + if( buf ) free(buf); + } + + +or + + + Ev buf, ev; + ... + /* get rows -- let routine allocate the row array */ + while( (buf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){ + /* process all rows */ + for(i=0; i<got; i++){ + /* point to the i'th row */ + ev = buf + i; + ... process + /* write out this batch of rows with the new column */ + if( dowrite ) + FunTableRowPut(fun2, buf, 1, i, NULL); + } + /* free row data */ + if( buf ) free(buf); + } + + + +Note that the difference between these calls is that the first one +outputs B<got> rows all at once and therefore passes +B<idx=0> in argument four, so that merging starts at the first raw +input row. In the second case, a check it made on each row to see +if it needs to be output. If so, the value of B<idx> is passed as +the value of the B<i> variable which points to the current row +being processed in the batch of input rows. + + +As shown above, successive calls to +FunTableRowPut() will write +rows sequentially. When you are finished writing all rows in a +table, you should call +FunFlush() to write out the FITS +binary table padding. However, this is not necessary if you +subsequently call FunClose() without doing any other I/O to the FITS +file. + + +Note that FunTableRowPut() also can be called as FunEventsPut(), for +backward compatibility. + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + +=cut diff --git a/funtools/doc/pod/funtbl.pod b/funtools/doc/pod/funtbl.pod new file mode 100644 index 0000000..1fb9bb0 --- /dev/null +++ b/funtools/doc/pod/funtbl.pod @@ -0,0 +1,137 @@ +=pod + +=head1 NAME + + + +B<funtbl - extract a table from Funtools ASCII output> + + + +=head1 SYNOPSIS + + + + + +funtable [-c cols] [-h] [-n table] [-p prog] [-s sep] <iname> + + + + + +=head1 DESCRIPTION + + + + +[NB: This program has been deprecated in favor of the ASCII text processing +support in funtools. You can now perform fundisp on funtools ASCII output +files (specifying the table using bracket notation) to extract tables +and columns.] + +The B<funtbl> script extracts a specified table (without the +header and comments) from a funtools ASCII output file and writes the +result to the standard output. The first non-switch argument is the +ASCII input file name (i.e. the saved output from funcnts, fundisp, +funhist, etc.). If no filename is specified, stdin is read. The +-n switch specifies which table (starting from 1) to extract. The +default is to extract the first table. The -c switch is a +space-delimited list of column numbers to output, e.g. -c "1 3 5" +will extract the first three odd-numbered columns. The default is to +extract all columns. The -s switch specifies the separator string to +put between columns. The default is a single space. The -h switch +specifies that column names should be added in a header line before +the data is output. Without the switch, no header is prepended. The +-p program switch allows you to specify an awk-like program to run +instead of the default (which is host-specific and is determined at +build time). The -T switch will output the data in rdb format (i.e., +with a 2-row header of column names and dashes, and with data columns +separated by tabs). The -help switch will print out a message +describing program usage. + + +For example, consider the output from the following funcnts command: + + [sh] funcnts -sr snr.ev "ann 512 512 0 9 n=3" + # source + # data file: /proj/rd/data/snr.ev + # arcsec/pixel: 8 + # background + # constant value: 0.000000 + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # summed background-subtracted results + upto net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 + 2 625.000 25.000 0.000 0.000 6976.00 0.090 0.004 + 3 1442.000 37.974 0.000 0.000 15936.00 0.090 0.002 + + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 + 2 478.000 21.863 0.000 0.000 5376.00 0.089 0.004 + 3 817.000 28.583 0.000 0.000 8960.00 0.091 0.003 + + + # the following source and background components were used: + source_region(s) + ---------------- + ann 512 512 0 9 n=3 + + reg counts pixels sumcnts sumpix + ---- ------------ --------- ------------ --------- + 1 147.000 25 147.000 25 + 2 478.000 84 625.000 109 + 3 817.000 140 1442.000 249 + + +There are four tables in this output. To extract the last one, you +can execute: + + [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -n 4 + 1 147.000 25 147.000 25 + 2 478.000 84 625.000 109 + 3 817.000 140 1442.000 249 + +Note that the output has been re-formatted so that only a single space +separates each column, with no extraneous header or comment information. + + +To extract only columns 1,2, and 4 from the last example (but with a header +prepended and tabs between columns), you can execute: + + [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -c "1 2 4" -h -n 4 -s "\t" + #reg counts sumcnts + 1 147.000 147.000 + 2 478.000 625.000 + 3 817.000 1442.000 + + +Of course, if the output has previously been saved in a file named +foo.out, the same result can be obtained by executing: + + [sh] funtbl -c "1 2 4" -h -n 4 -s "\t" foo.out + #reg counts sumcnts + 1 147.000 147.000 + 2 478.000 625.000 + 3 817.000 1442.000 + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funtext.pod b/funtools/doc/pod/funtext.pod new file mode 100644 index 0000000..9e20d1f --- /dev/null +++ b/funtools/doc/pod/funtext.pod @@ -0,0 +1,718 @@ +=pod + +=head1 NAME + + + +B<Funtext: Support for Column-based Text Files> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the options for processing column-based +text files. + + + +=head1 DESCRIPTION + + + + + + +Funtools will automatically sense and process "standard" +column-based text files as if they were FITS binary tables without any +change in Funtools syntax. In particular, you can filter text files +using the same syntax as FITS binary tables: + + fundisp foo.txt'[cir 512 512 .1]' + fundisp -T foo.txt > foo.rdb + funtable foo.txt'[pha=1:10,cir 512 512 10]' foo.fits + + + +The first example displays a filtered selection of a text file. The +second example converts a text file to an RDB file. The third example +converts a filtered selection of a text file to a FITS binary table. + + +Text files can also be used in Funtools image programs. In this case, +you must provide binning parameters (as with raw event files), using +the bincols keyword specifier: + + + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]] + + +For example: + + funcnts foo'[bincols=(x:1024,y:1024)]' "ann 512 512 0 10 n=10" + + +B<Standard Text Files> + + +Standard text files have the following characteristics: + + + +=over 4 + + + + +=item * + +Optional comment lines start with # + + +=item * + +Optional blank lines are considered comments + + +=item * + +An optional table header consists of the following (in order): + + +=over 4 + + + + +=item * + +a single line of alpha-numeric column names + + +=item * + +an optional line of unit strings containing the same number of cols + + +=item * + +an optional line of dashes containing the same number of cols + + +=back + + + + +=item * + +Data lines follow the optional header and (for the present) consist of + the same number of columns as the header. + + +=item * + +Standard delimiters such as space, tab, comma, semi-colon, and bar. + + +=back + + + + +Examples: + + + # rdb file + foo1 foo2 foo3 foos + ---- ---- ---- ---- + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + # multiple consecutive whitespace and dashes + foo1 foo2 foo3 foos + --- ---- ---- ---- + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + # comma delims and blank lines + foo1,foo2,foo3,foos + + 1,2.2,3,xxxx + 10,20.2,30,yyyy + + # bar delims with null values + foo1|foo2|foo3|foos + 1||3|xxxx + 10|20.2||yyyy + + # header-less data + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + + +The default set of token delimiters consists of spaces, tabs, commas, +semi-colons, and vertical bars. Several parsers are used +simultaneously to analyze a line of text in different ways. One way +of analyzing a line is to allow a combination of spaces, tabs, and +commas to be squashed into a single delimiter (no null values between +consecutive delimiters). Another way is to allow tab, semi-colon, and +vertical bar delimiters to support null values, i.e. two consecutive +delimiters implies a null value (e.g. RDB file). A successful parser +is one which returns a consistent number of columns for all rows, with +each column having a consistent data type. More than one parser can +be successful. For now, it is assumed that successful parsers all +return the same tokens for a given line. (Theoretically, there are +pathological cases, which will be taken care of as needed). Bad parsers +are discarded on the fly. + + +If the header does not exist, then names "col1", "col2", etc. are +assigned to the columns to allow filtering. Furthermore, data types +for each column are determined by the data types found in the columns +of the first data line, and can be one of the following: string, int, +and double. Thus, all of the above examples return the following +display: + + fundisp foo'[foo1>5]' + FOO1 FOO2 FOO3 FOOS + ---------- --------------------- ---------- ------------ + 10 20.20000000 30 yyyy + + +B<Comments Convert to Header Params> + + +Comments which precede data rows are converted into header parameters and +will be written out as such using funimage or funhead. Two styles of comments +are recognized: + + +1. FITS-style comments have an equal sign "=" between the keyword and +value and an optional slash "/" to signify a comment. The strict FITS +rules on column positions are not enforced. In addition, strings only +need to be quoted if they contain whitespace. For example, the following +are valid FITS-style comments: + + + # fits0 = 100 + # fits1 = /usr/local/bin + # fits2 = "/usr/local/bin /opt/local/bin" + # fits3c = /usr/local/bin /opt/local/bin /usr/bin + # fits4c = "/usr/local/bin /opt/local/bin" / path dir + + +Note that the fits3c comment is not quoted and therefore its value is the +single token "/usr/local/bin" and the comment is "opt/local/bin /usr/bin". +This is different from the quoted comment in fits4c. + + +2. Free-form comments can have an optional colon separator between the +keyword and value. In the absence of quote, all tokens after the +keyword are part of the value, i.e. no comment is allowed. If a string +is quoted, then slash "/" after the string will signify a comment. +For example: + + + # com1 /usr/local/bin + # com2 "/usr/local/bin /opt/local/bin" + # com3 /usr/local/bin /opt/local/bin /usr/bin + # com4c "/usr/local/bin /opt/local/bin" / path dir + + # com11: /usr/local/bin + # com12: "/usr/local/bin /opt/local/bin" + # com13: /usr/local/bin /opt/local/bin /usr/bin + # com14c: "/usr/local/bin /opt/local/bin" / path dir + + + +Note that com3 and com13 are not quoted, so the whole string is part of +the value, while comz4c and com14c are quoted and have comments following +the values. + + +Some text files have column name and data type information in the header. +You can specify the format of column information contained in the +header using the "hcolfmt=" specification. See below for a detailed +description. + +B<Multiple Tables in a Single File> + + +Multiple tables are supported in a single file. If an RDB-style file +is sensed, then a ^L (vertical tab) will signify end of +table. Otherwise, an end of table is sensed when a new header (i.e., +all alphanumeric columns) is found. (Note that this heuristic does not +work for single column tables where the column type is ASCII and the +table that follows also has only one column.) You also can specify +characters that signal an end of table condition using the B<eot=> +keyword. See below for details. + + +You can access the nth table (starting from 1) in a multi-table file +by enclosing the table number in brackets, as with a FITS extension: + + + fundisp foo'[2]' + +The above example will display the second table in the file. +(Index values start at 1 in oder to maintain logical compatibility +with FITS files, where extension numbers also start at 1). + + +B<TEXT() Specifier> + + +As with ARRAY() and EVENTS() specifiers for raw image arrays and raw +event lists respectively, you can use TEXT() on text files to pass +key=value options to the parsers. An empty set of keywords is +equivalent to not having TEXT() at all, that is: + + + fundisp foo + fundisp foo'[TEXT()]' + + +are equivalent. A multi-table index number is placed before the TEXT() +specifier as the first token, when indexing into a multi-table: + + fundisp foo'[2,TEXT(...)]' + + +The filter specification is placed after the TEXT() specifier, separated +by a comma, or in an entirely separate bracket: + + + fundisp foo'[TEXT(...),circle 512 512 .1]' + fundisp foo'[2,TEXT(...)][circle 512 512 .1]' + + +B<Text() Keyword Options> + + +The following is a list of keywords that can be used within the TEXT() +specifier (the first three are the most important): + + + +=over 4 + + + + + + +=item * + +delims="[delims]" + + +Specify token delimiters for this file. Only a single parser having these +delimiters will be used to process the file. + + fundisp foo.fits'[TEXT(delims="!")]' + fundisp foo.fits'[TEXT(delims="\t%")]' + + + + + +=item * + +comchars="[comchars]" + + +Specify comment characters. You must include "\n" to allow blank lines. +These comment characters will be used for all standard parsers (unless delims +are also specified). + + fundisp foo.fits'[TEXT(comchars="!\n")]' + + + + + +=item * + +cols="[name1:type1 ...]" + + +Specify names and data type of columns. This overrides header +names and/or data types in the first data row or default names and +data types for header-less tables. + + fundisp foo.fits'[TEXT(cols="x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e")]' + + +If the column specifier is the only keyword, then the cols= is not +required (in analogy with EVENTS()): + + fundisp foo.fits'[TEXT(x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e)]' + +Of course, an index is allowed in this case: + + fundisp foo.fits'[2,TEXT(x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e)]' + + + + + +=item * + +eot="[eot delim]" + + +Specify end of table string specifier for multi-table files. RDB +files support ^L. The end of table specifier is a string and the whole +string must be found alone on a line to signify EOT. For example: + + fundisp foo.fits'[TEXT(eot="END")]' + +will end the table when a line contains "END" is found. Multiple lines +are supported, so that: + + fundisp foo.fits'[TEXT(eot="END\nGAME")]' + +will end the table when a line contains "END" followed by a line +containing "GAME". + +In the absence of an EOT delimiter, a new table will be sensed when a new +header (all alphanumeric columns) is found. + + + + +=item * + +null1="[datatype]" + + +Specify data type of a single null value in row 1. +Since column data types are determined by the first row, a null value +in that row will result in an error and a request to specify names and +data types using cols=. If you only have a one null in row 1, you don't +need to specify all names and columns. Instead, use null1="type" to +specify its data type. + + + + +=item * + +alen=[n] + + +Specify size in bytes for ASCII type columns. +FITS binary tables only support fixed length ASCII columns, so a +size value must be specified. The default is 16 bytes. + + + + +=item * + +nullvalues=["true"|"false"] + + +Specify whether to expect null values. +Give the parsers a hint as to whether null values should be allowed. The +default is to try to determine this from the data. + + + + +=item * + +whitespace=["true"|"false"] + + +Specify whether surrounding white space should be kept as part of +string tokens. By default surrounding white space is removed from +tokens. + + + + +=item * + +header=["true"|"false"] + + +Specify whether to require a header. This is needed by tables +containing all string columns (and with no row containing dashes), in +order to be able to tell whether the first row is a header or part of +the data. The default is false, meaning that the first row will be +data. If a row dashes are present, the previous row is considered the +column name row. + + + + +=item * + +units=["true"|"false"] + + +Specify whether to require a units line. +Give the parsers a hint as to whether a row specifying units should be +allowed. The default is to try to determine this from the data. + + + + +=item * + +i2f=["true"|"false"] + + +Specify whether to allow int to float conversions. +If a column in row 1 contains an integer value, the data type for that +column will be set to int. If a subsequent row contains a float in +that same column, an error will be signaled. This flag specifies that, +instead of an error, the float should be silently truncated to +int. Usually, you will want an error to be signaled, so that you can +specify the data type using cols= (or by changing the value of +the column in row 1). + + + + +=item * + +comeot=["true"|"false"|0|1|2] + + +Specify whether comment signifies end of table. +If comeot is 0 or false, then comments do not signify end of table and +can be interspersed with data rows. If the value is true or 1 (the +default for standard parsers), then non-blank lines (e.g. lines +beginning with '#') signify end of table but blanks are allowed +between rows. If the value is 2, then all comments, including blank +lines, signify end of table. + + + + +=item * + +lazyeot=["true"|"false"] + + +Specify whether "lazy" end of table should be permitted (default is +true for standard formats, except rdb format where explicit ^L is required +between tables). A lazy EOT can occur when a new table starts directly +after an old one, with no special EOT delimiter. A check for this EOT +condition is begun when a given row contains all string tokens. If, in +addition, there is a mismatch between the number of tokens in the +previous row and this row, or a mismatch between the number of string +tokens in the prev row and this row, a new table is assumed to have +been started. For example: + + ival1 sval3 + ----- ----- + 1 two + 3 four + + jval1 jval2 tval3 + ----- ----- ------ + 10 20 thirty + 40 50 sixty + +Here the line "jval1 ..." contains all string tokens. In addition, +the number of tokens in this line (3) differs from the number of +tokens in the previous line (2). Therefore a new table is assumed +to have started. Similarly: + + ival1 ival2 sval3 + ----- ----- ----- + 1 2 three + 4 5 six + + jval1 jval2 tval3 + ----- ----- ------ + 10 20 thirty + 40 50 sixty + +Again, the line "jval1 ..." contains all string tokens. The number of +string tokens in the previous row (1) differs from the number of +tokens in the current row(3). We therefore assume a new table as been +started. This lazy EOT test is not performed if lazyeot is explicitly +set to false. + + + + +=item * + +hcolfmt=[header column format] + + +Some text files have column name and data type information in the header. +For example, VizieR catalogs have headers containing both column names +and data types: + + #Column e_Kmag (F6.3) ?(k_msigcom) K total magnitude uncertainty (4) [ucd=ERROR] + #Column Rflg (A3) (rd_flg) Source of JHK default mag (6) [ucd=REFER_CODE] + #Column Xflg (I1) [0,2] (gal_contam) Extended source contamination (10) [ucd=CODE_MISC] + + +while Sextractor files have headers containing column names alone: + + + # 1 X_IMAGE Object position along x [pixel] + # 2 Y_IMAGE Object position along y [pixel] + # 3 ALPHA_J2000 Right ascension of barycenter (J2000) [deg] + # 4 DELTA_J2000 Declination of barycenter (J2000) [deg] + +The hcolfmt specification allows you to describe which header lines +contain column name and data type information. It consists of a string +defining the format of the column line, using "$col" (or "$name") to +specify placement of the column name, "$fmt" to specify placement of the +data format, and "$skip" to specify tokens to ignore. You also can +specify tokens explicitly (or, for those users familiar with how +sscanf works, you can specify scanf skip specifiers using "%*"). +For example, the VizieR hcolfmt above might be specified in several ways: + + Column $col ($fmt) # explicit specification of "Column" string + $skip $col ($fmt) # skip one token + %*s $col ($fmt) # skip one string (using scanf format) + +while the Sextractor format might be specified using: + + $skip $col # skip one token + %*d $col # skip one int (using scanf format) + +You must ensure that the hcolfmt statement only senses actual column +definitions, with no false positives or negatives. For example, the +first Sextractor specification, "$skip $col", will consider any header +line containing two tokens to be a column name specifier, while the +second one, "%*d $col", requires an integer to be the first token. In +general, it is preferable to specify formats as explicitly as +possible. + + +Note that the VizieR-style header info is sensed automatically by the +funtools standard VizieR-like parser, using the hcolfmt "Column $col +($fmt)". There is no need for explicit use of hcolfmt in this case. + + + + +=item * + +debug=["true"|"false"] + + +Display debugging information during parsing. + + + +=back + + + +B<Environment Variables> + + +Environment variables are defined to allow many of these TEXT() values to be +set without having to include them in TEXT() every time a file is processed: + + + keyword environment variable + ------- -------------------- + delims TEXT_DELIMS + comchars TEXT_COMCHARS + cols TEXT_COLUMNS + eot TEXT_EOT + null1 TEXT_NULL1 + alen TEXT_ALEN + bincols TEXT_BINCOLS + hcolfmt TEXT_HCOLFMT + + +B<Restrictions and Problems> + + +As with raw event files, the '+' (copy extensions) specifier is not +supported for programs such as funtable. + + +String to int and int to string data conversions are allowed by the +text parsers. This is done more by force of circumstance than by +conviction: these transitions often happens with VizieR catalogs, +which we want to support fully. One consequence of allowing these +transitions is that the text parsers can get confused by columns which +contain a valid integer in the first row and then switch to a +string. Consider the following table: + + xxx yyy zzz + ---- ---- ---- + 111 aaa bbb + ccc 222 ddd + +The xxx column has an integer value in row one a string in row two, +while the yyy column has the reverse. The parser will erroneously +treat the first column as having data type int: + + fundisp foo.tab + XXX YYY ZZZ + ---------- ------------ ------------ + 111 'aaa' 'bbb' + 1667457792 '222' 'ddd' + +while the second column is processed correctly. This situation can be avoided +in any number of ways, all of which force the data type of the first column +to be a string. For example, you can edit the file and explicitly quote the +first row of the column: + + xxx yyy zzz + ---- ---- ---- + "111" aaa bbb + ccc 222 ddd + + [sh] fundisp foo.tab + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd' + +You can edit the file and explicitly set the data type of the first column: + + xxx:3A yyy zzz + ------ ---- ---- + 111 aaa bbb + ccc 222 ddd + + [sh] fundisp foo.tab + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd' + +You also can explicitly set the column names and data types of all columns, +without editing the file: + + [sh] fundisp foo.tab'[TEXT(xxx:3A,yyy:3A,zzz:3a)]' + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd' + +The issue of data type transitions (which to allow and which to disallow) +is still under discussion. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/funtools.pod b/funtools/doc/pod/funtools.pod new file mode 100644 index 0000000..ff43cfb --- /dev/null +++ b/funtools/doc/pod/funtools.pod @@ -0,0 +1,542 @@ +=pod + +=head1 NAME + + + +B<Funtools: FITS Users Need Tools> + + + +=head1 SYNOPSIS + + + + +This document is the Table of Contents for Funtools. + + + +=head1 DESCRIPTION + + + + +Funtools, is a "minimal buy-in" FITS library and utility package developed +at the the High Energy Astrophysics Division of SAO. The Funtools +library provides simplified access to a wide array of file types: +standard astronomical FITS images and binary tables, raw arrays and +binary event lists, and even tables of ASCII column data. A +sophisticated region filtering library (compatible with ds9) filters +images and tables using boolean operations between geometric shapes, +support world coordinates, etc. Funtools also supports advanced +capabilities such as optimized data searching using index files. + +The main goal of the Funtools project has been to develop a minimal buy-in +FITS library for researchers who are occasional (but serious) coders. In +this case, "minimal buy-in" means "easy to learn, easy to use, and easy to +re-learn next month". We have tried to achieve this goal by emphasizing two +essential capabilities. The first is the ability to develop FITS programs +without knowing much about FITS, i.e., without having to deal with the +arcane rules for generating a properly formatted FITS file. The second is +to support the use of already-familiar C/Unix facilities, especially C +structs and Unix stdio. Taken together, these two capabilities should allow +researchers to leverage their existing programming expertise while +minimizing the need to learn new and complex coding rules. + + + + +Choose from the following topics: + + + + +=over 4 + + + + +=item * + +Funtools User Programs + + +=over 4 + + + + +=item * + +funcalc: Funtools calculator (for binary tables) +[funcalc(1)] + + +=item * + +funcen: find centroid (for binary tables) +[funcen(1)] + + +=item * + +funcnts: count photons in specified regions +[funcnts(1)] + + +=item * + +funcone: cone search on RA, Dec columns +[funcone(1)] + + +=item * + +fundisp: display data in a Funtools data file +[fundisp(1)] + + +=item * + +funhead: display a header in a Funtools file +[funhead(1)] + + +=item * + +funhist: create a 1D histogram of a column +[funhist(1)] + + +=item * + +funimage: create a FITS image from a Funtools data file +[funimage(1)] + + +=item * + +funindex: create an index on a column in a binary table +[funindex(1)] + + +=item * + +funjoin: join two or more FITS binary tables on specified columns +[funjoin(1)] + + +=item * + +funmerge: merge one or more Funtools table files +[funmerge(1)] + + +=item * + +funsky: convert between image and sky coordinates, using WCS info from a FITS header +[funsky(1)] + + +=item * + +funtable: copy selected rows from a Funtools file to a FITS binary table +[funtable(1)] + + +=item * + +funtbl: extract a table from +Funtools ASCII output +[funtbl(1)] + + +=item * + +funtools and ds9 image display +[funds9(n)] + + +=back + + + + + +=item * + +Funtools Programming + + +=over 4 + + + + +=item * + +Funtools Programming Summary +[funlib(3)] + + +=item * + +Funtools Programming Tutorial +[funlib(3)] + + +=item * + +A Short Digression on Subroutine Order +[funlib(3)] + + +=item * + +Compiling and Linking +[funlib(3)] + + +=item * + +The Funtools Reference Handle +[funlib(3)] + + +=item * + +The Funtools Programming Reference Manual + + +=over 4 + + + + +=item * + +FunOpen: open a Funtools file +[funopen(3)] + + +=item * + +FunImageGet: retrieve image data +[funimageget(3)] + + +=item * + +FunImagePut: output image data +[funimageput(3)] + + +=item * + +FunImageRowGet: retrieve image data by row +[funimagerowget(3)] + + +=item * + +FunImageRowPut: output image data by row +[funimagerowput(3)] + + +=item * + +FunTableRowGet: retrieve rows from a table +[funtablerowget(3)] + + +=item * + +FunTableRowPut: output rows to a table +[funtablerowput(3)] + + +=item * + +FunColumnSelect: select columns in a table for access +[funcolumnselect(3)] + + +=item * + +FunColumnActivate: activate columns in a table for read/write +[funcolumnactivate(3)] + + +=item * + +FunColumnLookup: lookup info about the columns in a table +[funcolumnlookup(3)] + + +=item * + +FunInfoGet: get info about an image or table +[funinfoget(3)] + + +=item * + +FunInfoPut: put info about an image or table +[funinfoput(3)] + + +=item * + +FunParamGet: get header param +[funparamget(3)] + + +=item * + +FunParamPut: put header param +[funparamput(3)] + + +=item * + +FunFlush: flush I/O in a Funtools file +[funflush(3)] + + +=item * + +FunClose: close a Funtools file +[funclose(3)] + + +=back + + + + + +=item * + +Funtools Programming Examples +[funlib(3)] + + +=over 4 + + + + +=item * + +evmerge: merge new columns with existing columns + + +=item * + +evcols: add column and rows to binary tables + + +=item * + +imblank: blank out image values below a threshold + + +=back + + + + +=back + + + + + +=item * + +Funtools Data Files +[funfiles(n)] + + +=over 4 + + + + +=item * + +Supported Data Formats + + +=over 4 + + + + +=item * + +FITS File and Extensions + + +=item * + +Non-FITS Raw Event Files + + +=item * + +Non-FITS Array Files + + +=item * + +Column-based Text (ASCII) Files + + +=item * + +Database Views of Tables + + +=back + + + + +=item * + +Image Sections and Blocking + + +=item * + +Binning FITS Binary Tables and Non-FITS Event Files + + +=item * + +Disk Files and Other Supported File Types + + +=back + + + + + +=item * + +Funtools Data Filtering + + +=over 4 + + + + +=item * + +Table Filtering +[funfilters(n)] + + +=item * + +Fast Table Filtering using Indexes +[funidx(n)] + + +=item * + +Spatial Region Filtering +[funregions(n)] + + +=over 4 + + + + +=item * + +Region Geometry +[reggeometry(n)] + + +=item * + +Region Algebra +[regalgebra(n)] + + +=item * + +Region Coordinates +[regcoords(n)] + + +=item * + +Region Boundaries +[regbounds(n)] + + +=item * + +Differences Between Funtools and IRAF Regions +[regdiff(n)] + + +=back + + + + +=item * + +Combining Table and Region Filters +[funcombine(n)] + + +=back + + + + + +=item * + +Miscellaneous + + +=over 4 + + + + +=item * + +Funtools Environment Variables +[funenv(n)] + + +=item * + +Funtools ChangeLog + + +=back + + + + + +=back + + + + + + +=cut diff --git a/funtools/doc/pod/funview.pod b/funtools/doc/pod/funview.pod new file mode 100644 index 0000000..f0c2095 --- /dev/null +++ b/funtools/doc/pod/funview.pod @@ -0,0 +1,407 @@ +=pod + +=head1 NAME + + + +B<Funview: Database View Support for Tables> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the options for utilizing +database-inspired Views of tables. + + + +=head1 DESCRIPTION + + + + + +B<Database Views> + +In database parlance, a B<View> defines a "virtual table", i.e., +a description of row and/or column selection filters (but with no +permanent storage space allocated). When used in place of a table, a +View selects the specified rows and/or columns from one or more real +tables. Views enable you to see complicated data tables in a more +convenient format. They also can be used as a security mechanism, by +restricting user access to specific columns and/or rows. [See: + +http://www.cs.unibo.it/~ciaccia/COURSES/RESOURCES/SQLTutorial/sqlch5.htm + +for a good discussion of SQL Views.] + + +Funtools supports an expanded notion of Views for all tabular data +(FITS tables, raw binary tables, and ASCII column files). Funtools +Views allow you to pre-set values for the filter specification, the +columns to activate, and display format (though the latter is for +fundisp only). Setting the filter and column activation values +provides functionality equivalent to that of a classical database +View, while the ability to set the format is similar to classical +report writing capabilities. + +B<Funtools View Attributes> + +A Funtools View is a text file containing one or more of the following +columns: + + column description + ------ ----------------------------- + view name of view + file data file name or template + filter filter specification + columns columns to activate + format fundisp format specification + +All of the attribute columns are optional, including +the B<view> name itself. This means that a View can be named or +unnamed. Unnamed Views can refer to a specific file or a template of +files (obviously if neither the view or the file column is specified, +the input View specification will never be used). You can specify any +combination of filter, column, and format parameters. (It also is +possible to apply file-specific View to other files; see the discussion +on B<View Lists> below). Each column has a size limit of 1024 characters. + + +For example, consider the following View file: + + view file format columns filter + ---- ---------------------- ------ ------------ ------- + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 + x2 ${HOME}/data/snr.ev x y pi pha cir 512 512 .1 + x1 ${HOME}/data/snr.ev cir 512 512 .1 + x1a ${HOME}/data/snr.ev x y pi pha + x0 ${HOME}/data/snr.ev + xf I=%4d + xc x y pi pha + xr cir 512 512 .1 + *.ev x y pi pha + *.fit x y dx dy cir 400 400 3 + *.fits I=%3d x y dx dy cir 400 400 3 + +This database example is in rdb format, i.e. using tab delimiters and +permitting null values. Any valid ASCII table format is acceptable, +but if you use a format that does not permit null values, it will be +necessary to quote the null strings. + + +The first five entries (x3, x2, x1, x1a, x0) are named entries defining +default values specifically for the snr.ev data file. Typically, you +would use these Views by specifying View name, and the corresponding +file, filter, column, and format values would be used. Note that the x0 +View is essentially an alias for the pathname of this file. + + +The next three entries define defaults that can be applied to any +file. You typically would use these View names in conjunction with +a specific file name (see B<View Lists> below) so that the associated +parameter(s) were applied to that file. + + +The last three entry in the database define unnamed Views that +pertains to all files ending with the specified templates. In these +cases, any View that specifies a file name matching the file template +would be processed with the associated parameter attributes. + +B<Invoking a Funtools View (in Place of an Input File)> + +To use a Funtools View, you simply pre-pend the "v:" prefix to a View name or +a file name where an input file name usually is specified. For example: + + fundisp v:x3 + +specifies that the View named x3 (with its file name and associated +parameters) is processed as the input file to fundisp. Using the +example database, above, this is equivalent to: + + fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha" + +That is, the format is used with fundisp's -f (format) switch, while the +filename and extension are composed of the x3 View's filename and +region filter. + + +Similarly, executing a command such as: + + fundisp v:foo.fit + +will match the unnamed View associated with the template "*.fit". +This is equivalent to executing: + + fundisp foo.fit'[cir 400 400 3]' "x y dx dy" + +Of course, if you omit the "v:" prefix, then no View processing takes place: + + fundisp foo.fit # process foo.fit without any View parameters + fundisp x3 # error (assuming there is no file named x3) + + +B<Basic View Matching Rules> + + +When a "v:" prefix is recognized, Funtools searches for a View database +file in the following order: + + location description + ------------ ------------------------------------ + FUN_VIEWFILE environment variable (any file name) + ./.funtools.vu hidden file, default name + $HOME/.funtools.vu hidden file, default name + +The first View database file located is used to construct a new +filename, as well as an activation column specification and a format +specification. The following rules are used: + + +1. An attempt is made to match the input name (i.e., the part of the +input View after the "v:" prefix) against the B<view> column value +(if present) of each row in the database. If a match is found, the +values of all non-blank columns are saved for later use. Also note +that the first match terminates the search: i.e., the order of the +database rows matters. + + +2. If no B<view> match is made, an attempt is made to match the input +name against the B<file> column value (if present). Matching is +performed on the full pathname of both the input name and the +database file name, and on the non-directory (root) part of these +files. This means that the root specification: + + fundisp v:snr.ev + +will match a row in the database that has a full pathname in the file, +allowing you to use a B<file>-matched View without having to +specify the full pathname. In this example, the "v:snr.ev" View +specification will match the first row (v:x3) in the database: + + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 + +even though the row contains a fully qualified pathname as the file +value. Once again, values of all non-blank columns are saved, and the +first match terminates the search. + + +3. If neither a B<view> or a B<view> match has been found, +then a simple template match is attempted against the B<view> +values. Template matching supports a simplified version of file +globbing (not a regular expression), with support for a single "*" +(all characters), "?" (single character), or "[...]" (range) specification. + + +4. If no template match was found on the B<view> column, then a +simple template match is attempted against the B<file> columns. + + +5. If no match is found, then the filename (minus the "v:" prefix) is +returned. + +B<More on View Matching Rules: Single vs. Multiple Matches > + +The matching rules described above stop after the first match, +regardless of whether that match provides values for all three +parameters (filter, columns, and format). In cases where a B<view> +or B<file> match does not provide all three values, it is possible +that a template match might do so. With regard to the example View +database above, the x1 View provides only a filter, while omitting +both the format and columns values. But note that the final rows in +the database could provide the values via a template match on the +filename. This sort of multiple matching is especially valuable in +order to provide "global" values to several Views. + + +Obviously, multiple matching might not be wanted in every +case. Therefore, we support both multiple matching and single matching +according to the value of the FUN_VIEWMATCH environment variable. If +the FUN_VIEWMATCH environment variable exists and if its value begins +with "s", then a single match is used and missing parameters are not +filled in with subsequent template matches on the file name. That is, +matching rules above are followed exactly as explained above. If the +value of this environment variable begins with "m" (or does not exist), +then multiple matches are used to try to fill in missing parameters. +In this case, template matching always takes place and missing values are +taken from these template matches. + + +Thus, in the example above, the View specification: + + fundisp v:x1 + +will take the file name and filter value from the x1 View: + + x1 ${HOME}/data/snr.ev cir 512 512 .1 + +The column value then will be taken from the "*.ev" file template match +against the x1 file name: + + *.ev x y pi pha + +Note once again that order is important: missing values are taken in the +order in which the template matches are processed. + +B<View Lists: Applying a View to Any File> + + +It is possible to apply a named View, or even several Views, to any +data file by appending a B<viewlist> immediately after the standard "v:" +prefix. A viewlist takes the form: + + :v1,v2,...vn: + +where v1, v2, etc. are named Views. The two ":" colon characters surrounding +the list are required. Thus, the syntax for applying a viewlist to a file is: + + v::view1,view2,...viewn:filename + +Note that the name after the last ":" is assumed to be a file; it is +not permissible (or sensible) to use a View name. + + +For example, the View specification: + + fundisp v::x2:foo + +applies the x2 View to the file foo (even if there is a View named foo) +and (in using our example database) is equivalent to: + + ./fundisp foo'[cir 512 512 .1] "x y pi pha" + +The same command can be effected using a list of Views: + + fundisp v::x1,x1a:foo + + + +What happens if a viewlist is used and the file also matches a +template? Consider, for example, this View specification: + + fundisp v::x2:foo.fit + +Here, the x2 View will supply filter and column values, while the +template *.fit can also supply (different) filter and column +values. In this case, the explicitly specified Views of the viewlist +trump the matched view values. + + +On the other hand, if a file template match can supply a View value +that is not supplied by the viewlist, then that value will be taken +from the file template match. For example: + + fundisp v::x2:foo.fits + +does not explicitly supply a format value, but the file match on *.fits +can and does. You can avoid supplying missing values using file template +matching by replacing the first ":" with a "-" in a viewlist +specification: + + fundisp v:-x2:foo.fits + +The use of ":+" to explicitly allow file template matching is also +supported, but is the same as the default case. Note that the nuances +of viewlist support are subject to change as our experience and +understanding grow. + +B<Overriding Values Associated with a View> + + +To override values associated with a View, simply supply the override +values in the correct place on the command line. Thus, given +the example database described above, the command: + + fundisp v:x3 + +specifies that the View named x3, along with its file name and +associated parameters, be processed as the input file to fundisp in +this way: + + fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha" + +To override one or more of these values, simply specify a new value +for the format, filter, or columns. For example, if your input View file +contains a filter, then the View will use that filter as an override +of the View filter: + + fundisp v:x3'[cir 400 400 3]' + +will use the columns and format of the x3 View but not the x3 filter. Further +examples are: + + fundisp v:x3 "x y dx dy" # activate a different set of columns + fundisp -f "I=%3d" v:x3 # use a different format statement + + + +Note that extension names, extension index values, and other +non-filter specifications B<do not> override the View +filter. Thus: + + fundisp v:foo.fit[3] + +will still use the filter associated with the .fit template (see above), since +the "3" is an extension index, not a filter. + +B<Environment Variables> + +The following environment variables are used by Funtools Views: + + +=over 4 + + + + +=item * + +B<FUN_VIEWNAME> + + +The B<FUN_VIEWNAME> environment variable specifies the +name and location of the View database file. If not present, the +files ./.funtools.vu and $HOME/.funtools.vu are searched for, in +that order. + + + +=item * + +B<FUN_VIEWMATCH> + + +The B<FUN_VIEWMATCH> environment variable specifies whether a +single match or multiple match algorithm is used to locate parameter +values. If the value of this environment variable begins with "s", +then a single match is used and missing parameters are not filled in +with subsequent template matches on the file name. If the value begins +with "m", then multiple matches are used to try to fill in missing +parameters. The default is to use multiple matches. + + +=back + + + +B<Restrictions and Problems> + +Support for overriding a filter (while not overriding extension names, +extension indexes, etc.) requires that we can sense the presence of a +filter in a bracket specification. It is unclear yet whether our +algorithm is perfect. + + +Go to Funtools Help Index + +Last updated: August 3, 2007 + + + + + +=cut diff --git a/funtools/doc/pod/funvu.pod b/funtools/doc/pod/funvu.pod new file mode 100644 index 0000000..528ea50 --- /dev/null +++ b/funtools/doc/pod/funvu.pod @@ -0,0 +1,404 @@ +=pod + +=head1 NAME + + + +B<Funvu: database View support for tables> + + + +=head1 SYNOPSIS + + + + + +This document contains a summary of the options for utilizing +database-inspired Views of tables. + + + +=head1 DESCRIPTION + + + + + +B<Database Views> + +In database parlance, a B<View> defines a "virtual table", i.e., +a description of row and/or column selection filters (but with no +permanent storage space allocated). When used in place of a table, a +View selects the specified rows and/or columns from one or more real +tables. Views enable users to see complicated data tables in a more +convenient format. They also can be used as a security mechanism, by +restricting user access to specific columns and/or rows. [See: +http://www.cs.unibo.it/~ciaccia/COURSES/RESOURCES/SQLTutorial/sqlch5.htm +for a good discussion of SQL Views.] + + +Funtools supports an expanded notion of Views for all tabular data +(FITS tables, raw binary tables, and ASCII column files). Funtools +Views allow you to pre-set values for the filter specification, the +columns to activate and display format (though the latter is for +fundisp only). Setting the filter and column activation values +provides functionality equivalent to that of a classical database +View, while the ability to set the format is similar to classical +report writing capabilities. + +B<Funtools View Attributes> + +A Funtools View consists of one or more of the following attributes: + + column description + ------ ----------------------------- + view name of view + file data file name or template + filter filter specification + columns columns to activate + format fundisp format specification + +All of the attribute columns are optional, including +the B<view> name itself. This means that a View can be named or +unnamed. Unnamed Views can refer to a specific file or a template of +files (obviously if neither the view or the file column is specified, +the input View specification will never be used). You can specify any +combination of filter, column, and format parameters. (It also is +possible to apply file-specific View to other files; see the discussion +on B<View Lists> below). Each column has a size limit of 1024 characters. + + +For example, consider the following View database: + + view file format columns filter + ---- ---------------------- ------ ------------ ------- + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 + x2 ${HOME}/data/snr.ev x y pi pha cir 512 512 .1 + x1 ${HOME}/data/snr.ev cir 512 512 .1 + x1a ${HOME}/data/snr.ev x y pi pha + x0 ${HOME}/data/snr.ev + xf I=%4d + xc x y pi pha + xr cir 512 512 .1 + *.ev x y pi pha + *.fit x y dx dy cir 400 400 3 + *.fits I=%3d x y dx dy cir 400 400 3 + +This database example is in rdb format, i.e. using tab delimiters and +permitting null values. Any valid ASCII table format is acceptable, +but if you use a format that does not permit null values, it will be +necessary to quote the null strings. + + +The first five entries (x3, x2, x1, x1a, x0) are named entries defining +default values specifically for the snr.ev data file. Typically, you +would use these Views by specifying View name, and the corresponding +file, filter, column, and format values would be used. Note that the x0 +View is essentially an alias for the pathname of this file. + + +The next three entries define defaults that can be applied to any +file. Here, you typically would use these View names in conjunction with +a specific file name (see B<View Lists> below) so that the associated +parameter(s) were applied to that file. + + +The last three entry in the database define unnamed Views that +pertains to all files ending with the specified templates. In these +cases, any View that specifies a file name matching the file template +would be processed with the associated parameter attributes. + +B<Invoking a Funtools View (in Place of an Input File)> + +To specify a Funtools View, pre-pend the "v:" prefix to a View name or +a file name where an input file name usually is specified. For example: + + fundisp v:x3 + +specifies that the View named x3, its file name and associated +parameters, be processed as the input file to fundisp. Using the +example database, above, this is equivalent to: + + fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha" + +That is, the format is used with fundisp's -f (format) switch, while the +filename and extension are composed of the x3 View's filename and +region filter. + + +Similarly, executing a command such as: + + fundisp v:foo.fit + +will match the unnamed View associated with the template "*.fit". +This is equivalent to executing: + + fundisp foo.fit'[cir 400 400 3]' "x y dx dy" + +Of course, if you omit the "v:" prefix, then no View processing takes place: + + fundisp foo.fit # process foo.fit without any View parameters + fundisp x3 # error (assuming there is no file named x3) + + +B<Basic View Matching Rules> + + +When a "v:" prefix is recognized, Funtools searches for a View database +file in the following order: + + location description + ------------ ------------------------------------ + FUN_VIEWFILE environment variable (any file name) + ./.funtools.vu hidden file, default name + $HOME/.funtools.vu hidden file, default name + +The first View database file located is used to construct a new +filename, as well as an activation column specification and a format +specification. The following rules are used: + + +1. An attempt is made to match the input name (i.e. the part of the +input View after the "v:" prefix) against the B<view> column value +(if present) of each row in the database. If a match is found, the +values of all non-blank columns are saved for later use. Also note +that the first match terminates the search: i.e. the order of the +database rows does matter. + + +2. If no B<view> match is made, an attempt is made to match the input +name against the B<file> column value (if present). Matching is +performed on the full pathname of both the input name and the +database file name, and on the non-directory (root) part of these +files. This means that the root specification: + + fundisp v:snr.ev + +will match a row in the database that has a full pathname in the file, +allowing you to use a B<file>-matched View without having to +specify the full pathname. In this example, the "v:snr.ev" View +specification will match the first row (v:x3) in the database: + + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 + +even though the row contains a fully qualified pathname as the file +value. Once again, values of all non-blank columns are saved, and the +first match terminates the search. + + +3. If neither a B<view> or a B<view> match has been found, +then a simple template is attempted against the B<view> +values. The template matching supports a simplified version of file +globbing (not a regular expression), with support for a single "*" +(all characters), "?" (single character), or "[...]" (range) specification. + + +4. If no template match was found on the B<view> column, then a +simple template match is attempted against the B<file> columns. + + +5. If no match is found, then the filename (minus the "v:" prefix) is +returned. + +B<More on View Matching Rules: Single vs. Multiple Matches > + +The matching rules described above stop after the first match, +regardless of whether that match provides values for all three +parameters (filter, columns, and format). In cases where a B<view> +or B<file> match does not provide all three values, it is possible +that a template match might do so. With regard to the example View +database above, the x1 View provides only a filter, while omitting +both the format and columns values. But note that the final rows in +the database could provide the values via a template match on the +filename. This sort of multiple matching is especially valuable in +order to provide "global" values to several Views. + + +Obviously, multiple matching might not be wanted in every +case. Therefore, we support both multiple matching and single matching +according to the value of the FUN_VIEWMATCH environment variable. If +the FUN_VIEWMATCH environment variable exists and if its value begins +with "s", then a single match is used and missing parameters are not +filled in with subsequent template matches on the file name. That is, +matching rules above are followed exactly as explained above. If the +value of this environment variable begins with "m" (or does not exist), +then multiple matches are used to try to fill in missing parameters. +In this case, template matching always takes place and missing values are +taken from these template matches. + + +Thus, in the example above, the View specification: + + fundisp v:x1 + +will take the file name and filter value from the x1 View: + + x1 ${HOME}/data/snr.ev cir 512 512 .1 + +The column value then will be taken from the "*.ev" file template match +against the x1 file name: + + *.ev x y pi pha + +Note once again that order is important: missing values are taken in the +order in which the template matches are processed. + +B<View Lists: Applying a View to Any File> + + +It is possible to apply a named View, or even several Views, to any +data file by appending a B<viewlist> immediately after the standard "v:" +prefix. A viewlist takes the form: + + :v1,v2,...vn: + +where v1, v2, etc. are named Views. The two ":" colon characters surrounding +the list are required. Thus, the syntax for applying a viewlist to a file is: + + v::view1,view2,...viewn:filename + +Note that the name after the last ":" is assumed to be a file; it is +not permissible (or sensible) to use a View name. + + +For example, the View specification: + + fundisp v::x2:foo + +applies the x2 View to the file foo (even if there is a View named foo) +and (in using our example database) is equivalent to: + + ./fundisp foo'[cir 512 512 .1] "x y pi pha" + +The same command can be effected using a list of Views: + + fundisp v::x1,x1a:foo + + + +What happens if a viewlist is used and the file also matches a +template? Consider, for example, this View specification: + + fundisp v::x2:foo.fit + +Here, the x2 View will supply filter and column values, while the +template *.fit can also supply (different) filter and column +values. In this case, the explicitly specified Views of the viewlist +trump the matched view values. + + +On the other hand, if a file template match can supply a View value +that is not supplied by the viewlist, then that value will be taken +from the file template match. For example: + + fundisp v::x2:foo.fits + +does not explicitly supply a format value, but the file match on *.fits +can and does. You can avoid supplying missing values using file template +matching by replacing the first ":" with a "-" in a viewlist +specification: + + fundisp v:-x2:foo.fits + +The use of ":+" to explicitly allow file template matching is also +supported, but is the same as the default case. Note that the nuances +of viewlist support are subject to change as our experience and +understanding grows. + +B<Overriding Values Associated with a View> + + +To override values associated with a View, simply supply the override +values in the correct place on the command line. For example, given +the example database described above, the command: + + fundisp v:x3 + +specifies that the View named x3, along with its file name and +associated parameters, be processed as the input file to fundisp in +this way: + + fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha" + +To override one or more of these values, simply specify a new value +for the format, filter, or columns. For example, if your input View file +contains a filter, then the View will use that filter as an override +of the View filter: + + fundisp v:x3'[cir 400 400 3]' + +will use the columns and format of the x3 View but not the x3 filter. Further +examples are: + + fundisp v:x3 "x y dx dy" # activate a different set of columns + fundisp -f "I=%3d" v:x3 # use a different format statement + + + +Note that extension names, extension index values, and other +non-filter specifications B<do not> override the View +filter. Thus: + + fundisp v:foo.fit[3] + +will still use the filter associated with the .fit template (see above), since +the "3" is an extension index, not a filter. + +B<Environment Variables> + +The following environment variables are used by Funtools Views: + + +=over 4 + + + + +=item * + +B<FUN_VIEWNAME> + + +The B<FUN_VIEWNAME> environment variable specifies the +name and location of the View database file. If not present, the +files ./.funtools.vu and $HOME/.funtools.vu are searched for, in +that order. + + + +=item * + +B<FUN_VIEWMATCH> + + +The B<FUN_VIEWMATCH> environment variable specifies whether a +single match or multiple match algorithm is used to locate parameter +values. If the value of this environment variable begins with "s", +then a single match is used and missing parameters are not filled in +with subsequent template matches on the file name. If the value begins +with "m", then multiple matches are used to try to fill in missing +parameters. The default is to use multiple matches. + + +=back + + + +B<Restrictions and Problems> + +Support for overriding a filter (while not overriding extension names, +extension indexes, etc.) requires that we can sense the presence of a +filter in a bracket specification. It is unclear yet whether our +algorithm is perfect. + + +Go to Funtools Help Index + +Last updated: January 3, 2006 + + + + + +=cut diff --git a/funtools/doc/pod/regalgebra.pod b/funtools/doc/pod/regalgebra.pod new file mode 100644 index 0000000..4c6da88 --- /dev/null +++ b/funtools/doc/pod/regalgebra.pod @@ -0,0 +1,286 @@ +=pod + +=head1 NAME + + + +B<RegAlgebra: Boolean Algebra on Spatial Regions> + + + +=head1 SYNOPSIS + + + + + +This document describes the boolean arithmetic defined for +region expressions. + + + +=head1 DESCRIPTION + + + + + +When defining a region, several shapes can be combined using boolean +operations. The boolean operators are (in order of precedence): + + Symbol Operator Associativity + ------ -------- ------------- + ! not right to left + & and left to right + ^ exclusive or left to right + | inclusive or left to right + + +For example, to create a mask consisting of a large circle with a +smaller box removed, one can use the B<and> and B<not> +operators: + + CIRCLE(11,11,15) & !BOX(11,11,3,6) + + +and the resulting mask is: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:1111111111111111111111.................. + 2:1111111111111111111111.................. + 3:11111111111111111111111................. + 4:111111111111111111111111................ + 5:111111111111111111111111................ + 6:1111111111111111111111111............... + 7:1111111111111111111111111............... + 8:1111111111111111111111111............... + 9:111111111...1111111111111............... + 10:111111111...1111111111111............... + 11:111111111...1111111111111............... + 12:111111111...1111111111111............... + 13:111111111...1111111111111............... + 14:111111111...1111111111111............... + 15:1111111111111111111111111............... + 16:1111111111111111111111111............... + 17:111111111111111111111111................ + 18:111111111111111111111111................ + 19:11111111111111111111111................. + 20:1111111111111111111111.................. + 21:1111111111111111111111.................. + 22:111111111111111111111................... + 23:..11111111111111111..................... + 24:...111111111111111...................... + 25:.....11111111111........................ + 26:........................................ + 27:........................................ + 28:........................................ + 29:........................................ + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................ + +A three-quarter circle can be defined as: + + CIRCLE(20,20,10) & !PIE(20,20,270,360) + + +and looks as follows: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:........................................ + 2:........................................ + 3:........................................ + 4:........................................ + 5:........................................ + 6:........................................ + 7:........................................ + 8:........................................ + 9:........................................ + 10:........................................ + 11:...............111111111................ + 12:..............11111111111............... + 13:............111111111111111............. + 14:............111111111111111............. + 15:...........11111111111111111............ + 16:..........1111111111111111111........... + 17:..........1111111111111111111........... + 18:..........1111111111111111111........... + 19:..........1111111111111111111........... + 20:..........1111111111111111111........... + 21:..........1111111111.................... + 22:..........1111111111.................... + 23:..........1111111111.................... + 24:..........1111111111.................... + 25:...........111111111.................... + 26:............11111111.................... + 27:............11111111.................... + 28:..............111111.................... + 29:...............11111.................... + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................ + +Two non-intersecting ellipses can be made into the same region: + + ELL(20,20,10,20,90) | ELL(1,1,20,10,0) + + +and looks as follows: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:11111111111111111111.................... + 2:11111111111111111111.................... + 3:11111111111111111111.................... + 4:11111111111111111111.................... + 5:1111111111111111111..................... + 6:111111111111111111...................... + 7:1111111111111111........................ + 8:111111111111111......................... + 9:111111111111............................ + 10:111111111............................... + 11:...........11111111111111111............ + 12:........111111111111111111111111........ + 13:.....11111111111111111111111111111...... + 14:....11111111111111111111111111111111.... + 15:..11111111111111111111111111111111111... + 16:.1111111111111111111111111111111111111.. + 17:111111111111111111111111111111111111111. + 18:111111111111111111111111111111111111111. + 19:111111111111111111111111111111111111111. + 20:111111111111111111111111111111111111111. + 21:111111111111111111111111111111111111111. + 22:111111111111111111111111111111111111111. + 23:111111111111111111111111111111111111111. + 24:.1111111111111111111111111111111111111.. + 25:..11111111111111111111111111111111111... + 26:...11111111111111111111111111111111..... + 27:.....11111111111111111111111111111...... + 28:.......111111111111111111111111......... + 29:...........11111111111111111............ + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................ + +You can use several boolean operations in a single region expression, +to create arbitrarily complex regions. With the important exception +below, you can apply the operators in any order, using parentheses if +necessary to override the natural precedences of the operators. + + +NB: Using a panda shape is always much more efficient than explicitly +specifying "pie & annulus", due to the ability of panda to place a +limit on the number of pixels checked in the pie shape. If you are +going to specify the intersection of pie and annulus, use panda +instead. + + +As described in "help regreometry", the B<PIE> slice goes to the +edge of the field. To limit its scope, B<PIE> usually is is +combined with other shapes, such as circles and annuli, using boolean +operations. In this context, it is worth noting that that there is a +difference between B<-PIE> and B<&!PIE>. The former is a +global exclude of all pixels in the B<PIE> slice, while the latter +is a local excludes of pixels affecting only the region(s) with which +the B<PIE> is combined. For example, the following region uses +B<&!PIE> as a local exclude of a single circle. Two other circles +are also defined and are unaffected by the local exclude: + + + CIRCLE(1,8,1) + CIRCLE(8,8,7)&!PIE(8,8,60,120)&!PIE(8,8,240,300) + CIRCLE(15,8,2) + + 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 + - - - - - - - - - - - - - - - + 15: . . . . . . . . . . . . . . . + 14: . . . . 2 2 2 2 2 2 2 . . . . + 13: . . . 2 2 2 2 2 2 2 2 2 . . . + 12: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 11: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 10: . . . . 2 2 2 2 2 2 2 . . . . + 9: . . . . . . 2 2 2 . . . . 3 3 + 8: 1 . . . . . . . . . . . . 3 3 + 7: . . . . . . 2 2 2 . . . . 3 3 + 6: . . . . 2 2 2 2 2 2 2 . . . . + 5: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 4: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 3: . . . 2 2 2 2 2 2 2 2 2 . . . + 2: . . . . 2 2 2 2 2 2 2 . . . . + 1: . . . . . . . . . . . . . . . + + +Note that the two other regions are not affected by the B<&!PIE>, +which only affects the circle with which it is combined. + + +On the other hand, a B<-PIE> is an global exclude that does +affect other regions with which it overlaps: + + + CIRCLE(1,8,1) + CIRCLE(8,8,7) + -PIE(8,8,60,120) + -PIE(8,8,240,300) + CIRCLE(15,8,2) + + 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 + - - - - - - - - - - - - - - - + 15: . . . . . . . . . . . . . . . + 14: . . . . 2 2 2 2 2 2 2 . . . . + 13: . . . 2 2 2 2 2 2 2 2 2 . . . + 12: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 11: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 10: . . . . 2 2 2 2 2 2 2 . . . . + 9: . . . . . . 2 2 2 . . . . . . + 8: . . . . . . . . . . . . . . . + 7: . . . . . . 2 2 2 . . . . . . + 6: . . . . 2 2 2 2 2 2 2 . . . . + 5: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 4: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 3: . . . 2 2 2 2 2 2 2 2 2 . . . + 2: . . . . 2 2 2 2 2 2 2 . . . . + 1: . . . . . . . . . . . . . . . + + +The two smaller circles are entirely contained within the two exclude +B<PIE> slices and therefore are excluded from the region. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/regbounds.pod b/funtools/doc/pod/regbounds.pod new file mode 100644 index 0000000..c63a0bd --- /dev/null +++ b/funtools/doc/pod/regbounds.pod @@ -0,0 +1,203 @@ +=pod + +=head1 NAME + + + +B<RegBounds: Region Boundaries> + + + +=head1 SYNOPSIS + + + + +Describes how spatial region boundaries are handled. + + + +=head1 DESCRIPTION + + + + + +The golden rule for spatial region filtering was first enunciated by +Leon VanSpeybroeck in 1986: + + +Each photon will be counted once, and no photon will be counted +more than once. + + +This means that we must be careful about boundary +conditions. For example, if a circle is contained in an annulus such +that the inner radius of the annulus is the same as the radius of the +circle, then photons on that boundary must always be assigned to one +or the other region. That is, the number of photons in both regions +must equal the sum of the number of photons in each region taken +separately. + +With this in mind, the rules for determining whether a boundary image +pixel or table row are assigned to a region are defined below. + +B<Image boundaries : radially-symmetric shapes (circle, annuli, ellipse)> + +For image filtering, pixels whose center is inside the boundary are +included. This also applies non-radially-symmetric shapes. When a +pixel center is exactly on the boundary, the pixel assignment rule is: + + + +=over 4 + + + + +=item * + +the outer boundary of a symmetric shape does not include such pixels + + +=item * + +the inner boundary of a symmetric shape (annulus) includes such pixels + + +=back + + + +In this way, an annulus with radius from 0 to 1, centered exactly on a +pixel, includes the pixel on which it is centered, but none of its +neighbors. + +These rules ensure that when defining concentric shapes, no pixels are +omitted between concentric regions and no pixels are claimed by two +regions. When applied to small symmetric shapes, the shape is less +likely to be skewed, as would happen with non-radially-symmetric +rules. These rules differ from the rules for box-like shapes, which +are more likely to be positioned adjacent to one another. + +B<Image Boundaries: non-radially symmetric shapes (polygons, boxes)> + +For image filtering, pixels whose center is inside the boundary are +included. This also applies radially-symmetric shapes. When a pixel +center is exactly on the boundary of a non-radially symmetric region, +the pixel is included in the right or upper region, but not the left +or lower region. This ensures that geometrically adjoining regions +touch but don't overlap. + +B<Row Boundaries are Analytic> + +When filtering table rows, the boundary rules are the same as for +images, except that the calculation is not done on the center of a +pixel, (since table rows, especially X-ray events rows, often have +discrete, floating point positions) but are calculated exactly. That +is, an row is inside the boundary without regard to its integerized +pixel value. For rows that are exactly on a region boundary, the +above rules are applied to ensure that all rows are counted once and +no row is counted more than once. + + +Because row boundaries are calculated differently from image boundaries, +certain programs will give different results when filtering the same +region file. In particular, fundisp/funtable (which utilize analytic +row filtering) perform differently from funcnts (which performs image +filtering, even on tables). + +B<Image Boundaries vs. Row Boundaries: Practical Considerations> + + +You will sometimes notice a discrepancy between running funcnts on an +binary table file and running fundisp on the same file with the same filter. +For example, consider the following: + + fundisp test1.fits"[box(4219,3887,6,6,0)]" | wc + 8893 320148 3752846 + +Since fundisp has a 2-line header, there are actually 8891 photons +that pass the filter. But then run funtable and select only the +rows that pass this filter, placing them in a new file: + + ./funtable test1.fits"[box(4219,3887,6,6,0)]" test2.fits + +Now run funcnts using the original filter on the derived file: + + ./funcnts test2.fits "physical; box(4219,3887,6,6,0)" + + [... lot of processed output ...] + + # the following source and background components were used: + source region(s) + ---------------- + physical; box(4219,3887,6,6,0) + + reg counts pixels + ---- ------------ --------- + 1 7847.000 36 + +There are 1044 rows (events) that pass the row filter in fundisp (or +funtable) but fail to make it through funcnts. Why? + + +The reason can be traced to how analytic row filtering (fundisp, funtable) +differs from integerized pixel filtering(funcnts, funimage). Consider the +region: + + box(4219,3887,6,6,0) + +Analytically (i.e., using row filtering), positions will pass this +filter successfully if: + + 4216 <= x <= 4222 + 3884 <= y <= 3890 + +For example, photons with position values of x=4216.4 or y=3884.08 will pass. + + +Integerized image filtering is different in that the pixels that will +pass this filter have centers at: + + x = 4217, 4218, 4219, 4220, 4221, 4222 + y = 3885, 3886, 3887, 3888, 3889, 3890 + +Note that there are 6 pixels in each direction, as specified by the region. +That means that positions will pass the filter successfully if: + + 4217 <= (int)x <= 4222 + 3885 <= (int)y <= 3890 + +Photons with position values of x=4216.4 or y=3884.08 will NOT pass. + + +Note that the position values are integerized, in effect, binned into +image values. This means that x=4222.4 will pass this filter, but not +the analytic filter above. We do this to maintain the design goal that +either all counts in a pixel are included in an integerized filter, or +else none are included. + + +[It could be argued that the correct photon limits for floating point +row data really should be: + + 4216.5 <= x <= 4222.5 + 3884.5 <= y <= 3890.5 + +since each pixel extends for .5 on either side of the center. We chose +to the maintain integerized algorithm for all image-style filtering so +that funcnts would give the exact same results regardless of whether +a table or a derived non-blocked binned image is used.] + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/regcoords.pod b/funtools/doc/pod/regcoords.pod new file mode 100644 index 0000000..a026a5c --- /dev/null +++ b/funtools/doc/pod/regcoords.pod @@ -0,0 +1,247 @@ +=pod + +=head1 NAME + + + +B<RegCoords: Spatial Region Coordinates> + + + +=head1 SYNOPSIS + + + + + +This document describes the specification of coordinate systems, and the +interpretation of coordinate values, for spatial region filtering. + + + +=head1 DESCRIPTION + + + +B<Pixel coordinate systems> + +The default coordinate system for regions is PHYSICAL, which means +that region position and size values are taken from the original +data. (Note that this is a change from the original IRAF/PROS +implementation, in which the IMAGE coordinate system was the default.) +PHYSICAL coordinates always refer to pixel positions on the original +image (using IRAF LTM and LTV keywords). With PHYSICAL coordinates, +if a set of coordinates specifies the position of an object in an +original FITS file, the same coordinates will specify the same object +in any FITS derived from the original. Physical coordinates are +invariant with blocking of FITS files or taking sections of images, +even when a blocked section is written to a new file. + + +Thus, although a value in pixels refers, by default, to the PHYSICAL +coordinate system, you may specify that position values refer to the +image coordinate system using the B<global> or B<local> +properties commands: + + + global coordsys image + circle 512 512 100 + + +The B<global> command changes the coordinate system for all +regions that follow, while the B<local> command changes the +coordinate system only for the region immediately following: + + local coordsys image + circle 512 512 100 + circle 1024 1024 200 + +This changes the coordinate system only for the region that follows. +In the above example, the second region uses the global coordinate +system (PHYSICAL by default). + + +B<World Coordinate Systems> + +If World Coordinate System information is contained in the data file +being filtered, it also is possible to define regions using a sky +coordinate system. Supported systems include: + + + name description + ---- ----------- + PHYSICAL pixel coords of original file using LTM/LTV + IMAGE pixel coords of current file + FK4, B1950 sky coordinate systems + FK5, J2000 sky coordinate systems + GALACTIC sky coordinate systems + ECLIPTIC sky coordinate systems + ICRS currently same as J2000 + LINEAR linear wcs as defined in file + + +In addition, two mosaic coordinate systems have been defined that +utilize the (evolving) IRAF mosaic keywords: + + + name description + ---- ----------- + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV + +Again, to use one of these coordinate systems, the B<global> or +B<local> properties commands are used: + + + global coordsys galactic + + +B<WCS Positions and Sizes> + +In addition to pixels, positional values in a WCS-enabled region can +be specified using sexagesimal or degrees format: + + + position arguments description + ------------------ ----------- + [num] context-dependent (see below) + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + [num]:[num]:[num] hms for 'odd' position arguments + [num]:[num]:[num] dms for 'even' position arguments + [num]h[num]m[num]s explicit hms + [num]d[num]m[num]s explicit dms + + +If ':' is used as sexagesimal separator, the value is considered to be +specifying hours/minutes/seconds if it is the first argument of a +positional pair, and degrees/minutes/seconds for the second argument +of a pair (except for galactic coordinates, which always use degrees): + + + argument description + ----------- ----------- + 10:20:30.0 10 hours, 20 minutes, 30 seconds for 1st positional argument + 10 degrees, 20 minutes, 30 seconds for 2nd positional argument + 10h20m30.0 10 hours, 20 minutes, 30 seconds + 10d20m30.0 10 degrees, 20 minutes, 30 seconds + 10.20d 10.2 degrees + + +Similarly, the units of size values are defined by the formating +character(s) attached to a number: + + + size arguments description + -------------- ----------- + [num] context-dependent (see below) + [num]" arc seconds + [num]' arc minutes + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + + +For example: + + argument description + ----------- ----------- + 10 ten pixels + 10' ten minutes of arc + 10" ten seconds of arc + 10d ten degrees + 10p ten pixels + 0.5r half of a radian + + + +An example of using sky coordinate systems follows: + + + global coordsys B1950 + -box 175.54d 20.01156d 10' 10' + local coordsys J2000 + pie 179.57d 22.4d 0 360 n=4 && annulus 179.57d 22.4d 3' 24' n=5 + + +At the FK4 1950 coordinates 175.54d RA, 20.01156d DEC exclude a 10 +minute by 10 minute box. Then at the FK5 2000 coordinates 179.57d RA +22.4d DEC draw a radial profile regions pattern with 4 quadrants and 5 +annuli ranging from 3 minutes to 24 minutes in diameter. In this +example, the default coordinate system is overridden by the commands +in the regions spec. + +B<NB: The Meaning of Pure Numbers Are Context Sensitive> + + +When a "pure number" (i.e. one without a format directive such as 'd' +for 'degrees') is specified as a position or size, its interpretation +depends on the context defined by the 'coordsys' keyword. In general, +the rule is: + + +All pure numbers have implied units corresponding to the current +coordinate system. + + +If no coordinate system is explicitly specified, the default system is +implicitly assumed to be PHYSICAL. In practice this means that for +IMAGE and PHYSICAL systems, pure numbers are pixels. Otherwise, +for all systems other than LINEAR, pure numbers are degrees. For +LINEAR systems, pure numbers are in the units of the linear system. +This rule covers both positions and sizes. + + +As a corollary, when a sky-formatted number is used with the IMAGE +or PHYSICAL coordinate system (which includes the default case of no +coordsys being specified), the formatted number is assumed to be in +the units of the WCS contained in the current file. If no sky WCS is +specified, an error results. + + +Examples: + + + circle(512,512,10) + ellipse 202.44382d 47.181656d 0.01d 0.02d + + + +In the absence of a specified coordinate system, the circle uses the +default PHYSICAL units of pixels, while the ellipse explicitly uses degrees, +presumably to go with the WCS in the current file. + + + global coordsys=fk5 + global color=green font="system 10 normal" + circle 202.44382 47.181656 0.01 + circle 202.44382 47.181656 10p + ellipse(512p,512p,10p,15p,20) + + + + +Here, the circles use the FK5 units of degrees (except for the +explicit use of pixels in the second radius), while the ellipse +explicitly specifies pixels. The ellipse angle is in degrees. + + +Note that Chandra data format appears to use "coordsys=physical" +implicitly. Therefore, for most Chandra applications, valid regions +can be generated safely by asking ds9 to save/display regions in +pixels using the PHYSICAL coordsys. + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/regdiff.pod b/funtools/doc/pod/regdiff.pod new file mode 100644 index 0000000..e1c38ca --- /dev/null +++ b/funtools/doc/pod/regdiff.pod @@ -0,0 +1,99 @@ +=pod + +=head1 NAME + + + +B<RegDiff:Differences Between Funtools and IRAF Regions> + + + +=head1 SYNOPSIS + + + + +Describes the differences between Funtools/ds9 regions and the old IRAF/PROS +regions. + + + +=head1 DESCRIPTION + + + + + +We have tried to make Funtools regions compatible with their +predecessor, IRAF/PROS regions. For simple regions and simple boolean +algebra between regions, there should be no difference between the two +implementations. The following is a list of differences and +incompatibilities between the two: + + + +=over 4 + + + + + + +=item * + +If a pixel is covered by two different regions expressions, +Funtools assigns the mask value of the B<first> region that +contains that pixel. That is, successive regions B<do not> +overwrite previous regions in the mask, as was the case with the +original PROS regions. This means that one must define overlapping +regions in the reverse order in which they were defined in PROS. If +region N is fully contained within region M, then N should be defined +B<before> M, or else it will be "covered up" by the latter. This +change is necessitated by the use of optimized filter compilation, i.e., +Funtools only tests individual regions until a proper match is made. + + + + +=item * + +The B<PANDA> region has replaced the old PROS syntax in which +a B<PIE> accelerator was combined with an B<ANNULUS> accelerator +using B<AND>. That is, + + ANNULUS(20,20,0,15,n=4) & PIE(20,20,0,360,n=3) + +has been replaced by: + + PANDA(20,20,0,360,3,0,15,4) + +The PROS syntax was inconsistent with the meaning of the B<AND> operator. + + + + +=item * + +The meaning of pure numbers (i.e., without format specifiers) in +regions has been clarified, as has the syntax for specifying coordinate +systems. See the general discussion on +Spatial Region Filtering +for more information. + + + +=back + + + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/pod/reggeometry.pod b/funtools/doc/pod/reggeometry.pod new file mode 100644 index 0000000..0d442d5 --- /dev/null +++ b/funtools/doc/pod/reggeometry.pod @@ -0,0 +1,1156 @@ +=pod + +=head1 NAME + + + +B<RegGeometry: Geometric Shapes in Spatial Region Filtering> + + + +=head1 SYNOPSIS + + + + + +This document describes the geometry of regions available for spatial +filtering in IRAF/PROS analysis. + + + +=head1 DESCRIPTION + + + +B<Geometric shapes> + +Several geometric shapes are used to describe regions. The valid +shapes are: + + + shape: arguments: + ----- ---------------------------------------- + ANNULUS xcenter ycenter inner_radius outer_radius + BOX xcenter ycenter xwidth yheight (angle) + CIRCLE xcenter ycenter radius + ELLIPSE xcenter ycenter xwidth yheight (angle) + FIELD none + LINE x1 y1 x2 y2 + PIE xcenter ycenter angle1 angle2 + POINT x1 y1 + POLYGON x1 y1 x2 y2 ... xn yn + + + +All arguments are real values; integer values are automatically +converted to real where necessary. All angles are in degrees and +specify angles that run counter-clockwise from the positive y-axis. + + +Shapes can be specified using "command" syntax: + + [shape] arg1 arg2 ... + +or using "routine" syntax: + + [shape](arg1, arg2, ...) + +or by any combination of the these. (Of course, the parentheses must +balance and there cannot be more commas than necessary.) The shape +keywords are case-insensitive. Furthermore, any shape can be +specified by a three-character unique abbreviation. For example, one +can specify three circular regions as: + + + "foo.fits[CIRCLE 512 512 50;CIR(128 128, 10);cir(650,650,20)]" + + +(Quotes generally are required to protect the region descriptor +from being processed by the Unix shell.) + + + + +The B<annulus> shape specifies annuli, centered at xcenter, +ycenter, with inner and outer radii (r1, r2). For example, + + ANNULUS 25 25 5 10 + +specifies an annulus centered at 25.0 25.0 with an inner radius of 5.0 and +an outer radius of 10. Assuming (as will be done for all examples in this +document, unless otherwise noted) this shape is used in a mask of size 40x40, +it will look like this: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:....................111111111........... + 33:...................11111111111.......... + 32:.................111111111111111........ + 31:.................111111111111111........ + 30:................11111111111111111....... + 29:...............1111111.....1111111...... + 28:...............111111.......111111...... + 27:...............11111.........11111...... + 26:...............11111.........11111...... + 25:...............11111.........11111...... + 24:...............11111.........11111...... + 23:...............11111.........11111...... + 22:...............111111.......111111...... + 21:...............1111111.....1111111...... + 20:................11111111111111111....... + 19:.................111111111111111........ + 18:.................111111111111111........ + 17:...................11111111111.......... + 16:....................111111111........... + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................ + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + + + + + +The B<box> shape specifies an orthogonally oriented box, +centered at xcenter, ycenter, of size xwidth, yheight. It requires four +arguments and accepts an optional fifth argument to specify a rotation angle. +When the rotation angle is specified (in degrees), the box is rotated by +an angle that runs counter-clockwise from the positive y-axis. + + +The B<box> shape specifies a rotated box, centered at +xcenter, ycenter, of size xwidth, yheight. The box is rotated by an angle +specified in degrees that runs counter-clockwise from the positive y-axis. +If the angle argument is omitted, it defaults to 0. + + + + +The B<circle> shape specifies a circle, centered at xcenter, +ycenter, of radius r. It requires three arguments. + + + + +The B<ellipse> shape specifies an ellipse, centered at +xcenter, ycenter, with y-axis width a and the y-axis length b defined such +that: + + x**2/a**2 + y**2/b**2 = 1 + +Note that a can be less than, equal to, or greater than b. The ellipse +is rotated the specified number of degrees. The rotation is done according +to astronomical convention, counter-clockwise from the positive y-axis. +An ellipse defined by: + + ELLIPSE 20 20 5 10 45 + +will look like this: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:............111111...................... + 26:............11111111.................... + 25:............111111111................... + 24:............11111111111................. + 23:............111111111111................ + 22:............111111111111................ + 21:.............111111111111............... + 20:.............1111111111111.............. + 19:..............111111111111.............. + 18:...............111111111111............. + 17:...............111111111111............. + 16:................11111111111............. + 15:..................111111111............. + 14:...................11111111............. + 13:.....................111111............. + 12:........................................ + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + + + + + + +The B<field> shape specifies the entire field as a +region. It is not usually specified explicitly, but is used implicitly in the +case where no regions are specified, that is, in cases where either a null +string or some abbreviation of the string "none" is input. +B<Field> takes no arguments. + + + + +The B<pie> shape specifies an angular wedge of the entire field, +centered at xcenter, ycenter. The wedge runs between the two specified angles. +The angles are given in degrees, running counter-clockwise from the positive +x-axis. For example, + + PIE 20 20 90 180 + +defines a region from 90 degrees to 180 degrees, i.e., quadrant 2 of the +Cartesian plane. The display of such a region looks like this: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:11111111111111111111.................... + 39:11111111111111111111.................... + 38:11111111111111111111.................... + 37:11111111111111111111.................... + 36:11111111111111111111.................... + 35:11111111111111111111.................... + 34:11111111111111111111.................... + 33:11111111111111111111.................... + 32:11111111111111111111.................... + 31:11111111111111111111.................... + 30:11111111111111111111.................... + 29:11111111111111111111.................... + 28:11111111111111111111.................... + 27:11111111111111111111.................... + 26:11111111111111111111.................... + 25:11111111111111111111.................... + 24:11111111111111111111.................... + 23:11111111111111111111.................... + 22:11111111111111111111.................... + 21:11111111111111111111.................... + 20:........................................ + 19:........................................ + 18:........................................ + 17:........................................ + 16:........................................ + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................ + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + +The pie slice specified is always a counter-clockwise sweep between +the angles, starting at the first angle and ending at the second. Thus: + + PIE 10 15 30 60 + +describes a 30 degree sweep from 2 o'clock to 1 o'clock, while: + + PIE 10 15 60 30 + +describes a 330 degree counter-clockwise sweep from 1 o'clock to 2 o'clock +passing through 12 o'clock (0 degrees). Note in both of these examples that +the center of the slice can be anywhere on the plane. The second mask looks +like this: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:111111111111111111111111................ + 39:11111111111111111111111................. + 38:11111111111111111111111................. + 37:1111111111111111111111.................. + 36:1111111111111111111111.................. + 35:111111111111111111111................... + 34:11111111111111111111.................... + 33:11111111111111111111.................... + 32:1111111111111111111....................1 + 31:1111111111111111111..................111 + 30:111111111111111111.................11111 + 29:111111111111111111................111111 + 28:11111111111111111...............11111111 + 27:1111111111111111..............1111111111 + 26:1111111111111111.............11111111111 + 25:111111111111111............1111111111111 + 24:111111111111111..........111111111111111 + 23:11111111111111.........11111111111111111 + 22:11111111111111........111111111111111111 + 21:1111111111111.......11111111111111111111 + 20:111111111111......1111111111111111111111 + 19:111111111111....111111111111111111111111 + 18:11111111111....1111111111111111111111111 + 17:11111111111..111111111111111111111111111 + 16:1111111111.11111111111111111111111111111 + 15:1111111111111111111111111111111111111111 + 14:1111111111111111111111111111111111111111 + 13:1111111111111111111111111111111111111111 + 12:1111111111111111111111111111111111111111 + 11:1111111111111111111111111111111111111111 + 10:1111111111111111111111111111111111111111 + 9:1111111111111111111111111111111111111111 + 8:1111111111111111111111111111111111111111 + 7:1111111111111111111111111111111111111111 + 6:1111111111111111111111111111111111111111 + 5:1111111111111111111111111111111111111111 + 4:1111111111111111111111111111111111111111 + 3:1111111111111111111111111111111111111111 + 2:1111111111111111111111111111111111111111 + 1:1111111111111111111111111111111111111111 + +The pie slice goes to the edge of the field. To limit its scope, pie +usually is is combined with other shapes, such as circles and annuli, +using boolean operations. (See below and in "help regalgebra"). + + +Pie Performance Notes: + +Pie region processing time is proportional to the size of the image, +and not the size of the region. This is because the pie shape is the +only infinite length shape, and we essentially must check all y rows +for inclusion (unlike other regions, where the y limits can be +calculated beforehand). Thus, pie can run very slowly on large images. +In particular, it will run MUCH more slowly than the panda shape in +image-based region operations (such as funcnts). We recommend use of +panda over pie where ever possible. + + +If you must use pie, always try to put it last in a boolean && +expression. The reason for this is that the filter code is optimized +to exit as soon as the result is know. Since pie is the slowest +region, it is better to avoid executing it if another region can decide +the result. Consider, for example, the difference in time required to +process a Chandra ACIS file when a pie and circle are combined in +two different orders: + + + time ./funcnts nacis.fits "circle 4096 4096 100 && pie 4096 4096 10 78" +2.87u 0.38s 0:35.08 9.2% + + time ./funcnts nacis.fits "pie 4096 4096 10 78 && circle 4096 4096 100 " +89.73u 0.36s 1:03.50 141.8% + + + +Black-magic performance note: + + +Panda region processing uses a B<quick test> pie region instead of +the normal pie region when combining its annulus and pie shapes. This +B<qtpie> shape differs from the normal pie in that it utilizes the +y limits from the previous region with which it is combined. In a +panda shape, which is a series of annuli combined with pies, the +processing time is thus reduced to that of the annuli. + + +You can use the qtpie shape instead of pie in cases where you are +combining pie with another shape using the && operator. This will +cause the pie limits to be set using limits from the other shape, and +will speed up the processing considerably. For example, the above +execution of funcnts can be improved considerably using this technique: + + + time ./funcnts nacis.fits "circle 4096 4096 100 && qtpie 4096 4096 10 78" +4.66u 0.33s 0:05.87 85.0% + + + +We emphasize that this is a quasi-documented feature and might change in +the future. The qtpie shape is not recognized by ds9 or other programs. + + + + +The B<line> shape allows single pixels in a line between (x1,y1) and +(x2,y2) to be included or excluded. For example: + + LINE (5,6, 24,25) + +displays as: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:.......................1................ + 24:......................1................. + 23:.....................1.................. + 22:....................1................... + 21:...................1.................... + 20:..................1..................... + 19:.................1...................... + 18:................1....................... + 17:...............1........................ + 16:..............1......................... + 15:.............1.......................... + 14:............1........................... + 13:...........1............................ + 12:..........1............................. + 11:.........1.............................. + 10:........1............................... + 9:.......1................................ + 8:......1................................. + 7:.....1.................................. + 6:....1................................... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + + + + + +The B<point> shape allows single pixels to be included or +excluded. Although the (x,y) values are real numbers, they are truncated +to integer and the corresponding pixel is included or excluded, as specified. + + +Several points can be put in one region declaration; unlike the +original IRAF implementation, each now is given a different region mask value. +This makes it easier, for example, for funcnts to determine the number of +photons in the individual pixels. For example, + + POINT (5,6, 10,11, 20,20, 35,30) + +will give the different region mask values to all four points, as shown below: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:..................................4..... + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:........................................ + 24:........................................ + 23:........................................ + 22:........................................ + 21:........................................ + 20:...................3.................... + 19:........................................ + 18:........................................ + 17:........................................ + 16:........................................ + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................ + 11:.........2.............................. + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:....1................................... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + + + + + +The B<polygon> shape specifies a polygon with vertices +(x1, y1) ... (xn, yn). The polygon is closed automatically: one should +not specify the last vertex to be the same as the first. Any number of +vertices are allowed. For example, the following polygon defines a +right triangle as shown below: + + POLYGON (10,10, 10,30, 30,30) + + +looks like this: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:..........11111111111111111111.......... + 29:..........1111111111111111111........... + 28:..........111111111111111111............ + 27:..........11111111111111111............. + 26:..........1111111111111111.............. + 25:..........111111111111111............... + 24:..........11111111111111................ + 23:..........1111111111111................. + 22:..........111111111111.................. + 21:..........11111111111................... + 20:..........1111111111.................... + 19:..........111111111..................... + 18:..........11111111...................... + 17:..........1111111....................... + 16:..........111111........................ + 15:..........11111......................... + 14:..........1111.......................... + 13:..........111........................... + 12:..........11............................ + 11:..........1............................. + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + +Note that polygons can get twisted upon themselves if edge lines +cross. Thus: + + POL (10,10, 20,20, 20,10, 10,20) + +will produce an area which is two triangles, like butterfly wings, as shown +below: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:........................................ + 24:........................................ + 23:........................................ + 22:........................................ + 21:........................................ + 20:........................................ + 19:..........1........1.................... + 18:..........11......11.................... + 17:..........111....111.................... + 16:..........1111..1111.................... + 15:..........1111111111.................... + 14:..........1111..1111.................... + 13:..........111....111.................... + 12:..........11......11.................... + 11:..........1........1.................... + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + + + + + +The following are combinations of pie with different shapes +(called "panda" for "Pie AND Annulus") allow for easy specification of +radial sections: + + shape: arguments: + ----- --------- + PANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + BPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # box + EPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # ellipse + + +The B<panda> (B<P>ies B<AND> B<A>nnuli) shape can be +used to create combinations of pie and annuli markers. It is analogous +to a Cartesian product on those shapes, i.e., the result is several +shapes generated by performing a boolean AND between pies and +annuli. Thus, the panda and cpanda specify combinations of annulus and +circle with pie, respectively and give identical results. The bpanda +combines box and pie, while epanda combines ellipse and pie. + + +Consider the example shown below: + + PANDA(20,20, 0,360,3, 0,15,4) + +Here, 3 pie slices centered at 20, 20 are combined with 4 annuli, also +centered at 20, 20. The result is a mask with 12 regions (displayed in +base 16 to save characters): + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:..............44444444444............... + 33:............444444444444444............. + 32:...........88444444444444444............ + 31:.........888844443333344444444.......... + 30:........88888833333333333444444......... + 29:........88888733333333333344444......... + 28:.......8888877733333333333344444........ + 27:......888887777332222233333344444....... + 26:......888877777622222222333334444....... + 25:.....88887777766622222222333334444...... + 24:.....88887777666622222222233334444...... + 23:.....88887777666651111222233334444...... + 22:.....88877776666551111122223333444...... + 21:.....88877776666555111122223333444...... + 20:.....888777766665559999aaaabbbbccc...... + 19:.....888777766665559999aaaabbbbccc...... + 18:.....888777766665599999aaaabbbbccc...... + 17:.....88887777666659999aaaabbbbcccc...... + 16:.....888877776666aaaaaaaaabbbbcccc...... + 15:.....888877777666aaaaaaaabbbbbcccc...... + 14:......8888777776aaaaaaaabbbbbcccc....... + 13:......888887777bbaaaaabbbbbbccccc....... + 12:.......88888777bbbbbbbbbbbbccccc........ + 11:........888887bbbbbbbbbbbbccccc......... + 10:........888888bbbbbbbbbbbcccccc......... + 9:.........8888ccccbbbbbcccccccc.......... + 8:...........88ccccccccccccccc............ + 7:............ccccccccccccccc............. + 6:..............ccccccccccc............... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + + + + + +Several regions with different mask values can be combined in the +same mask. This supports comparing data from the different regions. +(For information on how to combine different shapes into a single +region, see "help regalgebra".) For example, consider the following +set of regions: + + ANNULUS 25 25 5 10 + ELLIPSE 20 20 5 10 315 + BOX 15 15 5 10 + +The resulting mask will look as follows: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:....................111111111........... + 33:...................11111111111.......... + 32:.................111111111111111........ + 31:.................111111111111111........ + 30:................11111111111111111....... + 29:...............1111111.....1111111...... + 28:...............111111.......111111...... + 27:...............11111.222222..11111...... + 26:...............111112222222..11111...... + 25:...............111112222222..11111...... + 24:...............111112222222..11111...... + 23:...............111112222222..11111...... + 22:...............111111222222.111111...... + 21:..............211111112222.1111111...... + 20:............322211111111111111111....... + 19:............32222111111111111111........ + 18:............22222111111111111111........ + 17:............222222211111111111.......... + 16:............22222222111111111........... + 15:............222222222................... + 14:............22222222.................... + 13:............222222...................... + 12:............33333....................... + 11:............33333....................... + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ + +Note that when a pixel is in 2 or more regions, it is arbitrarily +assigned to a one of the regions in question (often based on how a +give C compiler optimizes boolean expressions). + + +B<Region accelerators> + + +Two types of \fBaccelerators, to simplify region specification, +are provided as natural extensions to the ways shapes are described. +These are: extended lists of parameters, specifying multiple regions, +valid for annulus, box, circle, ellipse, pie, and points; and +B<n=>, valid for annulus, box, circle, ellipse, and pie (not +point). In both cases, one specification is used to define several +different regions, that is, to define shapes with different mask +values in the region mask. + + +The following regions accept B<accelerator> syntax: + + shape arguments + ----- ------------------------------------------ + ANNULUS xcenter ycenter radius1 radius2 ... radiusn + ANNULUS xcenter ycenter inner_radius outer_radius n=[number] + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + BOX xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + ELLIPSE xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + PIE xcenter ycenter angle1 angle2 (angle3) (angle4) (angle5) ... + PIE xcenter ycenter angle1 angle2 (n=[number]) + POINT x1 y1 x2 y2 ... xn yn + +Note that the circle accelerators are simply aliases for the annulus +accelerators. + + +For example, several annuli at the same center can be specified in one +region expression by specifying more than two radii. If B<N> +radii are specified, then B<N>-1 annuli result, with the outer +radius of each preceding annulus being the inner radius of the +succeeding annulus. Each annulus is considered a separate region, and +is given a separate mask value. For example, + + ANNULUS 20 20 0 2 5 10 15 20 + +specifies five different annuli centered at 20 20, and is equivalent to: + + ANNULUS 20.0 20.0 0 2 + ANNULUS 20.0 20.0 2 5 + ANNULUS 20.0 20.0 5 10 + ANNULUS 20.0 20.0 10 15 + ANNULUS 20.0 20.0 15 20 + +The mask is shown below: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:.............5555555555555.............. + 38:...........55555555555555555............ + 37:.........555555555555555555555.......... + 36:........55555555555555555555555......... + 35:......555555555555555555555555555....... + 34:.....55555555544444444444555555555...... + 33:....5555555544444444444444455555555..... + 32:....5555555444444444444444445555555..... + 31:...555555444444444444444444444555555.... + 30:..55555544444444444444444444444555555... + 29:..55555544444443333333334444444555555... + 28:.5555554444444333333333334444444555555.. + 27:.5555544444433333333333333344444455555.. + 26:555555444444333333333333333444444555555. + 25:555554444443333333333333333344444455555. + 24:555554444433333332222233333334444455555. + 23:555554444433333322222223333334444455555. + 22:555554444433333222222222333334444455555. + 21:555554444433333222111222333334444455555. + 20:555554444433333222111222333334444455555. + 19:555554444433333222111222333334444455555. + 18:555554444433333222222222333334444455555. + 17:555554444433333322222223333334444455555. + 16:555554444433333332222233333334444455555. + 15:555554444443333333333333333344444455555. + 14:555555444444333333333333333444444555555. + 13:.5555544444433333333333333344444455555.. + 12:.5555554444444333333333334444444555555.. + 11:..55555544444443333333334444444555555... + 10:..55555544444444444444444444444555555... + 9:...555555444444444444444444444555555.... + 8:....5555555444444444444444445555555..... + 7:....5555555544444444444444455555555..... + 6:.....55555555544444444444555555555...... + 5:......555555555555555555555555555....... + 4:........55555555555555555555555......... + 3:.........555555555555555555555.......... + 2:...........55555555555555555............ + 1:.............5555555555555.............. + + + +For boxes and ellipses, if an odd number of arguments is specified, +then the last argument is assumed to be an angle. Otherwise, the +angle is assumed to be zero. For example: + + ellipse 20 20 3 5 6 10 9 15 12 20 45 + +specifies an 3 ellipses at a 45 degree angle: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........33333333........................ + 35:......333333333333...................... + 34:.....3333333333333333................... + 33:....333333333333333333.................. + 32:....33333332222233333333................ + 31:...3333332222222222333333............... + 30:...33333222222222222233333.............. + 29:...333332222222222222223333............. + 28:...3333222222211112222223333............ + 27:...33332222211111111222223333........... + 26:...333322222111111111122223333.......... + 25:...3333222211111111111122223333......... + 24:....3332222111111..1111122223333........ + 23:....333322211111.....11112222333........ + 22:....33332222111.......11112223333....... + 21:.....33322221111.......11122223333...... + 20:.....33332221111.......11112223333...... + 19:.....33332222111.......11112222333...... + 18:......33332221111.......11122223333..... + 17:.......33322221111.....111112223333..... + 16:.......3333222211111..1111112222333..... + 15:........3333222211111111111122223333.... + 14:.........333322221111111111222223333.... + 13:..........33332222211111111222223333.... + 12:...........3333222222111122222223333.... + 11:............333322222222222222233333.... + 10:.............33333222222222222233333.... + 9:..............3333332222222222333333.... + 8:...............33333333222223333333..... + 7:.................333333333333333333..... + 6:..................3333333333333333...... + 5:.....................333333333333....... + 4:.......................33333333......... + 3:........................................ + 2:........................................ + 1:........................................ + +Note in the above example that the lower limit is not part of the +region for boxes, circles, and ellipses. This makes circles and annuli +equivalent, i.e.: + + circle 20 20 5 10 15 20 + annulus 20 20 5 10 15 20 + +both give the following region mask: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:.............3333333333333.............. + 38:...........33333333333333333............ + 37:.........333333333333333333333.......... + 36:........33333333333333333333333......... + 35:......333333333333333333333333333....... + 34:.....33333333322222222222333333333...... + 33:....3333333322222222222222233333333..... + 32:....3333333222222222222222223333333..... + 31:...333333222222222222222222222333333.... + 30:..33333322222222222222222222222333333... + 29:..33333322222221111111112222222333333... + 28:.3333332222222111111111112222222333333.. + 27:.3333322222211111111111111122222233333.. + 26:333333222222111111111111111222222333333. + 25:333332222221111111111111111122222233333. + 24:33333222221111111.....11111112222233333. + 23:3333322222111111.......1111112222233333. + 22:333332222211111.........111112222233333. + 21:333332222211111.........111112222233333. + 20:333332222211111.........111112222233333. + 19:333332222211111.........111112222233333. + 18:333332222211111.........111112222233333. + 17:3333322222111111.......1111112222233333. + 16:33333222221111111.....11111112222233333. + 15:333332222221111111111111111122222233333. + 14:333333222222111111111111111222222333333. + 13:.3333322222211111111111111122222233333.. + 12:.3333332222222111111111112222222333333.. + 11:..33333322222221111111112222222333333... + 10:..33333322222222222222222222222333333... + 9:...333333222222222222222222222333333.... + 8:....3333333222222222222222223333333..... + 7:....3333333322222222222222233333333..... + 6:.....33333333322222222222333333333...... + 5:......333333333333333333333333333....... + 4:........33333333333333333333333......... + 3:.........333333333333333333333.......... + 2:...........33333333333333333............ + 1:.............3333333333333.............. + + + + +As a final example, specifying several angles in one pie slice +expression is equivalent to specifying several separate slices with +the same center. As with the annulus, if B<N> angles are +specified, then B<N>-1 slices result, with the ending angle of +each preceding slice being the starting angle of the succeeding slice. +Each slice is considered a separate region, and is given a separate +mask value. For example, + + PIE 12 12 315 45 115 270 + +specifies three regions as shown below: + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:2222222222222222222222222222222222222222 + 39:2222222222222222222222222222222222222221 + 38:2222222222222222222222222222222222222211 + 37:2222222222222222222222222222222222222111 + 36:2222222222222222222222222222222222221111 + 35:3222222222222222222222222222222222211111 + 34:3222222222222222222222222222222222111111 + 33:3322222222222222222222222222222221111111 + 32:3322222222222222222222222222222211111111 + 31:3332222222222222222222222222222111111111 + 30:3332222222222222222222222222221111111111 + 29:3333222222222222222222222222211111111111 + 28:3333222222222222222222222222111111111111 + 27:3333322222222222222222222221111111111111 + 26:3333322222222222222222222211111111111111 + 25:3333322222222222222222222111111111111111 + 24:3333332222222222222222221111111111111111 + 23:3333332222222222222222211111111111111111 + 22:3333333222222222222222111111111111111111 + 21:3333333222222222222221111111111111111111 + 20:3333333322222222222211111111111111111111 + 19:3333333322222222222111111111111111111111 + 18:3333333332222222221111111111111111111111 + 17:3333333332222222211111111111111111111111 + 16:3333333333222222111111111111111111111111 + 15:3333333333222221111111111111111111111111 + 14:3333333333322211111111111111111111111111 + 13:3333333333322111111111111111111111111111 + 12:33333333333.1111111111111111111111111111 + 11:3333333333331111111111111111111111111111 + 10:333333333333.111111111111111111111111111 + 9:333333333333..11111111111111111111111111 + 8:333333333333...1111111111111111111111111 + 7:333333333333....111111111111111111111111 + 6:333333333333.....11111111111111111111111 + 5:333333333333......1111111111111111111111 + 4:333333333333.......111111111111111111111 + 3:333333333333........11111111111111111111 + 2:333333333333.........1111111111111111111 + 1:333333333333..........111111111111111111 + + + +The annulus, box, circle, ellipse, and pie shapes also accept an +B<n=[int]> syntax for specifying multiple regions. The +B<n=[int]>syntax interprets the previous (shape-dependent) +arguments as lower and upper limits for the region and creates n +shapes with evenly spaced boundaries. For example, if B<n=[int]> +is specified in an annulus, the two immediately preceding radii +(B<rn> and B<rm>) are divided into B<int> annuli, such +that the inner radius of the first is B<rn> and the outer radius +of the last is B<rm>. For example, + + ANNULUS 20 20 5 20 n=3 + +is equivalent to: + + ANNULUS 20 20 5 10 15 20 + +If this syntax is used with an ellipse or box, then the two preceding +pairs of values are taken to be lower and upper limits for a set of +ellipses or boxes. A circle uses the two preceding arguments for upper +and lower radii. For pie, the two preceding angles are divided into n +wedges such that the starting angle of the first is the lower bound +and the ending angle of the last is the upper bound. In all cases, +the B<n=[int]> syntax allows any single alphabetic character +before the "=", i.e, i=3, z=3, etc. are all equivalent. + + +Also note that for boxes and ellipses, the optional angle argument is +always specified after the B<n=[int]> syntax. For example: + + ellipse 20 20 4 6 16 24 n=3 45 + +specifies 3 elliptical regions at an angle of 45 degrees: + + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........33333333........................ + 39:.....33333333333333..................... + 38:....33333333333333333................... + 37:...33333333333333333333................. + 36:..33333333333333333333333............... + 35:.3333333333222223333333333.............. + 34:3333333322222222222233333333............ + 33:33333332222222222222223333333........... + 32:333333222222222222222222333333.......... + 31:3333322222222222222222222333333......... + 30:33333222222222111122222222333333........ + 29:333332222222111111112222222333333....... + 28:3333222222211111111111222222333333...... + 27:3333222222111111111111112222233333...... + 26:33332222221111111111111112222233333..... + 25:33332222211111111.111111112222233333.... + 24:333322222111111......111111222223333.... + 23:333322222111111.......111112222233333... + 22:33333222221111.........11111222223333... + 21:333332222211111.........11112222233333.. + 20:.33332222211111.........11111222223333.. + 19:.33333222221111.........111112222233333. + 18:..33332222211111.........11112222233333. + 17:..333332222211111.......111111222233333. + 16:...333322222111111......111111222223333. + 15:...333332222211111111.111111112222233333 + 14:....333332222211111111111111122222233333 + 13:.....33333222221111111111111122222233333 + 12:.....33333322222211111111111222222233333 + 11:......3333332222222111111112222222333333 + 10:.......333333222222221111222222222333333 + 9:........33333322222222222222222222333333 + 8:.........333333222222222222222222333333. + 7:..........33333332222222222222223333333. + 6:...........3333333322222222222233333333. + 5:.............3333333333222223333333333.. + 4:..............33333333333333333333333... + 3:................33333333333333333333.... + 2:..................33333333333333333..... + 1:....................33333333333333...... + + + +Both the variable argument syntax and the B<n=[int]> syntax must +occur alone in a region descriptor (aside from the optional angle for +boxes and ellipses). They cannot be combined. Thus, it is not valid +to precede or follow an B<n=[int]> accelerator with more angles or +radii, as in this example: + + # INVALID -- one too many angles before a=5 ... + # and no angles are allowed after a=5 + PIE 12 12 10 25 50 a=5 85 135 + +Instead, use three separate specifications, such as: + + PIE 12 12 10 25 + PIE 12 12 25 50 a=5 + PIE 12 12 85 135 + +The original (IRAF) implementation of region filtering permitted this +looser syntax, but we found it caused more confusion than it was worth +and therefore removed it. + + +NB: Accelerators may be combined with other shapes in a boolean +expression in any order. (This is a change starting with funtools +v1.1.1. Prior to this release, the accelerator shape had to be +specified last). The actual region mask id values returned depend on the +order in which the shapes are specified, although the total number of +pixels or rows that pass the filter will be consistent. For this +reason, use of accelerators in boolean expressions is discouraged in +programs such as funcnts, where region mask id values are used +to count events or image pixels. + + +[All region masks displayed in this document were generated using the +B<fundisp> routine and the undocumented "mask=all" argument (with +spaced removed using sed ): + + fundisp "funtools/funtest/test40.fits[ANNULUS 25 25 5 10]" mask=all |\ + sed 's/ //g' + +Note that you must supply an image of the appropriate size -- in this case, +a FITS image of dimension 40x40 is used.] + + + +=head1 SEE ALSO + + + +See funtools(n) for a list of Funtools help pages + + + +=cut diff --git a/funtools/doc/programs.html b/funtools/doc/programs.html new file mode 100644 index 0000000..00d4dc7 --- /dev/null +++ b/funtools/doc/programs.html @@ -0,0 +1,3497 @@ +<HTML> +<HEAD> +<TITLE>Funtools Programs</TITLE> +</HEAD> +<BODY> +<H2>Funtools Programs</H2> + +<H2>Summary</H2> + +<P> +<PRE> + +<A HREF="./programs.html#funcalc">funcalc</A> [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] [-u] <iname> [oname [columns]] + +<A HREF="./programs.html#funcen">funcen</A> [-i] [-n iter] [-t tol] [-v lev] <iname> <region> + +<A HREF="./programs.html#funcnts">funcnts</A> [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_cnts] + +<A HREF="./programs.html#funcone">funcone</A> [-n] [-x|-X|-j|-J] [[-l|-L] list] [-r ra_col] [-d dec_col] <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns] + +<A HREF="./programs.html#fundisp">fundisp</A> [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n] + +<A HREF="./programs.html#funhead">funhead</A> [-a] [-l] [-s] [-t] [-L] <iname> [oname ename] + +<A HREF="./programs.html#funhist">funhist</A> [-n|-w|-T] <iname> [column] [[lo_edge:hi_edge:]bins] + +<A HREF="./programs.html#funimage">funimage</A> [-a] [-l] [-p x|y] <iname> <oname> [bitpix=n] + +<A HREF="./programs.html#funindex">funindex</A> <iname> <key> [oname] + +<A HREF="./programs.html#funjoin">funjoin</A> [switches] <ifile1> <ifile2> ... <ifilen> <ofile> + +<A HREF="./programs.html#funmerge">funmerge</A> <iname1> <iname2> ... <oname> + +<A HREF="./programs.html#funsky">funsky</A> [switches] <iname1> [<lname2> <col1> <col2>] + +<A HREF="./programs.html#funtable">funtable</A> [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns] + +<A HREF="./programs.html#funtbl">funtbl</A> [-c cols] [-h] [-n table] [-p prog] [-s sep] [-T] <iname> +</PRE> + +<!-- =defdoc funcalc funcalc 1 --> + +<!-- =section funcalc NAME --> +<H2><A NAME="funcalc">funcalc - Funtools calculator (for binary tables)</A></H2> +<!-- =section funcalc SYNOPSIS --> +<B> +<PRE> +funcalc [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] <iname> [oname [columns]] +</PRE> +</B> + +<!-- =section funcalc OPTIONS --> +<P> +<PRE> + -a argstr # user arguments to pass to the compiled program + -e expr # funcalc expression + -f file # file containing funcalc expression + -l libs # libs to add to link command + -n # output generated code instead of compiling and executing + -p prog # generate named program, no execution + -u # die if any variable is undeclared (don't auto-declare) +</PRE> + +<!-- =section funcalc DESCRIPTION --> +<P> +<B>funcalc</B> is a calculator program that allows arbitrary +expressions to be constructed, compiled, and executed on columns in a +Funtools table (FITS binary table or raw event file). It works by +integrating user-supplied expression(s) into a template C program, +then compiling and executing the program. <B>funcalc</B> expressions +are C statements, although some important simplifications (such +as automatic declaration of variables) are supported. + +<P> +<B>funcalc</B> expressions can be specified in three ways: on the +command line using the <B>-e [expression]</B> switch, in a file using +the <B>-f [file]</B> switch, or from stdin (if neither <B>-e</B> nor +<B>-f</B> is specified). Of course a file containing <B>funcalc</B> +expressions can be read from stdin. + +<P> +Each invocation of <B>funcalc</B> requires an input Funtools table +file to be specified as the first command line argument. The output +Funtools table file is the second optional argument. It is needed only +if an output FITS file is being created (i.e., in cases where the +<B>funcalc</B> expression only prints values, no output file is +needed). If input and output file are both specified, a third optional +argument can specify the list of columns to activate (using +<A HREF="./library.html#funcolumnactivate">FunColumnActivate()</A>). Note +that <B>funcalc</B> determines whether or not to generate code for +writing an output file based on the presence or absence of an +output file argument. + +<P> +A <B>funcalc</B> expression executes on each row of a table and +consists of one or more C statements that operate on the columns of +that row (possibly using temporary variables). Within an expression, +reference is made to a column of the <B>current</B> row using the C +struct syntax <B>cur->[colname]</B>, e.g. cur->x, cur->pha, etc. +Local scalar variables can be defined using C declarations at very the +beginning of the expression, or else they can be defined automatically +by <B>funcalc</B> (to be of type double). Thus, for example, a swap of +columns x and y in a table can be performed using either of the +following equivalent <B>funcalc</B> expressions: + +<PRE> + double temp; + temp = cur->x; + cur->x = cur->y; + cur->y = temp; +</PRE> + +or: + +<PRE> + temp = cur->x; + cur->x = cur->y; + cur->y = temp; +</PRE> + +When this expression is executed using a command such as: +<PRE> + funcalc -f swap.expr itest.ev otest.ev +</PRE> +the resulting file will have values of the x and y columns swapped. + +<P> +By default, the data type of the variable for a column is the same as +the data type of the column as stored in the file. This can be changed +by appending ":[dtype]" to the first reference to that column. In the +example above, to force x and y to be output as doubles, specify the +type 'D' explicitly: +<PRE> + temp = cur->x:D; + cur->x = cur->y:D; + cur->y = temp; +</PRE> + +Data type specifiers follow standard FITS table syntax for defining +columns using TFORM: +<UL> +<LI>A: ASCII characters +<LI>B: unsigned 8-bit char +<LI>I: signed 16-bit int +<LI>U: unsigned 16-bit int (not standard FITS) +<LI>J: signed 32-bit int +<LI>V: unsigned 32-bit int (not standard FITS) +<LI>E: 32-bit float +<LI>D: 64-bit float +<LI>X: bits (treated as an array of chars) +</UL> +Note that only the first reference to a column should contain the +explicit data type specifier. + +<P> +Of course, it is important to handle the data type of the columns +correctly. One of the most frequent cause of error in <B>funcalc</B> +programming is the implicit use of the wrong data type for a column in +expression. For example, the calculation: +<PRE> + dx = (cur->x - cur->y)/(cur->x + cur->y); +</PRE> +usually needs to be performed using floating point arithmetic. In +cases where the x and y columns are integers, this can be done by +reading the columns as doubles using an explicit type specification: +<PRE> + dx = (cur->x:D - cur->y:D)/(cur->x + cur->y); +</PRE> + +Alternatively, it can be done using C type-casting in the expression: +<PRE> + dx = ((double)cur->x - (double)cur->y)/((double)cur->x + (double)cur->y); +</PRE> + +<P> +In addition to accessing columns in the current row, reference also +can be made to the <B>previous</B> row using <B>prev->[colname]</B>, +and to the <B>next</B> row using <B>next->[colname]</B>. Note that if +<B>prev->[colname]</B> is specified in the <B>funcalc</B> +expression, the very first row is not processed. If +<B>next->[colname]</B> is specified in the <B>funcalc</B> +expression, the very last row is not processed. In this way, +<B>prev</B> and <B>next</B> are guaranteed always to point to valid +rows. For example, to print out the values of the current x column +and the previous y column, use the C fprintf function in a +<B>funcalc</B> expression: +<PRE> + fprintf(stdout, "%d %d\n", cur->x, prev->y); +</PRE> + +<P> +New columns can be specified using the same <B>cur->[colname]</B> +syntax by appending the column type (and optional tlmin/tlmax/binsiz +specifiers), separated by colons. For example, cur->avg:D will define +a new column of type double. Type specifiers are the same those +used above to specify new data types for existing columns. + +<P> +For example, to create and output a new column that is the average value of the +x and y columns, a new "avg" column can be defined: +<PRE> + cur->avg:D = (cur->x + cur->y)/2.0 +</PRE> +Note that the final ';' is not required for single-line expressions. + +<P> +As with FITS TFORM data type specification, the column data type +specifier can be preceded by a numeric count to define an array, e.g., +"10I" means a vector of 10 short ints, "2E" means two single precision +floats, etc. A new column only needs to be defined once in a +<B>funcalc</B> expression, after which it can be used without +re-specifying the type. This includes reference to elements of a +column array: + +<PRE> + cur->avg[0]:2D = (cur->x + cur->y)/2.0; + cur->avg[1] = (cur->x - cur->y)/2.0; +</PRE> + +<P> +The 'X' (bits) data type is treated as a char array of dimension +(numeric_count/8), i.e., 16X is processed as a 2-byte char array. Each +8-bit array element is accessed separately: +<PRE> + cur->stat[0]:16X = 1; + cur->stat[1] = 2; +</PRE> +Here, a 16-bit column is created with the MSB is set to 1 and the LSB set to 2. + +<P> +By default, all processed rows are written to the specified output +file. If you want to skip writing certain rows, simply execute the C +"continue" statement at the end of the <B>funcalc</B> expression, +since the writing of the row is performed immediately after the +expression is executed. For example, to skip writing rows whose +average is the same as the current x value: + +<PRE> + cur->avg[0]:2D = (cur->x + cur->y)/2.0; + cur->avg[1] = (cur->x - cur->y)/2.0; + if( cur->avg[0] == cur->x ) + continue; +</PRE> + +<P> +If no output file argument is specified on the <B>funcalc</B> command +line, no output file is opened and no rows are written. This is useful +in expressions that simply print output results instead of generating +a new file: +<PRE> + fpv = (cur->av3:D-cur->av1:D)/(cur->av1+cur->av2:D+cur->av3); + fbv = cur->av2/(cur->av1+cur->av2+cur->av3); + fpu = ((double)cur->au3-cur->au1)/((double)cur->au1+cur->au2+cur->au3); + fbu = cur->au2/(double)(cur->au1+cur->au2+cur->au3); + fprintf(stdout, "%f\t%f\t%f\t%f\n", fpv, fbv, fpu, fbu); +</PRE> +In the above example, we use both explicit type specification +(for "av" columns) and type casting (for "au" columns) to ensure that +all operations are performed in double precision. + +<P> +When an output file is specified, the selected input table is +processed and output rows are copied to the output file. Note that +the output file can be specified as "stdout" in order to write the +output rows to the standard output. If the output file argument is +passed, an optional third argument also can be passed to specify which +columns to process. + +<p> +In a FITS binary table, it sometimes is desirable to copy all of the +other FITS extensions to the output file as well. This can be done by +appending a '+' sign to the name of the extension in the input file +name. See <B>funtable</B> for a related example. + +<P> +<B>funcalc</B> works by integrating the user-specified expression +into a template C program called <A HREF="./tabcalc.c">tabcalc.c</A>. +The completed program then is compiled and executed. Variable +declarations that begin the <B>funcalc</B> expression are placed in +the local declaration section of the template main program. All other +lines are placed in the template main program's inner processing +loop. Other details of program generation are handled +automatically. For example, column specifiers are analyzed to build a +C struct for processing rows, which is passed to +<A HREF="./library.html#funcolumnselect">FunColumnSelect()</A> and used +in <A HREF="./library.html#funtablerowget">FunTableRowGet()</A>. If +an unknown variable is used in the expression, resulting in a +compilation error, the program build is retried after defining the +unknown variable to be of type double. + +<P> +Normally, <B>funcalc</B> expression code is added to +<B>funcalc</B> row processing loop. It is possible to add code +to other parts of the program by placing this code inside +special directives of the form: +<PRE> + [directive name] + ... code goes here ... + end +</PRE> + +The directives are: +<UL> +<LI><B>global</B> add code and declarations in global space, before the main routine. + +<LI><B>local</B> add declarations (and code) just after the local declarations in +main + +<LI><B>before</B> add code just before entering the main row processing loop + +<LI><B>after</B> add code just after exiting the main row processing loop +</UL> + +Thus, the following <B>funcalc</B> expression will declare global +variables and make subroutine calls just before and just after the +main processing loop: +<PRE> + global + double v1, v2; + double init(void); + double finish(double v); + end + before + v1 = init(); + end + ... process rows, with calculations using v1 ... + after + v2 = finish(v1); + if( v2 < 0.0 ){ + fprintf(stderr, "processing failed %g -> %g\n", v1, v2); + exit(1); + } + end +</PRE> +Routines such as init() and finish() above are passed to the generated +program for linking using the <B>-l [link directives ...]</B> +switch. The string specified by this switch will be added to the link +line used to build the program (before the funtools library). For +example, assuming that init() and finish() are in the library +libmysubs.a in the /opt/special/lib directory, use: +<PRE> + funcalc -l "-L/opt/special/lib -lmysubs" ... +</PRE> + +<p> +User arguments can be passed to a compiled funcalc program using a string +argument to the "-a" switch. The string should contain all of the +user arguments. For example, to pass the integers 1 and 2, use: +<PRE> + funcalc -a "1 2" ... +</PRE> +The arguments are stored in an internal array and are accessed as +strings via the ARGV(n) macro. For example, consider the following +expression: +<PRE> + local + int pmin, pmax; + end + + before + pmin=atoi(ARGV(0)); + pmax=atoi(ARGV(1)); + end + + if( (cur->pha >= pmin) && (cur->pha <= pmax) ) + fprintf(stderr, "%d %d %d\n", cur->x, cur->y, cur->pha); +</PRE> +This expression will print out x, y, and pha values for all rows in which +the pha value is between the two user-input values: +<PRE> + funcalc -a '1 12' -f foo snr.ev'[cir 512 512 .1]' + 512 512 6 + 512 512 8 + 512 512 5 + 512 512 5 + 512 512 8 + + funcalc -a '5 6' -f foo snr.ev'[cir 512 512 .1]' + 512 512 6 + 512 512 5 + 512 512 5 +</PRE> + +<P> +Note that it is the user's responsibility to ensure that the correct +number of arguments are passed. The ARGV(n) macro returns a NULL if a +requested argument is outside the limits of the actual number of args, +usually resulting in a SEGV if processed blindly. To check the +argument count, use the ARGC macro: +<PRE> + local + long int seed=1; + double limit=0.8; + end + + before + if( ARGC >= 1 ) seed = atol(ARGV(0)); + if( ARGC >= 2 ) limit = atof(ARGV(1)); + srand48(seed); + end + + if ( drand48() > limit ) continue; +</PRE> + +<P> +The macro WRITE_ROW expands to the FunTableRowPut() call that writes +the current row. It can be used to write the row more than once. In +addition, the macro NROW expands to the row number currently being +processed. Use of these two macros is shown in the following example: +<PRE> + if( cur->pha:I == cur->pi:I ) continue; + a = cur->pha; + cur->pha = cur->pi; + cur->pi = a; + cur->AVG:E = (cur->pha+cur->pi)/2.0; + cur->NR:I = NROW; + if( NROW < 10 ) WRITE_ROW; +</PRE> + +<p> +If the <B>-p [prog]</B> switch is specified, the expression is not +executed. Rather, the generated executable is saved with the specified +program name for later use. + +<P> +If the <B>-n</B> switch is specified, the expression is not +executed. Rather, the generated code is written to stdout. This is +especially useful if you want to generate a skeleton file and add your +own code, or if you need to check compilation errors. Note that the +comment at the start of the output gives the compiler command needed +to build the program on that platform. (The command can change from +platform to platform because of the use of different libraries, +compiler switches, etc.) + +<P> +As mentioned previously, <B>funcalc</B> will declare a scalar +variable automatically (as a double) if that variable has been used +but not declared. This facility is implemented using a sed script +named <A HREF="./funcalc.sed">funcalc.sed</A>, which processes the +compiler output to sense an undeclared variable error. This script +has been seeded with the appropriate error information for gcc, and for +cc on Solaris, DecAlpha, and SGI platforms. If you find that automatic +declaration of scalars is not working on your platform, check this sed +script; it might be necessary to add to or edit some of the error +messages it senses. + +<P> +In order to keep the lexical analysis of <B>funcalc</B> expressions +(reasonably) simple, we chose to accept some limitations on how +accurately C comments, spaces, and new-lines are placed in the +generated program. In particular, comments associated with local +variables declared at the beginning of an expression (i.e., not in a +<B>local...end</B> block) will usually end up in the inner loop, not +with the local declarations: +<PRE> + /* this comment will end up in the wrong place (i.e, inner loop) */ + double a; /* also in wrong place */ + /* this will be in the the right place (inner loop) */ + if( cur->x:D == cur->y:D ) continue; /* also in right place */ + a = cur->x; + cur->x = cur->y; + cur->y = a; + cur->avg:E = (cur->x+cur->y)/2.0; +</PRE> +Similarly, spaces and new-lines sometimes are omitted or added in a +seemingly arbitrary manner. Of course, none of these stylistic +blemishes affect the correctness of the generated code. + +<P> +Because <B>funcalc</B> must analyze the user expression using the data +file(s) passed on the command line, the input file(s) must be opened +and read twice: once during program generation and once during +execution. As a result, it is not possible to use stdin for the +input file: <B>funcalc</B> cannot be used as a filter. We will +consider removing this restriction at a later time. + +<P> +Along with C comments, <B>funcalc</B> expressions can have one-line +internal comments that are not passed on to the generated C +program. These internal comment start with the <B>#</B> character and +continue up to the new-line: +<PRE> + double a; # this is not passed to the generated C file + # nor is this + a = cur->x; + cur->x = cur->y; + cur->y = a; + /* this comment is passed to the C file */ + cur->avg:E = (cur->x+cur->y)/2.0; +</PRE> + +<P> +As previously mentioned, input columns normally are identified by +their being used within the inner event loop. There are rare cases +where you might want to read a column and process it outside the main +loop. For example, qsort might use a column in its sort comparison +routine that is not processed inside the inner loop (and therefore not +implicitly specified as a column to be read). To ensure that such a +column is read by the event loop, use the <b>explicit</b> keyword. +The arguments to this keyword specify columns that should be read into +the input record structure even though they are not mentioned in the +inner loop. For example: + <PRE> + explicit pi pha +</PRE> +will ensure that the pi and pha columns are read for each row, +even if they are not processed in the inner event loop. The <b>explicit</b> +statement can be placed anywhere. + +<P> +Finally, note that <B>funcalc</B> currently works on expressions +involving FITS binary tables and raw event files. We will consider +adding support for image expressions at a later point, if there is +demand for such support from the community. + +<!-- =defdoc funcen funcen 1 --> + +<!-- =section funcen NAME --> +<H2><A NAME="funcen">funcen - find centroid (for binary tables)</A></H2> +<!-- =section funcen SYNOPSIS --> +<B> +<PRE> +funcen [-i] [-n iter] [-t tol] [-v lev] <iname> <region> +</PRE> +</B> + +<!-- =section funcen OPTIONS --> +<P> +<PRE> + -i # use image filtering (default: event filtering) + -n iter # max number of iterations (default: 0) + -t tol # pixel tolerance distance (default: 1.0) + -v [0,1,2,3] # output verbosity level (default: 0) +</PRE> + +<!-- =section funcen DESCRIPTION --> +<P> +<B>funcen</B> iteratively calculates the centroid position within one +or more regions of a Funtools table (FITS binary table or raw event +file). Starting with an input table, an initial region specification, +and an iteration count, the program calculates the average x and y +position within the region and then uses this new position as the +region center for the next iteration. Iteration terminates when the +maximum number of iterations is reached or when the input tolerance +distance is met for that region. A count of events in the final region +is then output, along with the pixel position value (and, where +available, WCS position). + +<P> +The first argument to the program specifies the Funtools table file to +process. Since the file must be read repeatedly, a value of "stdin" +is not permitted when the number of iterations is non-zero. Use +<A HREF="./files.html">Funtools Bracket Notation</A> to specify FITS +extensions and filters. + +<P> +The second required argument is the initial region descriptor. Multiple +regions are permitted. However, compound regions (accelerators, +variable argument regions and regions connected via boolean algebra) +are not permitted. Points and polygons also are illegal. These +restrictions might be lifted in a future version, if warranted. + +<p> +The <B>-n</B> (iteration number) switch specifies the maximum number of +iterations to perform. The default is 0, which means that the program will +simply count and display the number of events in the initial region(s). +Note that when iterations is 0, the data can be input via stdin. + +<p> +The <B>-t</B> (tolerance) switch specifies a floating point tolerance +value. If the distance between the current centroid position value and +the last position values is less than this value, iteration terminates. +The default value is 1 pixel. + +<p> +The <B>-v</B> (verbosity) switch specifies the verbosity level of the +output. The default is 0, which results in a single line of output for +each input region consisting of the following values: +<pre> + counts x y [ra dec coordsys] +</pre> +The last 3 WCS values are output if WCS information is available in the +data file header. Thus, for example: +<pre> + [sh] funcen -n 0 snr.ev "cir 505 508 5" + 915 505.00 508.00 345.284038 58.870920 j2000 + + [sh] funcen -n 3 snr.ev "cir 505 508 5" + 1120 504.43 509.65 345.286480 58.874587 j2000 +</pre> +The first example simply counts the number of events in the initial region. +The second example iterates the centroid calculation three times to determine +a final "best" position. + +<p> +Higher levels of verbosity obviously imply more verbose output. At +level 1, the output essentially contains the same information as level +0, but with keyword formatting: + + [sh] funcen -v 1 -n 3 snr.ev "cir 505 508 5" + event_file: snr.ev + initial_region: cir 505 508 5 + tolerance: 1.0000 + iterations: 1 + + events: 1120 + x,y(physical): 504.43 509.65 + ra,dec(j2000): 345.286480 58.874587 + final_region1: cir 504.43 509.65 5 +</pre> +Level 2 outputs results from intermediate calculations as well. + +<p> +Ordinarily, region filtering is performed using analytic (event) +filtering, i.e. that same style of filtering as is performed by +<b>fundisp</b> and <b>funtable</b>. Use the <b>-i</b> switch to specify image +filtering, i.e. the same style filtering as is performed by <b>funcnts</b>. +Thus, you can perform a quick calculation of counts in regions, using +either the analytic or image filtering method, by specifying the + <b>-n 0</b> and optional <b>-i</b> switches. These two method often +give different results because of how boundary events are processed: +<pre> + [sh] funcen snr.ev "cir 505 508 5" + 915 505.00 508.00 345.284038 58.870920 j2000 + + [sh] funcen -i snr.ev "cir 505 508 5" + 798 505.00 508.00 345.284038 58.870920 j2000 +</pre> +See <A HREF="./regbounds.html">Region Boundaries</A> for more information +about how boundaries are calculated using these two methods. + +<!-- =defdoc funcnts funcnts 1 --> + +<!-- =section funcnts NAME --> +<H2><A NAME="funcnts">funcnts - count photons in specified regions, with bkgd subtraction</A></H2> + +<!-- =section funcnts SYNOPSIS --> +<B> +<PRE> +funcnts [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_value] +</PRE> +</B> + +<!-- =section funcnts OPTIONS --> +<P> +<PRE> + -e "source_exposure[;bkgd_exposure]" + # source (bkgd) FITS exposure image using matching files + -w "source_exposure[;bkgd_exposure]" + # source (bkgd) FITS exposure image using WCS transform + -t "source_timecorr[;bkgd_timecorr]" + # source (bkgd) time correction value or header parameter name + -g # output using nice g format + -G # output using %.14g format (maximum precision) + -i "[column;]int1;int2..." # column-based intervals + -m # match individual source and bkgd regions + -p # output in pixels, even if wcs is present + -r # output inner/outer radii (and angles) for annuli (and pandas) + -s # output summed values + -v "scol[;bcol]" # src and bkgd value columns for tables + -T # output in starbase/rdb format + -z # output regions with zero area +</B> +</PRE> + +<!-- =section funcnts DESCRIPTION --> +<P> +<B>funcnts</B> counts photons in the specified source regions and +reports the results for each region. Regions are specified using the +<A HREF="./regions.html">Spatial Region Filtering</A> mechanism. +Photons are also counted in the specified bkgd regions applied to the +same data file or a different data file. (Alternatively, a constant +background value in counts/pixel**2 can be specified.) The bkgd regions +are either paired one-to-one with source regions or pooled and +normalized by area, and then subtracted from the source counts in each +region. Displayed results include the bkgd-subtracted counts in each +region, as well as the error on the counts, the area in +each region, and the surface brightness (cnts/area**2) calculated for +each region. + +<P> +The first argument to the program specifies the FITS input image, array, or +raw event file to process. If "stdin" is specified, data are read from +the standard input. Use <A HREF="./files.html">Funtools Bracket +Notation</A> to specify FITS extensions, image sections, and filters. + +<P> +The optional second argument is the source region descriptor. If no +region is specified, the entire field is used. + +<P> +The background arguments can take one of two forms, depending on +whether a separate background file is specified. If the source +file is to be used for background as well, the third argument can be +either the background region, or a constant value denoting background +cnts/pixel. Alternatively, the third argument can be a background +data file, in which case the fourth argument is the background region. +If no third argument is specified, a constant value of 0 is used +(i.e., no background). + +<P> +In summary, the following command arguments are valid: +<PRE> + [sh] funcnts sfile # counts in source file + [sh] funcnts sfile sregion # counts in source region + [sh] funcnts sfile sregion bregion # bkgd reg. is from source file + [sh] funcnts sfile sregion bvalue # bkgd reg. is constant + [sh] funcnts sfile sregion bfile bregion # bkgd reg. is from separate file +</PRE> + +<P> +NB: unlike other Funtools programs, source and background regions are +specified as separate arguments on the command line, rather than being +placed inside brackets as part of the source and background filenames. +This is because regions in funcnts are not simply used as data +filters, but also are used to calculate areas, exposure, etc. If you +put the source region inside the brackets (i.e. use it simply as a +filter) rather than specifying it as argument two, the program still +will only count photons that pass the region filter. However, the area +calculation will be performed on the whole field, since field() is the +default source region. This rarely is the desired behavior. On the +other hand, with FITS binary tables, it often is useful to put a column +filter in the filename brackets, so that only events matching the +column filter are counted inside the region. + +<P> +For example, to extract the counts within a radius of 22 pixels from the +center of the FITS binary table snr.ev and subtract the background determined +from the same image within an annulus of radii 50-100 pixels: +<PRE> + [sh] funcnts snr.ev "circle(502,512,22)" "annulus(502,512,50,100)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001 + + + # the following source and background components were used: + source region(s) + ---------------- + circle(502,512,22) + + reg counts pixels + ---- ------------ --------- + 1 4382.000 1513 + + background region(s) + -------------------- + annulus(502,512,50,100) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 +</PRE> +The area units for the output columns labeled "area", "surf_bri" +(surface brightness) and "surf_err" will be given either in +arc-seconds (if appropriate WCS information is in the data file +header(s)) or in pixels. If the data file has WCS info, but you do not +want arc-second units, use the <B>-p</B> switch to force output in +pixels. Also, regions having zero area are not normally included in +the primary (background-subtracted) table, but are included in the +secondary source and bkgd tables. If you want these regions to be +included in the primary table, use the <B>-z</B> switch. + +<P> +Note that a simple sed command will extract the background-subtracted results +for further analysis: +<PRE> + [sh] cat funcnts.sed + 1,/---- .*/d + /^$/,$d + + [sh] sed -f funcnts.sed funcnts.out + 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001 +</PRE> + +<P> +If separate source and background files are specified, <B>funcnts</B> will +attempt to normalize the the background area so that the background +pixel size is the same as the source pixel size. This normalization +can only take place if the appropriate WCS information is contained in +both files (e.g. degrees/pixel values in CDELT). If either +file does not contain the requisite size information, the normalization +is not performed. In this case, it is the user's responsibility to +ensure that the pixel sizes are the same for the two files. + +<P> +Normally, if more than one background region is specified, <B>funcnts</B> +will combine them all into a single region and use this background +region to produce the background-subtracted results for each source +region. The <B>-m</B> (match multiple backgrounds) switch tells +<B>funcnts</B> to make a one to one correspondence between background and +source regions, instead of using a single combined background region. +For example, the default case is to combine 2 background +regions into a single region and then apply that region to each of the +source regions: + +<PRE> + [sh] funcnts snr.ev "annulus(502,512,0,22,n=2)" "annulus(502,512,50,100,n=2)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 + 2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,22,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region(s) + -------------------- + annulus(502,512,50,100,n=2) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 +</PRE> +Note that the basic region filter rule "each photon is counted once +and no photon is counted more than once" still applies when using The +<B>-m</B> to match background regions. That is, if two background +regions overlap, the overlapping pixels will be counted in only one of +them. In a worst-case scenario, if two background regions are the same +region, the first will get all the counts and area and the second +will get none. + +<P> +Using the <B>-m</B> switch causes <B>funcnts</B> to use each of the two +background regions independently with each of the two source regions: + +<PRE> + [sh] funcnts -m snr.ev "annulus(502,512,0,22,n=2)" "ann(502,512,50,100,n=2)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 3087.015 56.954 150.985 2.395 23872.00 0.129 0.002 + 2 755.959 34.295 388.041 5.672 72959.99 0.010 0.000 + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,22,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region(s) + -------------------- + ann(502,512,50,100,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3975.000 9820 + 2 4681.000 13752 +</PRE> + +<P> +Note that most floating point quantities are displayed using "f" +format. You can change this to "g" format using the <B>-g</B> +switch. This can be useful when the counts in each pixel is very +small or very large. If you want maximum precision and don't care +about the columns lining up nicely, use <B>-G</B>, which outputs +all floating values as %.14g. + +<P> +When counting photons using the annulus and panda (pie and annuli) +shapes, it often is useful to have access to the radii (and panda +angles) for each separate region. The <B>-r</B> switch will add radii +and angle columns to the output table: + +<PRE> + [sh] funcnts -r snr.ev "annulus(502,512,0,22,n=2)" "ann(502,512,50,100,n=2)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + # radii: arcsecs + # angles: degrees + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err radius1 radius2 angle1 angle2 + ---- ------------ --------- ------------ --------- --------- --------- --------- --------- --------- --------- --------- + 1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 0.00 88.00 NA NA + 2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 88.00 176.00 NA NA + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,22,n=2) + + reg counts pixels + ---- ------------ --------- + 1 3238.000 373 + 2 1144.000 1140 + + background region(s) + -------------------- + ann(502,512,50,100,n=2) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 +</PRE> + +<P> +Radii are given in units of pixels or arc-seconds (depending on the +presence of WCS info), while the angle values (when present) are in +degrees. These columns can be used to plot radial profiles. For +example, the script <B>funcnts.plot</B> in the funtools +distribution) will plot a radial profile using gnuplot (version 3.7 or +above). A simplified version of this script is shown below: + +<PRE> + #!/bin/sh + + if [ x"$1" = xgnuplot ]; then + if [ x`which gnuplot 2>/dev/null` = x ]; then + echo "ERROR: gnuplot not available" + exit 1 + fi + awk ' + BEGIN{HEADER=1; DATA=0; FILES=""; XLABEL="unknown"; YLABEL="unknown"} + HEADER==1{ + if( $1 == "#" && $2 == "data" && $3 == "file:" ){ + if( FILES != "" ) FILES = FILES "," + FILES = FILES $4 + } + else if( $1 == "#" && $2 == "radii:" ){ + XLABEL = $3 + } + else if( $1 == "#" && $2 == "surf_bri:" ){ + YLABEL = $3 + } + else if( $1 == "----" ){ + printf "set nokey; set title \"funcnts(%s)\"\n", FILES + printf "set xlabel \" radius(%s)\"\n", XLABEL + printf "set ylabel \"surf_bri(%s)\"\n", YLABEL + print "plot \"-\" using 3:4:6:7:8 with boxerrorbars" + HEADER = 0 + DATA = 1 + next + } + } + DATA==1{ + if( NF == 12 ){ + print $9, $10, ($9+$10)/2, $7, $8, $7-$8, $7+$8, $10-$9 + } + else{ + exit + } + } + ' | gnuplot -persist - 1>/dev/null 2>&1 + + elif [ x"$1" = xds9 ]; then + awk ' + BEGIN{HEADER=1; DATA=0; XLABEL="unknown"; YLABEL="unknown"} + HEADER==1{ + if( $1 == "#" && $2 == "data" && $3 == "file:" ){ + if( FILES != "" ) FILES = FILES "," + FILES = FILES $4 + } + else if( $1 == "#" && $2 == "radii:" ){ + XLABEL = $3 + } + else if( $1 == "#" && $2 == "surf_bri:" ){ + YLABEL = $3 + } + else if( $1 == "----" ){ + printf "funcnts(%s) radius(%s) surf_bri(%s) 3\n", FILES, XLABEL, YLABEL + HEADER = 0 + DATA = 1 + next + } + } + DATA==1{ + if( NF == 12 ){ + print $9, $7, $8 + } + else{ + exit + } + } + ' + else + echo "funcnts -r ... | funcnts.plot [ds9|gnuplot]" + exit 1 + fi +</PRE> + +Thus, to run <B>funcnts</B> and plot the results using gnuplot (version 3.7 +or above), use: +<PRE> + funcnts -r snr.ev "annulus(502,512,0,50,n=5)" ... | funcnts.plot gnuplot +</PRE> + +<P> +The <B>-s</B> (sum) switch causes <B>funcnts</B> to produce an +additional table of summed (integrated) background subtracted values, +along with the default table of individual values: + +<PRE> + [sh] funcnts -s snr.ev "annulus(502,512,0,50,n=5)" "annulus(502,512,50,100)" + # source + # data file: snr.ev + # degrees/pix: 0.00222222 + # background + # data file: snr.ev + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # summed background-subtracted results + upto net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003 + 2 3776.817 65.254 457.183 4.914 79679.98 0.047 0.001 + 3 4025.492 71.972 1031.508 11.087 179775.96 0.022 0.000 + 4 4185.149 80.109 1840.851 19.786 320831.94 0.013 0.000 + 5 4415.540 90.790 2873.460 30.885 500799.90 0.009 0.000 + + + # background-subtracted results + reg counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003 + 2 895.818 35.423 345.182 3.710 60159.99 0.015 0.001 + 3 248.675 29.345 574.325 6.173 100095.98 0.002 0.000 + 4 159.657 32.321 809.343 8.699 141055.97 0.001 0.000 + 5 230.390 37.231 1032.610 11.099 179967.96 0.001 0.000 + + + # the following source and background components were used: + source region(s) + ---------------- + annulus(502,512,0,50,n=5) + + reg counts pixels sumcnts sumpix + ---- ------------ --------- ------------ --------- + 1 2993.000 305 2993.000 305 + 2 1241.000 940 4234.000 1245 + 3 823.000 1564 5057.000 2809 + 4 969.000 2204 6026.000 5013 + 5 1263.000 2812 7289.000 7825 + + background region(s) + -------------------- + annulus(502,512,50,100) + + reg counts pixels + ---- ------------ --------- + all 8656.000 23572 +</PRE> + +<P> +The <B>-t</B> and <B>-e</B> switches can be used to apply timing and +exposure corrections, respectively, to the data. Please note that +these corrections are meant to be used qualitatively, since +application of more accurate correction factors is a complex and +mission-dependent effort. The algorithm for applying these simple +corrections is as follows: +<PRE> + C = Raw Counts in Source Region + Ac= Area of Source Region + Tc= Exposure time for Source Data + Ec= Average exposure in Source Region, from exposure map + + B= Raw Counts in Background Region + Ab= Area of Background Region + Tb= (Exposure) time for Background Data + Eb= Average exposure in Background Region, from exposure map +</PRE> +Then, Net Counts in Source region is +<PRE> + Net= C - B * (Ac*Tc*Ec)/(Ab*Tb*Eb) +</PRE> +with the standard propagation of errors for the Error on Net. +The net rate would then be +<PRE> + Net Rate = Net/(Ac*Tc*Ec) +</PRE> +The average exposure in each region is calculated by summing up the +pixel values in the exposure map for the given region and then +dividing by the number of pixels in that region. Exposure maps often +are generated at a block factor > 1 (e.g., block 4 means that each +exposure pixel contains 4x4 pixels at full resolution) and +<B>funcnts</B> will deal with the blocking automatically. Using the +<B>-e</B> switch, you can supply both source and background exposure +files (separated by ";"), if you have separate source and background +data files. If you do not supply a background exposure file to go with +a separate background data file, <B>funcnts</B> assumes that exposure +already has been applied to the background data file. In addition, it +assumes that the error on the pixels in the background data file is +zero. + +<P> +NB: The <B>-e</B> switch assumes that the exposure map overlays the +image file <B>exactly</B>, except for the block factor. Each pixel in +the image is scaled by the block factor to access the corresponding +pixel in the exposure map. If your exposure map does not line up +exactly with the image, <B>do not use</B> the <B>-e</B> exposure +correction. In this case, it still is possible to perform exposure +correction <B>if</B> both the image and the exposure map have valid +WCS information: use the <B>-w</B> switch so that the transformation +from image pixel to exposure pixel uses the WCS information. That is, +each pixel in the image region will be transformed first from image +coordinates to sky coordinates, then from sky coordinates to exposure +coordinates. Please note that using <B>-w</B> can increase the time +required to process the exposure correction considerably. + +<P> +A time correction can be applied to both source and +background data using the <B>-t</B> switch. The value for the correction can +either be a numeric constant or the name of a header parameter in +the source (or background) file: +<PRE> + [sh] funcnts -t 23.4 ... # number for source + [sh] funcnts -t "LIVETIME;23.4" ... # param for source, numeric for bkgd +</PRE> +When a time correction is specified, it is applied to the net counts +as well (see algorithm above), so that the units of surface brightness +become cnts/area**2/sec. + +<P> +The <B>-i</B> (interval) switch is used to run <b>funcnts</b> on multiple +column-based intervals with only a single pass through the data. It is +equivalent to running <b>funcnts</b> several times with a different column +filter added to the source and background data each time. For each +interval, the full <b>funcnts</b> output is generated, with a linefeed +character (^L) inserted between each run. In addition, the output for +each interval will contain the interval specification in its header. +Intervals are very useful for generating X-ray hardness ratios +efficiently. Of course, they are only supported when the input data +are contained in a table. + +<P> +Two formats are supported for interval specification. The most general +format is semi-colon-delimited list of filters to be used as intervals: +<PRE> + funcnts -i "pha=1:5;pha=6:10;pha=11:15" snr.ev "circle(502,512,22)" ... +</PRE> +Conceptually, this will be equivalent to running <b>funcnts</b> three times: +<PRE> + funcnts snr.ev'[pha=1:5]' "circle(502,512,22)" + funcnts snr.ev'[pha=6:10]' "circle(502,512,22)" + funcnts snr.ev'[pha=11:15]' "circle(502,512,22)" +</PRE> +However, using the <B>-i</B> switch will require only one pass through +the data. + +<P> +Note that complex filters can be used to specify intervals: +<PRE> + funcnts -i "pha=1:5&&pi=4;pha=6:10&&pi=5;pha=11:15&&pi=6" snr.ev ... +</PRE> +The program simply runs the data through each filter in turn and generates +three <b>funcnts</b> outputs, separated by the line-feed character. + +<P> +In fact, although the intent is to support intervals for hardness ratios, +the specified filters do not have to be intervals at all. Nor does one +"interval" filter have to be related to another. For example: +<PRE> + funcnts -i "pha=1:5;pi=6:10;energy=11:15" snr.ev "circle(502,512,22)" ... +</PRE> +is equivalent to running <b>funcnts</b> three times with unrelated filter +specifications. + +<P> +A second interval format is supported for the simple case in which a +single column is used to specify multiple homogeneous intervals for +that column. In this format, a column name is specified first, +followed by intervals: +<PRE> + funcnts -i "pha;1:5;6:10;11:15" snr.ev "circle(502,512,22)" ... +</PRE> +This is equivalent to the first example, but requires less typing. The +<b>funcnts</b> program will simply prepend "pha=" before each of the specified +intervals. (Note that this format does not contain the "=" character in +the column argument.) + +<P> +Ordinarily, when <b>funcnts</b> is run on a FITS binary table (or a +raw event table), one integral count is accumulated for each row +(event) contained within a given region. The <B>-v "scol[;bcol]"</B> +(value column) switch will accumulate counts using the value from the +specified column for the given event. If only a single column is +specified, it is used for both the source and background regions. Two +separate columns, separated by a semi-colon, can be specified for source +and background. The special token '$none' can be used to specify that +a value column is to be used for one but not the other. For example, +'pha;$none' will use the pha column for the source but use integral +counts for the background, while '$none;pha' will do the converse. +If the value column is of type logical, then the value used will be 1 +for T and 0 for F. Value columns are used, for example, to integrate +probabilities instead of integral counts. + +<P> +If the <B>-T</B> (rdb table) switch is used, the output will conform +to starbase/rdb data base format: tabs will be inserted between +columns rather than spaces and line-feed will be inserted between +tables. + +<P> +Finally, note that <B>funcnts</B> is an image program, even though it +can be run directly on FITS binary tables. This means that image +filtering is applied to the rows in order to ensure that the same +results are obtained regardless of whether a table or the equivalent +binned image is used. Because of this, however, the number of counts +found using <B>funcnts</B> can differ from the number of events found +using row-filter programs such as <B>fundisp</B> or <B>funtable</B> +For more information about these difference, see the discussion of +<A HREF="./regbounds.html">Region Boundaries</A>. + +<!-- =defdoc funcone funcone 1 --> + +<!-- =section funcone NAME --> +<H2><A NAME="funcone">funcone - cone search of a binary table containing RA, Dec columns</A></H2> + +<!-- =section funcone SYNOPSIS --> +<B> +<PRE> +funcone <switches> <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns] +</PRE> +</B> + +<!-- =section funcone OPTIONS --> +<P> +<PRE> + -d deccol:[hdr] # Dec column name, units (def: DEC:d) + -j # join columns from list file + -J # join columns from list file, output all rows + -l listfile # read centers and radii from a list + -L listfile # read centers and radii from a list, output list rows + -n # don't use cone limits as a filter + -r racol:[hdr] # RA column name, units (def: RA:h) + -x # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols + -X # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols, output all rows +</PRE> + +<!-- =section funcone DESCRIPTION --> +<P> +Funcone performs a cone search on the RA and Dec columns of a FITS +binary table. The distance from the center RA, Dec position to the RA, +Dec in each row in the table is calculated. Rows whose distance is +less than the specified radius are output. + +<P> +The first argument to the program specifies the FITS file, raw event +file, or raw array file. If "stdin" is specified, data are read from +the standard input. Use <A HREF="./files.html">Funtools Bracket +Notation</A> to specify FITS extensions, and filters. The second +argument is the output FITS file. If "stdout" is specified, the FITS +binary table is written to the standard output. + +<P> +The third and fourth required arguments are the RA and Dec center +position. By default, RA is specified in hours while Dec is specified +in degrees. You can change the units of either of these by appending +the character "d" (degrees), "h" (hours) or "r" (radians). Sexagesimal +notation is supported, with colons or spaces separating hms and dms. +(When using spaces, please ensure that the entire string is quoted.) + +<P> +The fifth required argument is the radius of the cone search. By default, +the radius value is given in degrees. The units can be changed by appending +the character "d" (degrees), "r" (radians), "'" (arc minutes) or +'"' (arc seconds). + +<P> +By default, all +columns of the input file are copied to the output file. Selected +columns can be output using an optional sixth argument in the form: +<PRE> + "column1 column1 ... columnN" +</PRE> +A seventh argument allows you to output selected columns from the list +file when <B>-j</B> switch is used. Note that the RA and Dec columns +used in the cone calculation must not be de-selected. + +<P> +Also by default, the RA and Dec column names are named "RA" and "Dec", +and are given in units of hours and degrees respectively. You can +change both the name and the units using the -r [RA] and/or -d [Dec] +switches. Once again, one of "h", "d", or "r" is appended to the +column name to specify units but in this case, there must be a colon ":" +between the name and the unit specification. + +<P> +If the <B>-l [listfile]</B> switch is used, then one or more of the +center RA, center Dec, and radius can be taken from a list file (which +can be a FITS table or an ASCII column text file). In this case, the +third (center RA), fourth (center Dec), and fifth (radius) command +line arguments can either be a column name in the list file (if that +parameter varies) or else a numeric value (if that parameter is +static). When a column name is specified for the RA, Dec, or radius, +you can append a colon followed by "h", "d", or "r" to specify units +(also ' and " for radius). The cone search algorithm is run once for +each row in the list, taking RA, Dec, and radius values from the +specified columns or from static numeric values specified on the +command line. + +<P> +When using a list, all valid rows from each iteration are written to a +single output file. Use the <B>-x</B> switch to help delineate which +line of the list file was used to produce the given output row(s). +This switch causes the values for the center RA, Dec, radius, and row +number to be appended to the output file, in columns called RA_CEN, +DEC_CEN, RAD_CEN and CONE_KEY, respectively. Alternatively, the +<B>-j</B> (join) switch will append all columns from the list row to +the output row (essentially a join of the list row and input row), +along with the CONE_KEY row number. These two switches are mutually +exclusive. + +<P> +The <b>-X</b> and <b>-J</b> switches write out the same data as their +lower case counterparts for each row satisfying a cone search. In +addition, these switches also write out rows from the event file that +do not satisfy any cone search. In such cases, that CONE_KEY column +will be given a value of -1 and the center and list position information +will be set to zero for the given row. Thus, all rows of the input +event file are guaranteed to be output, with rows satisfying at least +one cone search having additional search information. + +<p> +The <b>-L</b> switch acts similarly to the <b>-l</b> switch in that it +takes centers from a list file. However, it also implicitly sets the +-j switch, so that output rows are the join of the input event row and +the center position row. In addition, this switch also writes out all +center position rows for which no event satisfies the cone search +criteria of that row. The CONE_KEY column will be given a value of -2 +for center rows that were not close to any data row and the event +columns will be zeroed out for such rows. In this way, all centers +rows are guaranteed to be output at least once. + +<p> +If any of "all row" switches (<b>-X</b>, <b>-J</b>, or <b>-L</b>) are +specified, then a new column named JSTAT is added to the output table. +The positive values in this column indicate the center position row number +(starting from 1) in the list file that this data row successful matched +in a cone search. A value of -1 means that the data row did not match +any center position. A value of -2 means that the center position was +not matched by any data row. + +<P> +Given a center position and radius, the cone search algorithm +calculates limit parameters for a box enclosing the specified cone, +and only tests rows whose positions values lie within those limits. +For small files, the overhead associated with this cone limit +filtering can cause the program to run more slowly than if all events +were tested. You can turn off cone limit filtering using the <B>-n</B> +switch to see if this speeds up the processing (especially useful when +processing a large list of positions). + +<P> +For example, the default cone search uses columns "RA" and "Dec" in hours +and degrees (respectively) and RA position in hours, Dec and radius in degrees: +<PRE> + funone in.fits out.fits 23.45 34.56 0.01 +</PRE> +To specify the RA position in degrees: +<PRE> + funcone in.fits out.fits 23.45d 34.56 0.01 +</PRE> +To get RA and Dec from a list but use a static value for radius (and +also write identifying info for each row in the list): +<PRE> + funcone -x -l list.txt in.fits out.fits MYRA MYDec 0.01 +</PRE> +User specified columns in degrees, RA position in hours (sexagesimal +notation), Dec position in degrees (sexagesimal notation) and radius +in arc minutes: +<PRE> + funcone -r myRa:d -d myDec in.fits out.fits 12:30:15.5 30:12 15' +</PRE> + +<!-- =defdoc fundisp fundisp 1 --> + +<!-- =section fundisp NAME --> +<H2><A NAME="fundisp">fundisp - display data in a Funtools data file</A></H2> + +<!-- =section fundisp SYNOPSIS --> +<B> +<PRE> +fundisp [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n] +</PRE> +</B> + +<!-- =section fundisp OPTIONS --> +<P> +<PRE> + -f # format string for display + -l # display image as a list containing the columns X, Y, VAL + -n # don't output header + -F [c] # use specified character as column separator (def: space) + -T # output in rdb/starbase format (tab separators) +</PRE> + +<!-- =section fundisp DESCRIPTION --> +<P> +<B>fundisp</B> displays the data in the specified +<A HREF="./files.html">FITS Extension</A> +and/or +<A HREF="./files.html#sections">Image Section</A> +of a FITS file, or in a +<A HREF="./files.html#sections">Section</A> +of a non-FITS array or raw event file. +<P> +The first argument to the program specifies the FITS input image, array, or +raw event file to display. If "stdin" is specified, data are read from +the standard input. Use <A HREF="./files.html">Funtools Bracket +Notation</A> to specify FITS extensions, image sections, and filters. + +<P> +If the data being displayed are columns (either in a FITS binary table +or a raw event file), the individual rows are listed. Filters can be +added using bracket notation. Thus: +<PRE> + [sh] fundisp "test.ev[time-(int)time>.15]" + X Y PHA PI TIME DX DY + ------- ------- ------- --------- ---------------- ---------- ---------- + 10 8 10 8 17.1600 8.50 10.50 + 9 9 9 9 17.1600 9.50 9.50 + 10 9 10 9 18.1600 9.50 10.50 + 10 9 10 9 18.1700 9.50 10.50 + 8 10 8 10 17.1600 10.50 8.50 + 9 10 9 10 18.1600 10.50 9.50 + 9 10 9 10 18.1700 10.50 9.50 + 10 10 10 10 19.1600 10.50 10.50 + 10 10 10 10 19.1700 10.50 10.50 + 10 10 10 10 19.1800 10.50 10.50 +</PRE> +[NB: The FITS binary table test file test.ev, as well as the FITS +image test.fits, are contained in the funtools funtest directory.] + +<P> +When a table is being displayed using <B>fundisp</B>, a second optional +argument can be used to specify the columns to display. For example: +<PRE> + [sh] fundisp "test.ev[time-(int)time>=.99]" "x y time" + X Y TIME + -------- -------- --------------------- + 5 -6 40.99000000 + 4 -5 59.99000000 + -1 0 154.99000000 + -2 1 168.99000000 + -3 2 183.99000000 + -4 3 199.99000000 + -5 4 216.99000000 + -6 5 234.99000000 + -7 6 253.99000000 +</PRE> + +<P> +The special column <B>$REGION</B> can be specified to display the +region id of each row: +<PRE> + [sh $] fundisp "test.ev[time-(int)time>=.99&&annulus(0 0 0 10 n=3)]" 'x y time $REGION' + X Y TIME REGION + -------- -------- --------------------- ---------- + 5 -6 40.99000000 3 + 4 -5 59.99000000 2 + -1 0 154.99000000 1 + -2 1 168.99000000 1 + -3 2 183.99000000 2 + -4 3 199.99000000 2 + -5 4 216.99000000 2 + -6 5 234.99000000 3 + -7 6 253.99000000 3 +</PRE> +<P> +Here only rows with the proper fractional time and whose position also is +within one of the three annuli are displayed. +<P> +Columns can be excluded from display using a minus sign before the +column: +<PRE> + [sh $] fundisp "test.ev[time-(int)time>=.99]" "-time" + X Y PHA PI DX DY + -------- -------- -------- ---------- ----------- ----------- + 5 -6 5 -6 5.50 -6.50 + 4 -5 4 -5 4.50 -5.50 + -1 0 -1 0 -1.50 0.50 + -2 1 -2 1 -2.50 1.50 + -3 2 -3 2 -3.50 2.50 + -4 3 -4 3 -4.50 3.50 + -5 4 -5 4 -5.50 4.50 + -6 5 -6 5 -6.50 5.50 + -7 6 -7 6 -7.50 6.50 +</PRE> +All columns except the time column are displayed. +<P> +The special column <B>$N</B> can be specified to display the +ordinal value of each row. Thus, continuing the previous example: +<PRE> + fundisp "test.ev[time-(int)time>=.99]" '-time $n' + X Y PHA PI DX DY N + ------- -------- -------- ---------- ----------- ----------- ---------- + 5 -6 5 -6 5.50 -6.50 337 + 4 -5 4 -5 4.50 -5.50 356 + -1 0 -1 0 -1.50 0.50 451 + -2 1 -2 1 -2.50 1.50 465 + -3 2 -3 2 -3.50 2.50 480 + -4 3 -4 3 -4.50 3.50 496 + -5 4 -5 4 -5.50 4.50 513 + -6 5 -6 5 -6.50 5.50 531 + -7 6 -7 6 -7.50 6.50 550 +</PRE> +Note that the column specification is enclosed in single quotes to protect +'$n' from begin expanded by the shell. + +<P> +In general, the rules for activating and de-activating columns are: +<UL> +<LI> If only exclude columns are specified, then all columns but +the exclude columns will be activated. +<LI> If only include columns are specified, then only the specified columns +are activated. +<LI> If a mixture of include and exclude columns are specified, then +all but the exclude columns will be active; this last case +is ambiguous and the rule is arbitrary. +</UL> +In addition to specifying columns names explicitly, the special +symbols <B>+</B> and <B>-</B> can be used to activate and +de-activate <B>all</B> columns. This is useful if you want to +activate the $REGION column along with all other columns. According +to the rules, the syntax "$REGION" only activates the region column +and de-activates the rest. Use "+ $REGION" to activate all +columns as well as the region column. + +<P> +If the data being displayed are image data (either in a FITS primary +image, a FITS image extension, or an array file), an mxn pixel display +is produced, where m and n are the dimensions of the image. By +default, pixel values are displayed using the same data type as in the +file. However, for integer data where the BSCALE and BZERO header parameters +are present, the data is displayed as floats. In either case, the +display data type can be overridden using an optional second argument +of the form: +<PRE> + bitpix=n +</PRE> +where n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, +respectively. + +<P> +Of course, running <B>fundisp</B> on anything but the smallest image +usually results in a display whose size makes it unreadable. +Therefore, one can uses bracket notation (see below) +to apply section and/or blocking to the image before generating a +display. For example: +<PRE> + [sh] fundisp "test.fits[2:6,2:7]" bitpix=-32 + 2 3 4 5 6 + ---------- ---------- ---------- ---------- ---------- + 2: 3.00 4.00 5.00 6.00 7.00 + 3: 4.00 5.00 6.00 7.00 8.00 + 4: 5.00 6.00 7.00 8.00 9.00 + 5: 6.00 7.00 8.00 9.00 10.00 + 6: 7.00 8.00 9.00 10.00 11.00 + 7: 8.00 9.00 10.00 11.00 12.00 +</PRE> + +<P> +Note that is is possible to display a FITS binary table as an image +simply by passing the table through <B>funimage</B> first: +<PRE> + [sh] ./funimage test.ev stdout | fundisp "stdin[2:6,2:7]" bitpix=8 + 2 3 4 5 6 + ------- ------- ------- ------- ------- + 2: 3 4 5 6 7 + 3: 4 5 6 7 8 + 4: 5 6 7 8 9 + 5: 6 7 8 9 10 + 6: 7 8 9 10 11 + 7: 8 9 10 11 12 +</PRE> + +If the <B>-l</B> (list) switch is used, then an image is displayed as a +list containing the columns: X, Y, VAL. For example: +<PRE> + fundisp -l "test1.fits[2:6,2:7]" bitpix=-32 + X Y VAL + ---------- ---------- ----------- + 2 2 6.00 + 3 2 1.00 + 4 2 1.00 + 5 2 1.00 + 6 2 1.00 + 2 3 1.00 + 3 3 5.00 + 4 3 1.00 + 5 3 1.00 + 6 3 1.00 + 2 4 1.00 + 3 4 1.00 + 4 4 4.00 + 5 4 1.00 + 6 4 1.00 + 2 5 1.00 + 3 5 1.00 + 4 5 1.00 + 5 5 3.00 + 6 5 1.00 + 2 6 1.00 + 3 6 1.00 + 4 6 1.00 + 5 6 1.00 + 6 6 2.00 + 2 7 1.00 + 3 7 1.00 + 4 7 1.00 + 5 7 1.00 + 6 7 1.00 +</PRE> + +<p> +If the <B>-n</B> (nohead) switch is used, then no header is output for +tables. This is useful, for example, when fundisp output is being +directed into gnuplot. + +<P> +The <B>fundisp</B> program uses a default set of display formats: +<PRE> + datatype TFORM format + -------- ----- -------- + double D "%21.8f" + float E "%11.2f" + int J "%10d" + short I "%8d" + byte B "%6d" + string A "%12.12s" + bits X "%8x" + logical L "%1x" +</PRE> +Thus, the default display of 1 double and 2 shorts gives: +<PRE> + [sh] fundisp snr.ev "time x y" + + TIME X Y + --------------------- -------- -------- + 79494546.56818075 546 201 + 79488769.94469175 548 201 + ... +</PRE> +You can change the display format for individual columns or for all +columns of a given data types by means of the -f switch. The format +string that accompanies -f is a space-delimited list of keyword=format +values. The keyword values can either be column names (in which case +the associated format pertains only to that column) or FITS table +TFORM specifiers (in which case the format pertains to all columns +having that data type). For example, you can change the double and +short formats for all columns like this: +<PRE> + [sh] fundisp -f "D=%22.11f I=%3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- --- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ... +</PRE> + +<p> +Alternatively, you can change the format of the time and x columns like this: +<pre> + [sh] fundisp -f "time=%22.11f x=%3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- -------- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ... +</pre> +Note that there is a potential conflict if a column has the same name +as one of the TFORM specifiers. In the examples above, the the "X" +column in the table has the same name as the X (bit) datatype. To +resolve this conflict, the format string is processed such that +TFORM datatype specifiers are checked for first, using a +case-sensitive comparison. If the specified format value is not an +upper case TFORM value, then a case-insensitive check is made on the +column name. This means that, in the examples above, "X=%3d" will refer +to the X (bit) datatype, while "x=%3d" will refer to the X column: +<pre> + [sh] fundisp -f "X=%3d" snr.ev "x y" + + X Y + -------- -------- + 546 201 + 548 201 + ... + + [sh] fundisp -f "x=%3d" snr.ev "x y" + + X Y + --- -------- + 546 201 + 548 201 + ... +</pre> +As a rule, therefore, it is best always to specify the column name in +lower case and TFORM data types in upper case. + +<p> +The <B>-f [format]</B> will change the format for a single execution +of fundisp. You also can use the <B>FUN_FORMAT</B> envronment variable +to change the format for all invocations of fundisp. The format of this +environment variable's value is identical to that used with +the <B>-f</B> switch. This global value can be overridden in +individual cases by use of the <B>-f [format]</B> switch. + +<P> +Caveats: Please also note that it is the user's responsibility to +match the format specifier to the column data type correctly. Also +note that, in order to maintain visual alignment between names and +columns, the column name will be truncated (on the left) if the +format width is less than the length of the name. However, truncation +is not performed if the output is in RDB format (using the -T switch). + +<p> +[An older-style format string is supported but deprecated. It +consists of space-delimited C format statements for all data types, +specified in the following order: +<PRE> + double float int short byte string bit. +</PRE> +This order of the list is based on the assumption that people generally +will want to change the float formats. +<P> +If "-" is entered instead of a format statement for a given data type, the +default format is used. Also, the format string can be terminated without +specifying all formats, and defaults will be used for the rest of the +list. Note that you must supply a minimum field width, i.e., "%6d" and +"%-6d" are legal, "%d" is not legal. + +By using -f [format], you can change the double and short formats like this: +<PRE> + [sh] fundisp -f "22.11f - - 3d" snr.ev "time x y" + + TIME X Y + ---------------------- --- --- + 79494546.56818075478 546 201 + 79488769.94469174743 548 201 + ... +</PRE> +NB: This format is deprecated and will be removed in a future release.] + +<P> +The <B>-F[c]</B> switch can be used to specify a (single-character) +column separator (where the default is a space). Note that column +formatting will almost certainly also add spaces to pad individual +columns to the required width. These can be removed with a program +such as sed, at the cost of generating unaligned columns. For example: +<PRE> +fundisp -F',' snr.ev'[cir 512 512 .1]' + X, Y, PHA, PI, TIME, DX, DY +--------,--------,--------,--------,---------------------,--------,-------- + 512, 512, 6, 7, 79493997.45854475, 578, 574 + 512, 512, 8, 9, 79494575.58943175, 579, 573 + 512, 512, 5, 6, 79493631.03866175, 578, 575 + 512, 512, 5, 5, 79493290.86521725, 578, 575 + 512, 512, 8, 9, 79493432.00990875, 579, 573 + +fundisp -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g' + X,Y,PHA,PI,TIME,DX,DY +--------,--------,--------,--------,---------------------,--------,-------- + 512,512,6,7,79493997.45854475,578,574 + 512,512,8,9,79494575.58943175,579,573 + 512,512,5,6,79493631.03866175,578,575 + 512,512,5,5,79493290.86521725,578,575 + 512,512,8,9,79493432.00990875,579,573 + +fundisp -f "x=%3d y=%3d pi=%1d pha=%1d time=%20.11f dx=%3d dy=%3d" -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g' + X,Y,A,I,TIME,DX,DY +---,---,-,-,--------------------,---,--- +512,512,6,7,79493997.45854474604,578,574 +512,512,8,9,79494575.58943174779,579,573 +512,512,5,6,79493631.03866174817,578,575 +512,512,5,5,79493290.86521725357,578,575 +512,512,8,9,79493432.00990875065,579,573 + +</PRE> + +<P> +If the <B>-T</B> (rdb table) switch is used, the output will conform +to starbase/rdb data base format: tabs will be inserted between +columns rather than spaces. This format is not available when +displaying image pixels (except in conjunction with the <B>-l</B> +switch). + +<P> +Finally, note that <B>fundisp</B> can be used to create column filters from +the auxiliary tables in a FITS file. For example, the following shell code +will generate a good-time interval (GTI) filter for X-ray data files that +contain a standard GTI extension: +<PRE> + #!/bin/sh + sed '1,/---- .*/d + /^$/,$d' | awk 'tot>0{printf "||"};{printf "time="$1":"$2; tot++}' +</PRE> +If this script is placed in a file called "mkgti", it can be used in a +command such as: +<PRE> + fundisp foo.fits"[GTI]" | mkgti > gti.filter +</PRE> +The resulting filter file can then be used in various funtools programs: +<PRE> + funcnts foo.fits"[@gti.filter]" ... +</PRE> +to process only the events in the good-time intervals. + +<!-- =defdoc funhead funhead 1 --> + +<!-- =section funhead NAME --> +<H2><A NAME="funhead">funhead - display a header in a Funtools file</A></H2> + +<!-- =section funhead SYNOPSIS --> +<B> +<PRE> +funhead [-a] [-s] [-t] [-L] <iname> [oname ename] +</PRE> +</B> + +<!-- =section funhead OPTIONS --> +<P> +<PRE> + -a # display all extension headers + -s # display 79 chars instead of 80 before the new-line + -t # prepend data type char to each line of output + -L # output in rdb/starbase list format +</PRE> + +<!-- =section funhead DESCRIPTION --> +<P> +<B>funhead</B> displays the FITS header parameters in the specified +<A HREF="./files.html">FITS Extension</A>. +<P> +The first argument to the program specifies the Funtools input file +to display. If "stdin" is specified, data are read from +the standard input. <A HREF="./files.html">Funtools Bracket +Notation</A> is used to specify particular FITS extension to process. +Normally, the full 80 characters of each header card is output, +followed by a new-line. + +<P> +If the <B>-a</B> switch is specified, the header from each FITS +extensions in the file is displayed. Note, however, that the <B>-a</B> +switch does not work with FITS files input via stdin. We hope to +remove this restriction in a future release. + +<P> +If the <B>-s</B> switch is specified, only 79 characters are output +before the new-line. This helps the display on 80 character terminals. + +<P> +If the <B>-t</B> switch is specified, the data type of the parameter +is output as a one character prefix, followed by 77 characters of the +param. The parameter data types are defined as: FUN_PAR_UNKNOWN +('u'), FUN_PAR_COMMENT ('c'), FUN_PAR_LOGICAL ('l'), FUN_PAR_INTEGER +('i'), FUN_PAR_STRING ('s'), FUN_PAR_REAL ('r'), FUN_PAR_COMPLEX ('x'). + +<P> +If the <B>-L</B> (rdb table) switch is used, the output will conform +to starbase/rdb data base list format. + +<P> +For example to display the EVENTS extension (binary table): +<PRE> + [sh] funhead "foo.fits[EVENTS]" + XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE + BITPIX = 8 / Binary data + NAXIS = 2 / Table is a matrix + NAXIS1 = 20 / Width of table in bytes + NAXIS2 = 30760 / Number of entries in table + PCOUNT = 0 / Random parameter count + GCOUNT = 1 / Group count + TFIELDS = 7 / Number of fields in each row + EXTNAME = 'EVENTS ' / Table name + EXTVER = 1 / Version number of table + TFORM1 = '1I ' / Data type for field + TTYPE1 = 'X ' / Label for field + TUNIT1 = ' ' / Physical units for field + TFORM2 = '1I ' / Data type for field + etc. ... + END +</PRE> + +<P> +To display the third header: +<PRE> + [sh] funhead "foo.fits[3]" + XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE + BITPIX = 8 / Binary data + NAXIS = 2 / Table is a matrix + NAXIS1 = 32 / Width of table in bytes + NAXIS2 = 40 / Number of entries in table + PCOUNT = 0 / Random parameter count + GCOUNT = 1 / Group count + TFIELDS = 7 / Number of fields in each row + EXTNAME = 'TGR ' / Table name + EXTVER = 1 / Version number of table + TFORM1 = '1D ' / Data type for field + etc. ... + END +</PRE> + +<P> +To display the primary header (i.e., extension 0): +<PRE> + sh> funhead "coma.fits[0]" + SIMPLE = T /STANDARD FITS FORMAT + BITPIX = 16 /2-BYTE TWOS-COMPL INTEGER + NAXIS = 2 /NUMBER OF AXES + NAXIS1 = 800 / + NAXIS2 = 800 / + DATATYPE= 'INTEGER*2' /SHORT INTEGER + END +</PRE> + +<P> +The funhead program also can edit (i.e. add, delete, or modify) or +display individual headers parameters. Edit mode is signalled by the +presence of two additional command-line arguments: output file and +edit command file, in that order. Edit mode acts as a filter: the +output file will contain the entire input FITS file, including other +extensions. The edit command file can be "stdin", in which case edit +command are read from the standard input. + +<P> +The edit command file contains parameter comments (having '#' in the +first column) and delete and assignment(modify or add) operations. A +delete operation is specified by preceding the parameter name with a +minus sign "-". A display operation (very useful in interactive +sessions, i.e., where the edit commands are taken from stdin) is +specified by preceding the parameter name with a question mark "?". In +either case, a parameter value need not be specified. An assignment +operation is specified in the same two ways that a parameter is +specified in a text header (but without the comment character that +precedes header params), i.e.: + +<UL> +<LI> FITS-style comments have an equal sign "=" between the keyword and +value and an optional slash "/" to signify a comment. The strict FITS +rules on column positions are not enforced. + +<LI> Free-form comments can have an optional colon separator between the +keyword and value. In the absence of quote, all tokens after the +keyword are part of the value, i.e. no comment is allowed. +</UL> + +<P> +For example, the following interactive session checks for the +existence of parameters, adds new parameters, modifies them, and +modifies and deletes existing parameters: +<PRE> + sh$ ./funhead snr.ev foo.fits - + # look for FOO1 + ? FOO1 + WARNING: FOO1 not found + # add new foo1 + FOO1 = 100 + # add foo2 + FOO2 = 200 + # reset foo1 to a different value + FOO1 -1 + # delete foo2 + -FOO2 + # change existing value + EXTVER 2 + ? XS-SORT + XS-SORT = 'EOF ' / type of event sort + # delete existing value + -XS-SORT + # exit + ^D +</PRE> + +<P> +See <A HREF="./text.html">Column-based Text Files</A> +for more information about header parameter format. + +<P> + +<P> +<!-- =defdoc funhist funhist 1 --> + +<!-- =section funhist NAME --> +<H2><A NAME="funhist">funhist - create a 1D histogram of a column (from a FITS binary table or raw event file) or an image</A></H2> + +<!-- =section funhist SYNOPSIS --> +<B> +<PRE> +funhist [-n|-w|-T] <iname> [column] [[lo:hi:]bins] +</PRE> +</B> + +<!-- =section funhist OPTIONS --> +<P> +<PRE> + -n # normalize bin value by the width of each bin + -w # specify bin width instead of number of bins in arg3 + -T # output in rdb/starbase format (tab separators) +</PRE> + +<!-- =section funhist DESCRIPTION --> +<P> +<B>funhist</B> creates a one-dimensional histogram from the specified +columns of a <A HREF="./files.html">FITS Extension</A> +binary table of a FITS file (or from a non-FITS raw event file), or +from a FITS image or array, and writes that histogram as an ASCII +table. Alternatively, the program can perform a 1D projection of one +of the image axes. + +<P> +The first argument to the program is required, and specifies the +Funtools file: FITS table or image, raw event file, or array. If +"stdin" is specified, data are read from the standard input. Use +<A HREF="./files.html">Funtools Bracket Notation</A> to specify FITS +extensions, and filters. + +<P> +For a table, the second argument also is required. It specifies the +column to use in generating the histogram. If the data file is of +type image (or array), the column is optional: if "x" (or "X"), "y" +(or "Y") is specified, then a projection is performed over the x +(dim1) or y (dim2) axes, respectively. (That is, this projection will +give the same results as a histogram performed on a table containing +the equivalent x,y event rows.) If no column name is specified or +"xy" (or "XY") is specified for the image, then a histogram is +performed on the values contained in the image pixels. + +<P> +The argument that follows is optional and specifies the number of bins +to use in creating the histogram and, if desired, the range of bin +values. For image and table histograms, the range should specify the +min and max data values. For image histograms on the x and y axes, +the range should specify the min and max image bin values. If this +argument is omitted, the number of output bins for a table is +calculated either from the TLMIN/TLMAX headers values (if these exist +in the table FITS header for the specified column) or by going through +the data to calculate the min and max value. For an image, the number +of output bins is calculated either from the DATAMIN/DATAMAX header +values, or by going through the data to calculate min and max value. +(Note that this latter calculation might fail if the image cannot be +fit in memory.) If the data are floating point (table or image) and +the number of bins is not specified, an arbitrary default of 128 is +used. + +<P> +For binary table processing, the <B>-w</B> (bin width) switch can be used +to specify the width of each bin rather than the number of bins. Thus: +<PRE> + funhist test.ev pha 1:100:5 +</PRE> +means that 5 bins of width 20 are used in the histogram, while: +<PRE> + funhist -w test.ev pha 1:100:5 +</PRE> +means that 20 bins of width 5 are used in the histogram. + +<P> +The data are divvied up into the specified number of bins and the +resulting 1D histogram (or projection) is output in ASCII table +format. For a table, the output displays the low_edge (inclusive) and +hi_edge (exclusive) values for the data. For example, a 15-row table +containing a "pha" column whose values range from -7.5 to 7.5 +can be processed thus: + +<PRE> + [sh] funhist test.ev pha + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: -7.5 7.5 15 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 22 -7.50000000 -6.50000000 + 2 21 -6.50000000 -5.50000000 + 3 20 -5.50000000 -4.50000000 + 4 19 -4.50000000 -3.50000000 + 5 18 -3.50000000 -2.50000000 + 6 17 -2.50000000 -1.50000000 + 7 16 -1.50000000 -0.50000000 + 8 30 -0.50000000 0.50000000 + 9 16 0.50000000 1.50000000 + 10 17 1.50000000 2.50000000 + 11 18 2.50000000 3.50000000 + 12 19 3.50000000 4.50000000 + 13 20 4.50000000 5.50000000 + 14 21 5.50000000 6.50000000 + 15 22 6.50000000 7.50000000 + + [sh] funhist test.ev pha 1:6 + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: 0.5 6.5 6 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 16 0.50000000 1.50000000 + 2 17 1.50000000 2.50000000 + 3 18 2.50000000 3.50000000 + 4 19 3.50000000 4.50000000 + 5 20 4.50000000 5.50000000 + 6 21 5.50000000 6.50000000 + + [sh] funhist test.ev pha 1:6:3 + # data file: /home/eric/data/test.ev + # column: pha + # min,max,bins: 0.5 6.5 3 + + bin value lo_edge hi_edge + ------ --------- --------------------- --------------------- + 1 33 0.50000000 2.50000000 + 2 37 2.50000000 4.50000000 + 3 41 4.50000000 6.50000000 +</PRE> + +<P> +For a table histogram, the <B>-n</B>(normalize) switch can be used to +normalize the bin value by the width of the bin (i.e., hi_edge-lo_edge): +<PRE> + [sh] funhist -n test.ev pha 1:6:3 + # data file: test.ev + # column: pha + # min,max,bins: 0.5 6.5 3 + # width normalization (val/(hi_edge-lo_edge)) is applied + + bin value lo_edge hi_edge + ------ --------------------- --------------------- --------------------- + 1 16.50000000 0.50000000 2.50000000 + 2 6.16666667 2.50000000 4.50000000 + 3 4.10000000 4.50000000 6.50000000 +</PRE> +This could used, for example, to produce a light curve with values +having units of counts/second instead of counts. + +<P> +For an image histogram, the output displays the low and high image +values (both inclusive) used to generate the histogram. For example, +in the following example, 184 pixels had a value of 1, 31 had a value +of 2, while only 2 had a value of 3,4,5,6, or 7: +<PRE> + [sh] funhist test.fits + # data file: /home/eric/data/test.fits + # min,max,bins: 1 7 7 + + bin value lo_val hi_val + ------ --------------------- --------------------- --------------------- + 1 184.00000000 1.00000000 1.00000000 + 2 31.00000000 2.00000000 2.00000000 + 3 2.00000000 3.00000000 3.00000000 + 4 2.00000000 4.00000000 4.00000000 + 5 2.00000000 5.00000000 5.00000000 + 6 2.00000000 6.00000000 6.00000000 + 7 2.00000000 7.00000000 7.00000000 +</PRE> + +<P> +For the axis projection of an image, the output displays the low and +high image bins (both inclusive) used to generate the projection. For +example, in the following example, 21 counts had their X bin value of +2, etc.: +<PRE> + [sh] funhist test.fits x 2:7 + # data file: /home/eric/data/test.fits + # column: X + # min,max,bins: 2 7 6 + + bin value lo_bin hi_bin + ------ --------------------- --------------------- --------------------- + 1 21.00000000 2.00000000 2.00000000 + 2 20.00000000 3.00000000 3.00000000 + 3 19.00000000 4.00000000 4.00000000 + 4 18.00000000 5.00000000 5.00000000 + 5 17.00000000 6.00000000 6.00000000 + 6 16.00000000 7.00000000 7.00000000 + + [sh] funhist test.fits x 2:7:2 + # data file: /home/eric/data/test.fits + # column: X + # min,max,bins: 2 7 2 + + bin value lo_bin hi_bin + ------ --------------------- --------------------- --------------------- + 1 60.00000000 2.00000000 4.00000000 + 2 51.00000000 5.00000000 7.00000000 +</PRE> + +<P> +You can use gnuplot or other plotting programs to graph the +results, using a script such as: +<PRE> + #!/bin/sh + sed -e '1,/---- .*/d + /^$/,$d' | \ + awk '\ + BEGIN{print "set nokey; set title \"funhist\"; set xlabel \"bin\"; set ylabel \"counts\"; plot \"-\" with boxes"} \ + {print $3, $2, $4-$3}' | \ + gnuplot -persist - 1>/dev/null 2>&1 +</PRE> + +Similar plot commands are supplied in the script <B>funhist.plot</B>: +<PRE> + funhist test.ev pha ... | funhist.plot gnuplot +</PRE> + +<!-- =defdoc funimage funimage 1 --> + +<!-- =section funimage NAME --> +<H2><A NAME="funimage">funimage - create a FITS image from a Funtools data file</A></H2> + +<!-- =section funimage SYNOPSIS --> +<B> +<PRE> +funimage [-a] <iname> <oname> [bitpix=n] +funimage [-l] <iname> <oname> <xcol:xdims> <ycol:ydims> <vcol> [bitpix=n] +funimage [-p x|y] <iname> <oname> [bitpix=n] +</PRE> +</B> + +<!-- =section funimage OPTIONS --> +<P> +<PRE> + -a # append to existing output file as an image extension + -l # input is a list file containing xcol, ycol, value + -p [x|y] # project along x or y axis to create a 1D image +</PRE> + +<!-- =section funimage DESCRIPTION --> +<P> +<B>funimage</B> creates a primary FITS image from the specified +<A HREF="./files.html">FITS Extension</A> +and/or +<A HREF="./files.html#sections">Image Section</A> +of a FITS file, or from an +<A HREF="./files.html#sections">Image Section</A> +of a non-FITS array, or from a raw event file. +<P> +The first argument to the program specifies the FITS input image, +array, or raw event file to process. If "stdin" is specified, data are +read from the standard input. Use <A HREF="./files.html">Funtools +Bracket Notation</A> to specify FITS extensions, image sections, and +filters. The second argument is the output FITS file. If "stdout" is +specified, the FITS image is written to the standard output. By +default, the output pixel values are of the same data type as those of the +input file (or type "int" when binning a table), but this can be +overridden using an optional third argument of the form: +<PRE> + bitpix=n +</PRE> +where n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, +respectively. + +<P> +If the input data are of type image, the appropriate section is +extracted and blocked (based on how the +<A HREF="./files.html#sections">Image Section</A> is specified), and +the result is written to the FITS primary image. When an integer +image containing the BSCALE and BZERO keywords is converted to float, +the pixel values are scaled and the scaling keywords are deleted from the +output header. When converting integer scaled data to integer +(possibly of a different size), the pixels are not scaled and the +scaling keywords are retained. + +<P> +If the input data is a binary table or raw event file, these are +binned into an image, from which a section is extracted and blocked, +and written to a primary FITS image. In this case, it is necessary to +specify the two columns that will be used in the 2D binning. This can +be done on the command line using the <B>bincols=(x,y)</B> keyword: +<PRE> + funcnts "foo.ev[EVENTS,bincols=(detx,dety)]" +</PRE> +The full form of the <B>bincols=</B> specifier is: +<PRE> + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]]) +</PRE> +where the tlmin, tlmax, and binsiz specifiers determine the image binning +dimensions: +<PRE> + dim = (tlmax - tlmin)/binsiz (floating point data) + dim = (tlmax - tlmin)/binsiz + 1 (integer data) +</PRE> +Using this syntax, it is possible to bin any two columns of a binary +table at any bin size. Note that the tlmin, tlmax, and binsiz +specifiers can be omitted if TLMIN, TLMAX, and TDBIN header parameters +(respectively) are present in the FITS binary table header for the +column in question. Note also that if only one parameter is specified, +it is assumed to be tlmax, and tlmin defaults to 1. If two parameters +are specified, they are assumed to be tlmin and tlmax. +See <A HREF="./files.html#binning">Binning FITS Binary Tables and Non-FITS +Event Files</A> for more information about binning parameters. + +<p> +By default, a new 2D FITS image file is created and the image is written +to the primary HDU. If the <B>-a</B> (append) switch is specified, +the image is appended to an existing FITS file as an IMAGE extension. +(If the output file does not exist, the switch is effectively ignored +and the image is written to the primary HDU.) This can be useful in a +shell programming environment when processing multiple FITS images +that you want to combine into a single final FITS file. + +<P> +<B>funimage</B> also can take input from a table containing columns of +x, y, and value (e.g., the output from <B>fundisp -l</B> which +displays each image x and y and the number of counts at that +position.) When the <B>-l</B> (list) switch is used, the input file is +taken to be a FITS or ASCII table containing (at least) three columns +that specify the x and y image coordinates and the value of that +image pixel. In this case, <B>funimage</B> requires four extra +arguments: xcolumn:xdims, ycolumn:ydims, vcolumn and bitpix=n. The x +and y col:dim information takes the form: +<PRE> + name:dim # values range from 1 to dim + name:min:max # values range from min to max + name:min:max:binsiz # dimensions scaled by binsize +</PRE> +In particular, the min value should be used whenever the +minimum coordinate value is something other than one. For example: +<PRE> + funimage -l foo.lst foo.fits xcol:0:512 ycol:0:512 value bitpix=-32 +</PRE> + +<P> +The list feature also can be used to read unnamed columns from standard +input: simply replace the column name with a null string. Note +that the dimension information is still required: +<PRE> + funimage -l stdin foo.fits "":0:512 "":0:512 "" bitpix=-32 + 240 250 1 + 255 256 2 + ... + ^D +</PRE> + +<P> +The list feature provides a simple way to generate a blank image. +If you pass a <A HREF="./text.html">Column-based Text File</A> +to funimage in which the text header contains the required image +information, then funimage will correctly make a blank image. For +example, consider the following text file (called foo.txt): +<PRE> + x:I:1:10 y:I:1:10 + ------ ------ + 0 0 +</PRE> +This text file defines two columns, x and y, each of data type 32-bit int and +image dimension 10. The command: +<PRE> + funimage foo.txt foo.fits bitpix=8 +</PRE> +will create an empty FITS image called foo.fits containing a 10x10 +image of unsigned char: +<PRE> + fundisp foo.fits + 1 2 3 4 5 6 7 8 9 10 + ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ + 10: 0 0 0 0 0 0 0 0 0 0 + 9: 0 0 0 0 0 0 0 0 0 0 + 8: 0 0 0 0 0 0 0 0 0 0 + 7: 0 0 0 0 0 0 0 0 0 0 + 6: 0 0 0 0 0 0 0 0 0 0 + 5: 0 0 0 0 0 0 0 0 0 0 + 4: 0 0 0 0 0 0 0 0 0 0 + 3: 0 0 0 0 0 0 0 0 0 0 + 2: 0 0 0 0 0 0 0 0 0 0 + 1: 1 0 0 0 0 0 0 0 0 0 + +</PRE> + +Note that the text file must contain at least +one row of data. However, in the present example, event position 0,0 is +outside the limits of the image and will be ignored. (You can, of course, +use real x,y values to seed the image with data.) + +<P> +Furthermore, you can use the TEXT filter specification to obviate the need for +an input text file altogether. The following command will create the same +10x10 char image without an actual input file: +<PRE> + funimage stdin'[TEXT(x:I:10,y:I:10)]' foo.fits bitpix=8 < /dev/null +or + funimage /dev/null'[TEXT(x:I:10,y:I:10)]' foo.fits bitpix=8 +</PRE> + +<P> +You also can use either of these methods to generate a region mask simply +by appending a region inside the filter brackets and specfying <B>mask=all</B> +along with the bitpix. For example, the following command will generate a +10x10 char mask using 3 regions: +<PRE> + funimage stdin'[TEXT(x:I:10,y:I:10),cir(5,5,4),point(10,1),-cir(5,5,2)]' \ + foo.fits bitpix=8,mask=all < /dev/null +</PRE> +The resulting mask looks like this: +<PRE> + fundisp foo.fits + 1 2 3 4 5 6 7 8 9 10 + ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ + 10: 0 0 0 0 0 0 0 0 0 0 + 9: 0 0 0 0 0 0 0 0 0 0 + 8: 0 0 1 1 1 1 1 0 0 0 + 7: 0 1 1 1 1 1 1 1 0 0 + 6: 0 1 1 0 0 0 1 1 0 0 + 5: 0 1 1 0 0 0 1 1 0 0 + 4: 0 1 1 0 0 0 1 1 0 0 + 3: 0 1 1 1 1 1 1 1 0 0 + 2: 0 0 1 1 1 1 1 0 0 0 + 1: 0 0 0 0 0 0 0 0 0 2 +</PRE> + +<P> +You can use <B>funimage</B> to create 1D image projections along the x +or y axis using the <b>-p [x|y]</b> switch. This capability works for +both images and tables. For example consider a FITS table named ev.fits +containing the following rows: +<PRE> + X Y + -------- -------- + 1 1 + 1 2 + 1 3 + 1 4 + 1 5 + 2 2 + 2 3 + 2 4 + 2 5 + 3 3 + 3 4 + 3 5 + 4 4 + 4 5 + 5 5 + +</PRE> +A corresponding 5x5 image, called dim2.fits, would therefore contain: +<PRE> + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 5: 1 1 1 1 1 + 4: 1 1 1 1 0 + 3: 1 1 1 0 0 + 2: 1 1 0 0 0 + 1: 1 0 0 0 0 +</PRE> +A projection along the y axis can be performed on either the table or +the image: +<PRE> + funimage -p y ev.fits stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 + + funimage -p y dim2.fits stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 +</PRE> + +<P> +Furthermore, you can create a 1D image projection along any column of +a table by using the <b>bincols=[column]</b> filter specification and +specifying a single column. For example, the following command +projects the same 1D image along the y axis of a table as use of +the <b>-p y</b> switch: +<PRE> + funimage ev.fits'[bincols=y]' stdout | fundisp stdin + 1 2 3 4 5 + ---------- ---------- ---------- ---------- ---------- + 1: 1 2 3 4 5 +</PRE> + +<P> +Examples: +<P> +Create a FITS image from a FITS binary table: +<PRE> + [sh] funimage test.ev test.fits +</PRE> + +<P> +Display the FITS image generated from a blocked section of FITS binary table: +<PRE> + [sh] funimage "test.ev[2:8,3:7,2]" stdout | fundisp stdin + 1 2 3 + --------- --------- --------- + 1: 20 28 36 + 2: 28 36 44 +</PRE> + +<!-- =defdoc funindex funindex 1 --> + +<!-- =section funindex NAME --> +<H2><A NAME="funindex">funindex - create an index for a column of a FITS binary table</A></H2> + +<!-- =section funindex SYNOPSIS --> +<B> +<PRE> +funindex <switches> <iname> <key> [oname] +</PRE> +</B> + +<!-- =section funindex OPTIONS --> +<P> +<PRE> + NB: these options are not compatible with Funtools processing. Please + use the defaults instead. + -c # compress output using gzip" + -a # ASCII output, ignore -c (default: FITS table)" + -f # FITS table output (default: FITS table)" + -l # long output, i.e. with key value(s) (default: long)" + -s # short output, i.e. no key value(s) (default: long)" +</PRE> + +<!-- =section funindex DESCRIPTION --> +<P> +The funindex script creates an index for the specified column (key) by +running funtable -s (sort) and then saving the column value and the +record number for each sorted row. This index will be used automatically + by funtools filtering of that column, provided the index file's modification +date is later than that of the data file. + +<p> +The first required argument is the name of the FITS binary table +to index. Please note that text files cannot be indexed at this time. +The second required argument is the column (key) name to index. While +multiple keys can be specified in principle, the funtools index processing +assume a single key and will not recognize files containing multiple keys. + +<P> +By default, the output index file name is [root]_[key].idx, where [root] +is the root of the input file. Funtools looks for this specific file name +when deciding whether to use an index for faster filtering. Therefore, the +optional third argument (output file name) should not be used for funtools +processing. + +<P> +For example, to create an index on column Y for a given FITS file, use: +<PRE> + funindex foo.fits Y +</PRE> +This will generate an index named foo_y.idx, which will be used by funtools +for filters involving the Y column. + +<!-- =defdoc funjoin funjoin 1 --> + +<!-- =section funjoin NAME --> +<H2><A NAME="funjoin">funjoin - join two or more FITS binary tables on specified columns</A></H2> + +<!-- =section funjoin SYNOPSIS --> +<B> +<PRE> +funjoin [switches] <ifile1> <ifile2> ... <ifilen> <ofile> +</PRE> +</B> + +<!-- =section funjoin OPTIONS --> +<P> +<PRE> + -a cols # columns to activate in all files + -a1 cols ... an cols # columns to activate in each file + -b 'c1:bvl,c2:bv2' # blank values for common columns in all files + -bn 'c1:bv1,c2:bv2' # blank values for columns in specific files + -j col # column to join in all files + -j1 col ... jn col # column to join in each file + -m min # min matches to output a row + -M max # max matches to output a row + -s # add 'jfiles' status column + -S col # add col as status column + -t tol # tolerance for joining numeric cols [2 files only] +</PRE> + +<!-- =section funjoin DESCRIPTION --> +<B>funjoin</B> joins rows from two or more (up to 32) +<A HREF="./files.html">FITS Binary Table files</A>, based on the values +of specified join columns in each file. NB: the join columns must have +an index file associated with it. These files are generated using the +<B>funindex</B> program. + +<P> +The first argument to the program specifies the first input FITS table +or raw event file. If "stdin" is specified, data are read from the +standard input. Subsequent arguments specify additional event files +and tables to join. The last argument is the output FITS file. + +<P> +NB: Do <B>not</B> use <A HREF="./files.html">Funtools Bracket +Notation</A> to specify FITS extensions and row filters when running +funjoin or you will get wrong results. Rows are accessed and joined +using the index files directly, and this bypasses all filtering. + +<P> +The join columns are specified using the <B>-j col</B> switch (which +specifies a column name to use for all files) or with <B>-j1 col1</B>, +<B>-j2 col2</B>, ... <B>-jn coln</B> switches (which specify a column +name to use for each file). A join column must be specified for each file. +If both <B>-j col</B> and <B>-jn coln</B> are specified for a given +file, then the latter is used. Join columns must either be of type +string or type numeric; it is illegal to mix numeric and string +columns in a given join. For example, to join three files using the +same key column for each file, use: +<PRE> + funjoin -j key in1.fits in2.fits in3.fits out.fits +</PRE> +A different key can be specified for the third file in this way: +<PRE> + funjoin -j key -j3 otherkey in1.fits in2.fits in3.fits out.fits +</PRE> + +<P> +The <B>-a "cols"</B> switch (and <B>-a1 "col1"</B>, +<B>-a2 "cols2"</B> counterparts) can be used to specify columns to +activate (i.e. write to the output file) for each input file. By +default, all columns are output. + +<P> +If two or more columns from separate files have the same name, the +second (and subsequent) columns are renamed to have an underscore +and a numeric value appended. + +<P> +The <B>-m min</B> and <B>-M max</B> switches specify the minimum +and maximum number of joins required to write out a row. The default +minimum is 0 joins (i.e. all rows are written out) and the default maximum +is 63 (the maximum number of possible joins with a limit of 32 input files). +For example, to write out only those rows in which exactly two files +have columns that match (i.e. one join): +<PRE> + funjoin -j key -m 1 -M 1 in1.fits in2.fits in3.fits ... out.fits +</PRE> + +<P> +A given row can have the requisite number of joins without all of the +files being joined (e.g. three files are being joined but only two +have a given join key value). In this case, all of the columns of the +non-joined file are written out, by default, using blanks (zeros or NULLs). +The <B>-b c1:bv1,c2:bv2</B> and +<B>-b1 'c1:bv1,c2:bv2' -b2 'c1:bv1,c2:bv2' ...</B> +switches can be used to set the blank value for columns common to all +files and/or columns in a specified file, respectively. Each blank value +string contains a comma-separated list of column:blank_val specifiers. +For floating point values (single or double), a case-insensitive string +value of "nan" means that the IEEE NaN (not-a-number) should be +used. Thus, for example: +<PRE> + funjoin -b "AKEY:???" -b1 "A:-1" -b3 "G:NaN,E:-1,F:-100" ... +</PRE> +means that a non-joined AKEY column in any file will contain the +string "???", the non-joined A column of file 1 will contain a value +of -1, the non-joined G column of file 3 will contain IEEE NaNs, while +the non-joined E and F columns of the same file will contain values -1 +and -100, respectively. Of course, where common and specific blank values +are specified for the same column, the specific blank value is used. + +<P> +To distinguish which files are non-blank components of a given row, +the <B>-s</B> (status) switch can be used to add a bitmask column named +"JFILES" to the output file. In this column, a bit is set for each +non-blank file composing the given row, with bit 0 corresponds to the +first file, bit 1 to the second file, and so on. The file names +themselves are stored in the FITS header as parameters named JFILE1, +JFILE2, etc. The <B>-S col</B> switch allows you to change the name +of the status column from the default "JFILES". + +<P> +A join between rows is the Cartesian product of all rows in one file +having a given join column value with all rows in a second file having +the same value for its join column and so on. Thus, if file1 has 2 +rows with join column value 100, file2 has 3 rows with the same value, +and file3 has 4 rows, then the join results in 2*3*4=24 rows being output. + +<P> +The join algorithm directly processes the index file associated with +the join column of each file. The smallest value of all the current +columns is selected as a base, and this value is used to join +equal-valued columns in the other files. In this way, the index files +are traversed exactly once. + +<P> +The <B>-t tol</B> switch specifies a tolerance value for numeric +columns. At present, a tolerance value can join only two files at a +time. (A completely different algorithm is required to join more than +two files using a tolerance, somethng we might consider implementing +in the future.) + +<P> +The following example shows many of the features of funjoin. The input files +t1.fits, t2.fits, and t3.fits contain the following columns: +<PRE> + [sh] fundisp t1.fits + AKEY KEY A B + ----------- ------ ------ ------ + aaa 0 0 1 + bbb 1 3 4 + ccc 2 6 7 + ddd 3 9 10 + eee 4 12 13 + fff 5 15 16 + ggg 6 18 19 + hhh 7 21 22 + +fundisp t2.fits + AKEY KEY C D + ----------- ------ ------ ------ + iii 8 24 25 + ggg 6 18 19 + eee 4 12 13 + ccc 2 6 7 + aaa 0 0 1 + +fundisp t3.fits + AKEY KEY E F G +------------ ------ -------- -------- ----------- + ggg 6 18 19 100.10 + jjj 9 27 28 200.20 + aaa 0 0 1 300.30 + ddd 3 9 10 400.40 +</PRE> + +<P> +Given these input files, the following funjoin command: +<PRE> + + funjoin -s -a1 "-B" -a2 "-D" -a3 "-E" -b \ + "AKEY:???" -b1 "AKEY:XXX,A:255" -b3 "G:NaN,E:-1,F:-100" \ + -j key t1.fits t2.fits t3.fits foo.fits +</PRE> +will join the files on the KEY column, outputting all columns except B +(in t1.fits), D (in t2.fits) and E (in t3.fits), and setting blank +values for AKEY (globally, but overridden for t1.fits) and A (in file +1) and G, E, and F (in file 3). A JFILES column will be output to +flag which files were used in each row: +<PRE> + + AKEY KEY A AKEY_2 KEY_2 C AKEY_3 KEY_3 F G JFILES + ------------ ------ ------ ------------ ------ ------ ------------ ------ -------- ----------- -------- + aaa 0 0 aaa 0 0 aaa 0 1 300.30 7 + bbb 1 3 ??? 0 0 ??? 0 -100 nan 1 + ccc 2 6 ccc 2 6 ??? 0 -100 nan 3 + ddd 3 9 ??? 0 0 ddd 3 10 400.40 5 + eee 4 12 eee 4 12 ??? 0 -100 nan 3 + fff 5 15 ??? 0 0 ??? 0 -100 nan 1 + ggg 6 18 ggg 6 18 ggg 6 19 100.10 7 + hhh 7 21 ??? 0 0 ??? 0 -100 nan 1 + XXX 0 255 iii 8 24 ??? 0 -100 nan 2 + XXX 0 255 ??? 0 0 jjj 9 28 200.20 4 + +</PRE> +<!-- =defdoc funmerge funmerge 1 --> + +<!-- =section funmerge NAME --> +<H2><A NAME="funmerge">funmerge - merge one or more Funtools table files</A></H2> + +<!-- =section funmerge SYNOPSIS --> +<B> +<PRE> +funmerge [-w|-x] -f [colname] <iname1> <iname2> ... <oname> +</PRE> +</B> + +<!-- =section funmerge OPTIONS --> +<P> +<PRE> + -f # output a column specifying file from which this event came + -w # adjust position values using WCS info + -x # adjust position values using WCS info and save old values +</PRE> + +<!-- =section funmerge DESCRIPTION --> +<P> +<B>funmerge</B> merges FITS data from one or more +<A HREF="./files.html">FITS Binary Table files</A> +or raw event files. +<P> +The first argument to the program specifies the first input FITS table +or raw event file. If "stdin" is specified, data are read from the +standard input. Use <A HREF="./files.html">Funtools Bracket +Notation</A> to specify FITS extensions and row filters. Subsequent +arguments specify additional event files and tables to merge. (NB: Stdin +cannot not be used for any of these additional input file arguments.) +The last argument is the output FITS file. The columns in each input table +must be identical. + +<P> +If an input file begins with the '@' character, it is processed as an +include file, i.e., as a text file containing event file names (as +well as blank lines and/or comment lines starting with the '#' sign). +If standard input is specified as an include file ('@stdin'), then +file names are read from the standard input until EOF (^D). Event +files and include files can be mixed on a command line. + +<P> +Rows from each table are written sequentially to the output +file. If the switch <B>-f [colname]</B> is specified on the command +line, an additional column is added to each row containing the number +of the file from which that row was taken (starting from one). In +this case, the corresponding file names are stored in the header +parameters having the prefix <B>FUNFIL</B>, i.e., FUNFIL01, +FUNFIL02, etc. + +<P> +Using the <B>-w</B> switch (or <B>-x</B> switch as described +below), <B>funmerge</B> also can adjust the position column values +using the WCS information in each file. (By position columns, we mean +the columns that the table is binned on, i.e., those columns defined +by the <B>bincols=</B> switch, or (X,Y) by default.) To perform WCS +alignment, the WCS of the first file is taken as the base WCS. Each +position in subsequent files is adjusted by first converting it to the +sky coordinate in its own WCS coordinate system, then by converting +this sky position to the sky position of the base WCS, and finally +converting back to a pixel position in the base system. Note that in +order to perform WCS alignment, the appropriate WCS and TLMIN/TLMAX +keywords must already exist in each FITS file. +<P> +When performing WCS alignment, you can save the original positions in +the output file by using the <B>-x</B> (for "xtra") switch instead +of the <B>-w</B> switch (i.e., using this switch also implies using +<B>-w</B>) The old positions are saved in columns having the same +name as the original positional columns, with the added prefix "OLD_". +<P> +Examples: + +<P> +Merge two tables, and preserve the originating file number for +each row in the column called "FILE" (along with the corresponding +file name in the header): +<PRE> + [sh] funmerge -f "FILE" test.ev test2.ev merge.ev +</PRE> + +<P> +Merge two tables with WCS alignment, saving the old position values in +2 additional columns: +<PRE> + [sh] funmerge -x test.ev test2.ev merge.ev +</PRE> + +<P> +This program only works on raw event files and binary tables. We have +not yet implemented image and array merging. + +<!-- =defdoc funsky funsky 1 --> + +<!-- =section funsky NAME --> +<H2><A NAME="funsky">funsky - convert between image and sky coordinates</A></H2> + +<!-- =section funsky SYNOPSIS --> +<B> +<PRE> + funsky iname[ext] # RA,Dec (deg) or image pix from stdin + funsky iname[ext] [lname] # RA, Dec (deg) or image pix from list + funsky iname[ext] [col1] [col2] # named cols:units from stdin + funsky iname[ext] [lname] [col1] [col2] # named cols:units from list +</PRE> +</B> + +<!-- =section funsky OPTIONS --> +<P> +<PRE> + -d # always use integer tlmin conversion (as ds9 does) + -r # convert x,y to RA,Dec (default: convert RA,Dec to x,y) + -o # include offset from the nominal target position (in arcsec) + -v # display input values also (default: display output only) + -T # output display in rdb format (w/header,tab delimiters) +</PRE> + +<!-- =section funsky DESCRIPTION --> +<P> +Funsky converts input sky coordinates (RA, Dec) to image coordinates (or vice +versa) using the WCS information contained in the specified FITS file. Several +calling sequences are supported in order to make it easy to specify +coordinate positions in different ways. + +<P> +The first required argument is always the input FITS file (or +extension) containing the WCS information in an extension header. Note +that the data from this file is not used. By default, the program +converts input RA and Dec values to X and Y using this WCS +information. If the WCS is associated with a FITS image, then the X,Y +values are image values. If the WCS is associated with a binary table, +then the X, Y values are physical values. To convert X,Y to RA and +Dec, use the <B>-r</B> (reverse) switch. + +<P> +If no other command arguments are supplied, then the input positions +are read from the standard input. Each line is assumed to contain a +single coordinate position consisting of an RA in degrees (or X in +pixels) followed by a Dec in degrees (or Y in pixels). The usual +delimiters are supported (spaces, commas, tabs). For example: +<PRE> + # read from stdin, default column names and units + [sh] funsky snr.ev + 22.982695 58.606523 # input RA (hrs), Dec(deg) + 510.00 510.00 + 22.982127 58.607634 # input + 512.00 510.50 + 22.981700 58.614301 # input + 513.50 513.50 + ^D # end of input +</PRE> + +<p> +If a second argument is supplied, this argument is assumed to be +a file containing RA (X) and Dec (Y) positions. The file can either be +an ASCII table or a FITS binary table. The order of columns is +unimportant, if the table has a column header. In this case, the +names of the columns must be one of "RA", "DEC", or "X", "Y" for sky +to image and image to sky conversions, respectively. If the table has +no header, then once again, RA (X) is assumed to first, followed +by DEC (Y). +For example: +<PRE> + # read from file, default column names and units + [sh] cat hd.in + RA DEC + --------- --------- + 22.982695 58.606523 + 22.982127 58.607634 + 22.981700 58.614301 + + [sh] funsky snr.ev hd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 +</PRE> + +<P> +If three arguments are supplied, then the input positions again are +read from the standard input. Each line is assumed to contain a single +coordinate position consisting of an RA (or X in pixels) followed by a +Dec (or Y in pixels), with the usual delimiters supported. However, +the second and third arguments now specify the column names and/or +sky units using a colon-delimited syntax: +<PRE> + [colname]:[h|d|r] +</PRE> +If the colname is omitted, the names default to "RA", "DEC", "X", "Y", +"COL1", or "COL2" as above. If the units are omitted, the default is degrees +for both RA and Dec. When the -r switch is used (convert from image +to sky) the units are applied to the output instead of the input. The following +examples will serve to illustrate the options: +<PRE> + # read from stdin, specifying column names (def. units: degrees) + [sh] cat hd.in + MYRA MYDEC + --------- --------- + 22.982695 58.606523 + 22.982127 58.607634 + 22.981700 58.614301 + + [sh] funsky snr.ev MYRA MYDEC < hd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read from stdin, specifying column names and units + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev MYRA:d MYDEC:d < dd.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read stdin, convert image to sky, specifying output sky units + [sh] cat im.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] cat im.in | funsky -r snr.ev :d :d + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 +</PRE> + +<P> +Finally, four command arguments specify both and input file and column names +and/or units: +<PRE> + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev dd.in MYRA:d MYDEC:d + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # read file, convert image to sky, specifying output sky units + [sh] cat im.in + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] funsky -r snr.ev im.in :d :d + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 +</PRE> + +<P> +By default, the output of funsky consists only of the converted coordinate +position(s), one per output line. This makes parsing in shell scripts easy. +Use the <B>-v</B> (verbose) switch to specify that the input +coordinates should be pre-pended to each line. For example: +<PRE> + [sh] cat dd.in + MYRA MYDEC + --------- --------- + 344.740432 58.606523 + 344.731900 58.607634 + 344.725500 58.614301 + + [sh] funsky snr.ev dd.in MYRA:d MYDEC:d + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + [sh] funsky -v snr.ev dd.in MYRA:d MYDEC:d + 344.740432 58.606523 510.00 510.00 + 344.731900 58.607634 512.00 510.50 + 344.725500 58.614301 513.50 513.50 +</PRE> + +<P> +In addition, a full starbase table can be output using the <B>-T</B> +(table) switch. This switch can be used with or without the -v +switch. If the -T and -v are both specified, then a descriptive header +parameters are output before the table (mainly to remind you of the +sky units): +<PRE> + # output table in non-verbose mode + [sh] funsky -T snr.ev dd.in MYRA:d MYDEC:d + X Y + ------------ ------------ + 510.00 510.00 + 512.00 510.50 + 513.50 513.50 + + # output table in verbose mode + [sh] funsky -T -v snr.ev dd.in MYRA:d MYDEC:d + # IFILE = /Users/eric/data/snr.ev + # ICOL1 = MYRA + # ICOL2 = MYDEC + # IUNITS1 = d + # IUNITS2 = d + # OCOL1 = X + # OCOL2 = Y + + MYRA MYDEC X Y + ------------ ------------ ------------ ------------ + 344.740432 58.606523 510.00 510.00 + 344.731900 58.607634 512.00 510.50 + 344.725500 58.614301 513.50 513.50 +</PRE> + +<P> +Finally, the <B>-d</B> (ds9) switch mimicks ds9's use of integer TLMIN +and TLMAX values for all coordinate transformations. FITS conventions +seem to call for use of floating point TLMIN and TLMAX when the data are +floats. This convention is followed by funsky but results in a +small discrepancy with ds9's converted values for floating point +data. We will remedy this conflict in the future, maybe. + +<!-- =defdoc funtable funtable 1 --> + +<!-- =section funtable NAME --> +<H2><A NAME="funtable">funtable - copy selected rows from a Funtools file to a FITS binary table</A></H2> + +<!-- =section funtable SYNOPSIS --> +<B> +<PRE> +funtable [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns] +</PRE> +</B> + +<!-- =section funtable OPTIONS --> +<P> +<PRE> + -a # append to existing output file as a table extension + -i # for image data, only generate X and Y columns + -m # for tables, write a separate file for each region + -s "col1 ..." # columns on which to sort + -z # for image data, output zero-valued pixels +</PRE> + +<!-- =section funtable DESCRIPTION --> +<P> +<B>funtable</B> selects rows from the specified +<A HREF="./files.html">FITS Extension</A> +(binary table only) of a FITS file, or from a non-FITS raw event +file, and writes those rows to a FITS binary table file. It also +will create a FITS binary table from an image or a raw array file. + +<P> +The first argument to the program specifies the FITS file, raw event +file, or raw array file. If "stdin" is specified, data are read from +the standard input. Use <A HREF="./files.html">Funtools Bracket +Notation</A> to specify FITS extensions, and filters. The second +argument is the output FITS file. If "stdout" is specified, the FITS +binary table is written to the standard output. By default, all +columns of the input file are copied to the output file. Selected +columns can be output using an optional third argument in the form: +<PRE> + "column1 column1 ... columnN" +</PRE> + +<P> +The <B>funtable</B> program generally is used to select rows from a +FITS binary table using +<A HREF="./filters.html">Table Filters</A> +and/or +<A HREF="./regions.html">Spatial Region Filters</A>. +For example, you can copy only selected rows (and output only selected +columns) by executing in a command such as: +<PRE> + [sh] funtable "test.ev[pha==1&&pi==10]" stdout "x y pi pha" | fundisp stdin + X Y PHA PI + ------- ------- ------- --------- + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 + 1 10 1 10 +</PRE> +<P> +The special column <B>$REGION</B> can be specified to write the +region id of each row: +<PRE> + [sh $] funtable "test.ev[time-(int)time>=.99&&annulus(0 0 0 10 n=3)]" stdout 'x y time $REGION' | fundisp stdin + X Y TIME REGION + -------- -------- --------------------- ---------- + 5 -6 40.99000000 3 + 4 -5 59.99000000 2 + -1 0 154.99000000 1 + -2 1 168.99000000 1 + -3 2 183.99000000 2 + -4 3 199.99000000 2 + -5 4 216.99000000 2 + -6 5 234.99000000 3 + -7 6 253.99000000 3 +</PRE> +<P> +Here only rows with the proper fractional time and whose position also is +within one of the three annuli are written. +<P> +Columns can be excluded from display using a minus sign before the +column: +<PRE> + [sh $] funtable "test.ev[time-(int)time>=.99]" stdout "-time" | fundisp stdin + X Y PHA PI DX DY + -------- -------- -------- ---------- ----------- ----------- + 5 -6 5 -6 5.50 -6.50 + 4 -5 4 -5 4.50 -5.50 + -1 0 -1 0 -1.50 0.50 + -2 1 -2 1 -2.50 1.50 + -3 2 -3 2 -3.50 2.50 + -4 3 -4 3 -4.50 3.50 + -5 4 -5 4 -5.50 4.50 + -6 5 -6 5 -6.50 5.50 + -7 6 -7 6 -7.50 6.50 +</PRE> +All columns except the time column are written. +<P> +In general, the rules for activating and de-activating columns are: +<UL> +<LI> If only exclude columns are specified, then all columns but +the exclude columns will be activated. +<LI> If only include columns are specified, then only the specified columns +are activated. +<LI> If a mixture of include and exclude columns are specified, then +all but the exclude columns will be active; this last case +is ambiguous and the rule is arbitrary. +</UL> +In addition to specifying columns names explicitly, the special +symbols <EM>+</EM> and <EM>-</EM> can be used to activate and +de-activate <EM>all</EM> columns. This is useful if you want to +activate the $REGION column along with all other columns. According +to the rules, the syntax "$REGION" only activates the region column +and de-activates the rest. Use "+ $REGION" to activate all +columns as well as the region column. + +<P> +Ordinarily, only the selected table is copied to the output file. In +a FITS binary table, it sometimes is desirable to copy all of the +other FITS extensions to the output file as well. This can be done by +appending a '+' sign to the name of the extension in the input file +name. For example, the first command below copies only the EVENT table, +while the second command copies other extensions as well: +<PRE> + [sh] funtable "/proj/rd/data/snr.ev[EVENTS]" events.ev + [sh] funtable "/proj/rd/data/snr.ev[EVENTS+]" eventsandmore.ev +</PRE> + +<P> +If the input file is an image or a raw array file, then +<B>funtable</B> will generate a FITS binary table from the pixel +values in the image. Note that it is not possible to specify the +columns to output (using command-line argument 3). Instead, there are +two ways to create such a binary table from an image. By default, a +3-column table is generated, where the columns are "X", "Y", and +"VALUE". For each pixel in the image, a single row (event) is +generated with the "X" and "Y" columns assigned the dim1 and dim2 +values of the image pixel, respectively and the "VALUE" column +assigned the value of the pixel. With sort of table, running +<B>funhist</B> on the "VALUE" column will give the same results as +running <B>funhist</B> on the original image. + +<P> +If the <B>-i</B> ("individual" rows) switch is specified, then only +the "X" and "Y" columns are generated. In this case, each positive +pixel value in the image generates n rows (events), where n is equal +to the integerized value of that pixel (plus 0.5, for floating point +data). In effect, <B>-i</B> approximately recreates the rows of a +table that would have been binned into the input image. (Of course, +this is only approximately correct, since the resulting x,y positions +are integerized.) + +<P> +If the <B>-s [col1 col2 ... coln]</B> ("sort") switch is specified, +the output rows of a binary table will be sorted using the +specified columns as sort keys. The sort keys must be scalar columns +and also must be part of the output file (i.e. you cannot sort on a +column but not include it in the output). This facility uses the +<B>_sort</B> program (included with funtools), which must be accessible +via your path. + +<p> +For binary tables, the <B>-m</B> ("multiple files") switch will +generate a separate file for each region in the filter specification +i.e. each file contains only the rows from that region. Rows +which pass the filter but are not in any region also are put in a +separate file. + +<P> +The separate output file names generated by the <B>-m</B> switch are +produced automatically from the root output file to contain the region id of +the associated region. (Note that region ids start at 1, so that the +file name associated with id 0 contains rows that pass the filter but +are not in any given region.) Output file names are generated as follows: + +<UL> +<LI> A $n specification can be used anywhere in the root file name (suitably +quoted to protect it from the shell) and will be expanded to be the id +number of the associated region. For example: +<PRE> + funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' 'foo.goo_$n.fits' +</PRE> +will generate files named foo.goo_0.fits (for rows not in any region but +still passing the filter), foo.goo_1.fits (rows in region id #1, the first +region), foo.goo_2.fits (rows in region id #2), etc. Note that single quotes +in the output root are required to protect the '$' from the shell. + +<LI> If $n is not specified, then the region id will be placed before +the first dot (.) in the filename. Thus: +<PRE> + funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' foo.evt.fits +</PRE> +will generate files named foo0.evt.fits (for rows not in any region but +still passing the filter), foo1.evt.fits (rows in region id #1), +foo2.evt.fits (rows in region id #2), etc. + +<LI> If no dot is specified in the root output file name, then +the region id will be appended to the filename. Thus: +<PRE> + funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' 'foo_evt' +</PRE> +will generate files named foo_evt0 (for rows not in any region but +still passing the filter), foo_evt1 (rows in region id #1), +foo_evt2 (rows in region id #2), etc. +</UL> +The multiple file mechanism provide a simple way to generate +individual source data files with a single pass through the data. + +<p> +By default, a new FITS file is created and the binary table is written +to the first extension. If the <B>-a</B> (append) switch is specified, +the table is appended to an existing FITS file as a BINTABLE extension. +Note that the output FITS file must already exist. + +<P> +If the <B>-z</B> ("zero" pixel values) switch is specified and +<B>-i</B> is not specified, then pixels having a zero value will +be output with their "VALUE" column set to zero. Obviously, this +switch does not make sense when individual events are output. + +<!-- =defdoc funtbl funtbl 1 --> + +<!-- =section funtbl NAME --> +<H2><A NAME="funtbl">funtbl - extract a table from Funtools ASCII output</A></H2> + +<!-- =section funtbl SYNOPSIS --> +<B> +<PRE> +funtable [-c cols] [-h] [-n table] [-p prog] [-s sep] <iname> +</PRE> +</B> + +<!-- =section funtbl DESCRIPTION --> +<P> +[NB: This program has been deprecated in favor of the ASCII text processing +support in funtools. You can now perform fundisp on funtools ASCII output +files (specifying the table using bracket notation) to extract tables +and columns.] + +The <B>funtbl</B> script extracts a specified table (without the +header and comments) from a funtools ASCII output file and writes the +result to the standard output. The first non-switch argument is the +ASCII input file name (i.e. the saved output from funcnts, fundisp, +funhist, etc.). If no filename is specified, stdin is read. The +-n switch specifies which table (starting from 1) to extract. The +default is to extract the first table. The -c switch is a +space-delimited list of column numbers to output, e.g. -c "1 3 5" +will extract the first three odd-numbered columns. The default is to +extract all columns. The -s switch specifies the separator string to +put between columns. The default is a single space. The -h switch +specifies that column names should be added in a header line before +the data is output. Without the switch, no header is prepended. The +-p program switch allows you to specify an awk-like program to run +instead of the default (which is host-specific and is determined at +build time). The -T switch will output the data in rdb format (i.e., +with a 2-row header of column names and dashes, and with data columns +separated by tabs). The -help switch will print out a message +describing program usage. + +<P> +For example, consider the output from the following funcnts command: +<PRE> + [sh] funcnts -sr snr.ev "ann 512 512 0 9 n=3" + # source + # data file: /proj/rd/data/snr.ev + # arcsec/pixel: 8 + # background + # constant value: 0.000000 + # column units + # area: arcsec**2 + # surf_bri: cnts/arcsec**2 + # surf_err: cnts/arcsec**2 + + # summed background-subtracted results + upto net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 + 2 625.000 25.000 0.000 0.000 6976.00 0.090 0.004 + 3 1442.000 37.974 0.000 0.000 15936.00 0.090 0.002 + + + # background-subtracted results + reg net_counts error background berror area surf_bri surf_err + ---- ------------ --------- ------------ --------- --------- --------- --------- + 1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008 + 2 478.000 21.863 0.000 0.000 5376.00 0.089 0.004 + 3 817.000 28.583 0.000 0.000 8960.00 0.091 0.003 + + + # the following source and background components were used: + source_region(s) + ---------------- + ann 512 512 0 9 n=3 + + reg counts pixels sumcnts sumpix + ---- ------------ --------- ------------ --------- + 1 147.000 25 147.000 25 + 2 478.000 84 625.000 109 + 3 817.000 140 1442.000 249 +</PRE> +<P> +There are four tables in this output. To extract the last one, you +can execute: +<PRE> + [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -n 4 + 1 147.000 25 147.000 25 + 2 478.000 84 625.000 109 + 3 817.000 140 1442.000 249 +</PRE> +Note that the output has been re-formatted so that only a single space +separates each column, with no extraneous header or comment information. + +<P> +To extract only columns 1,2, and 4 from the last example (but with a header +prepended and tabs between columns), you can execute: +<PRE> + [sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -c "1 2 4" -h -n 4 -s "\t" + #reg counts sumcnts + 1 147.000 147.000 + 2 478.000 625.000 + 3 817.000 1442.000 +</PRE> +<P> +Of course, if the output has previously been saved in a file named +foo.out, the same result can be obtained by executing: +<PRE> + [sh] funtbl -c "1 2 4" -h -n 4 -s "\t" foo.out + #reg counts sumcnts + 1 147.000 147.000 + 2 478.000 625.000 + 3 817.000 1442.000 +</PRE> + +<!-- =section funcalc SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funcen SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funcnts SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funcone SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section fundisp SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funhead SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funhist SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funimage SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funindex SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funjoin SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funmerge SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funsky SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funtable SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =section funtbl SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: April 1, 2007</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/regalgebra.html b/funtools/doc/regalgebra.html new file mode 100644 index 0000000..c31dbf7 --- /dev/null +++ b/funtools/doc/regalgebra.html @@ -0,0 +1,278 @@ +<!-- =defdoc regalgebra regalgebra n --> +<HTML> +<HEAD> +<TITLE>Region Algebra</TITLE> +</HEAD> +<BODY> + +<!-- =section regalgebra NAME --> +<H2><A NAME="regalgebra">RegAlgebra: Boolean Algebra on Spatial Regions</A></H2> + +<!-- =section regalgebra SYNOPSIS --> +<H2>Summary</H2> +<P> +This document describes the boolean arithmetic defined for +region expressions. + +<!-- =section regalgebra DESCRIPTION --> +<H2>Description</H2> +<P> +When defining a region, several shapes can be combined using boolean +operations. The boolean operators are (in order of precedence): +<PRE> + Symbol Operator Associativity + ------ -------- ------------- + ! not right to left + & and left to right + ^ exclusive or left to right + | inclusive or left to right + +</PRE> +For example, to create a mask consisting of a large circle with a +smaller box removed, one can use the <B>and</B> and <B>not</B> +operators: +<PRE> + CIRCLE(11,11,15) & !BOX(11,11,3,6) +</PRE> + +and the resulting mask is: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:1111111111111111111111.................. + 2:1111111111111111111111.................. + 3:11111111111111111111111................. + 4:111111111111111111111111................ + 5:111111111111111111111111................ + 6:1111111111111111111111111............... + 7:1111111111111111111111111............... + 8:1111111111111111111111111............... + 9:111111111...1111111111111............... + 10:111111111...1111111111111............... + 11:111111111...1111111111111............... + 12:111111111...1111111111111............... + 13:111111111...1111111111111............... + 14:111111111...1111111111111............... + 15:1111111111111111111111111............... + 16:1111111111111111111111111............... + 17:111111111111111111111111................ + 18:111111111111111111111111................ + 19:11111111111111111111111................. + 20:1111111111111111111111.................. + 21:1111111111111111111111.................. + 22:111111111111111111111................... + 23:..11111111111111111..................... + 24:...111111111111111...................... + 25:.....11111111111........................ + 26:........................................ + 27:........................................ + 28:........................................ + 29:........................................ + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................ +</PRE> +A three-quarter circle can be defined as: +<PRE> + CIRCLE(20,20,10) & !PIE(20,20,270,360) +</PRE> + +and looks as follows: +<PRE> + + 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:........................................ + 2:........................................ + 3:........................................ + 4:........................................ + 5:........................................ + 6:........................................ + 7:........................................ + 8:........................................ + 9:........................................ + 10:........................................ + 11:...............111111111................ + 12:..............11111111111............... + 13:............111111111111111............. + 14:............111111111111111............. + 15:...........11111111111111111............ + 16:..........1111111111111111111........... + 17:..........1111111111111111111........... + 18:..........1111111111111111111........... + 19:..........1111111111111111111........... + 20:..........1111111111111111111........... + 21:..........1111111111.................... + 22:..........1111111111.................... + 23:..........1111111111.................... + 24:..........1111111111.................... + 25:...........111111111.................... + 26:............11111111.................... + 27:............11111111.................... + 28:..............111111.................... + 29:...............11111.................... + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................ +</PRE> +Two non-intersecting ellipses can be made into the same region: +<PRE> + ELL(20,20,10,20,90) | ELL(1,1,20,10,0) +</PRE> + +and looks as follows: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 1:11111111111111111111.................... + 2:11111111111111111111.................... + 3:11111111111111111111.................... + 4:11111111111111111111.................... + 5:1111111111111111111..................... + 6:111111111111111111...................... + 7:1111111111111111........................ + 8:111111111111111......................... + 9:111111111111............................ + 10:111111111............................... + 11:...........11111111111111111............ + 12:........111111111111111111111111........ + 13:.....11111111111111111111111111111...... + 14:....11111111111111111111111111111111.... + 15:..11111111111111111111111111111111111... + 16:.1111111111111111111111111111111111111.. + 17:111111111111111111111111111111111111111. + 18:111111111111111111111111111111111111111. + 19:111111111111111111111111111111111111111. + 20:111111111111111111111111111111111111111. + 21:111111111111111111111111111111111111111. + 22:111111111111111111111111111111111111111. + 23:111111111111111111111111111111111111111. + 24:.1111111111111111111111111111111111111.. + 25:..11111111111111111111111111111111111... + 26:...11111111111111111111111111111111..... + 27:.....11111111111111111111111111111...... + 28:.......111111111111111111111111......... + 29:...........11111111111111111............ + 30:........................................ + 31:........................................ + 32:........................................ + 33:........................................ + 34:........................................ + 35:........................................ + 36:........................................ + 37:........................................ + 38:........................................ + 39:........................................ + 40:........................................ +</PRE> +You can use several boolean operations in a single region expression, +to create arbitrarily complex regions. With the important exception +below, you can apply the operators in any order, using parentheses if +necessary to override the natural precedences of the operators. + +<P> +NB: Using a panda shape is always much more efficient than explicitly +specifying "pie & annulus", due to the ability of panda to place a +limit on the number of pixels checked in the pie shape. If you are +going to specify the intersection of pie and annulus, use panda +instead. + +<P> +As described in "help regreometry", the <B>PIE</B> slice goes to the +edge of the field. To limit its scope, <B>PIE</B> usually is is +combined with other shapes, such as circles and annuli, using boolean +operations. In this context, it is worth noting that that there is a +difference between <B>-PIE</B> and <B>&!PIE</B>. The former is a +global exclude of all pixels in the <B>PIE</B> slice, while the latter +is a local excludes of pixels affecting only the region(s) with which +the <B>PIE</B> is combined. For example, the following region uses +<B>&!PIE</B> as a local exclude of a single circle. Two other circles +are also defined and are unaffected by the local exclude: +<PRE> + + CIRCLE(1,8,1) + CIRCLE(8,8,7)&!PIE(8,8,60,120)&!PIE(8,8,240,300) + CIRCLE(15,8,2) + + 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 + - - - - - - - - - - - - - - - + 15: . . . . . . . . . . . . . . . + 14: . . . . 2 2 2 2 2 2 2 . . . . + 13: . . . 2 2 2 2 2 2 2 2 2 . . . + 12: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 11: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 10: . . . . 2 2 2 2 2 2 2 . . . . + 9: . . . . . . 2 2 2 . . . . 3 3 + 8: 1 . . . . . . . . . . . . 3 3 + 7: . . . . . . 2 2 2 . . . . 3 3 + 6: . . . . 2 2 2 2 2 2 2 . . . . + 5: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 4: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 3: . . . 2 2 2 2 2 2 2 2 2 . . . + 2: . . . . 2 2 2 2 2 2 2 . . . . + 1: . . . . . . . . . . . . . . . + +</PRE> +Note that the two other regions are not affected by the <B>&!PIE</B>, +which only affects the circle with which it is combined. + +<P> +On the other hand, a <B>-PIE</B> is an global exclude that does +affect other regions with which it overlaps: +<PRE> + + CIRCLE(1,8,1) + CIRCLE(8,8,7) + -PIE(8,8,60,120) + -PIE(8,8,240,300) + CIRCLE(15,8,2) + + 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 + - - - - - - - - - - - - - - - + 15: . . . . . . . . . . . . . . . + 14: . . . . 2 2 2 2 2 2 2 . . . . + 13: . . . 2 2 2 2 2 2 2 2 2 . . . + 12: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 11: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 10: . . . . 2 2 2 2 2 2 2 . . . . + 9: . . . . . . 2 2 2 . . . . . . + 8: . . . . . . . . . . . . . . . + 7: . . . . . . 2 2 2 . . . . . . + 6: . . . . 2 2 2 2 2 2 2 . . . . + 5: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 4: . . 2 2 2 2 2 2 2 2 2 2 2 . . + 3: . . . 2 2 2 2 2 2 2 2 2 . . . + 2: . . . . 2 2 2 2 2 2 2 . . . . + 1: . . . . . . . . . . . . . . . + +</PRE> +The two smaller circles are entirely contained within the two exclude +<B>PIE</B> slices and therefore are excluded from the region. + +<!-- =section regalgebra SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 17, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/regbounds.html b/funtools/doc/regbounds.html new file mode 100644 index 0000000..4fef30f --- /dev/null +++ b/funtools/doc/regbounds.html @@ -0,0 +1,179 @@ +<!-- =defdoc regbounds regbounds n --> +<HTML> +<HEAD> +<TITLE>Spatial Region Boundaries</TITLE> +</HEAD> +<BODY> + +<!-- =section regbounds NAME --> +<H2><A NAME="regbounds">RegBounds: Region Boundaries</A></H2> + +<!-- =section regbounds SYNOPSIS --> +<H2>Summary</H2> +Describes how spatial region boundaries are handled. + +<!-- =section regbounds DESCRIPTION --> +<H2>Description</H2> +<P> +The golden rule for spatial region filtering was first enunciated by +Leon VanSpeybroeck in 1986: + +<P> +<B>Each photon will be counted once, and no photon will be counted +more than once</B>. + +<P> +This means that we must be careful about boundary +conditions. For example, if a circle is contained in an annulus such +that the inner radius of the annulus is the same as the radius of the +circle, then photons on that boundary must always be assigned to one +or the other region. That is, the number of photons in both regions +must equal the sum of the number of photons in each region taken +separately. + +With this in mind, the rules for determining whether a boundary image +pixel or table row are assigned to a region are defined below. + +<H2>Image boundaries : radially-symmetric shapes (circle, annuli, ellipse)</H2> + +For image filtering, pixels whose center is inside the boundary are +included. This also applies non-radially-symmetric shapes. When a +pixel center is exactly on the boundary, the pixel assignment rule is: + +<UL> +<LI> the outer boundary of a symmetric shape does not include such pixels +<LI> the inner boundary of a symmetric shape (annulus) includes such pixels +</UL> + +In this way, an annulus with radius from 0 to 1, centered exactly on a +pixel, includes the pixel on which it is centered, but none of its +neighbors. + +These rules ensure that when defining concentric shapes, no pixels are +omitted between concentric regions and no pixels are claimed by two +regions. When applied to small symmetric shapes, the shape is less +likely to be skewed, as would happen with non-radially-symmetric +rules. These rules differ from the rules for box-like shapes, which +are more likely to be positioned adjacent to one another. + +<H2>Image Boundaries: non-radially symmetric shapes (polygons, boxes)</H2> + +For image filtering, pixels whose center is inside the boundary are +included. This also applies radially-symmetric shapes. When a pixel +center is exactly on the boundary of a non-radially symmetric region, +the pixel is included in the right or upper region, but not the left +or lower region. This ensures that geometrically adjoining regions +touch but don't overlap. + +<H2>Row Boundaries are Analytic</H2> + +When filtering table rows, the boundary rules are the same as for +images, except that the calculation is not done on the center of a +pixel, (since table rows, especially X-ray events rows, often have +discrete, floating point positions) but are calculated exactly. That +is, an row is inside the boundary without regard to its integerized +pixel value. For rows that are exactly on a region boundary, the +above rules are applied to ensure that all rows are counted once and +no row is counted more than once. + +<P> +Because row boundaries are calculated differently from image boundaries, +certain programs will give different results when filtering the same +region file. In particular, fundisp/funtable (which utilize analytic +row filtering) perform differently from funcnts (which performs image +filtering, even on tables). + +<H2>Image Boundaries vs. Row Boundaries: Practical Considerations</H2> + +<P> +You will sometimes notice a discrepancy between running funcnts on an +binary table file and running fundisp on the same file with the same filter. +For example, consider the following: +<PRE> + fundisp test1.fits"[box(4219,3887,6,6,0)]" | wc + 8893 320148 3752846 +</PRE> +Since fundisp has a 2-line header, there are actually 8891 photons +that pass the filter. But then run funtable and select only the +rows that pass this filter, placing them in a new file: +<PRE> + ./funtable test1.fits"[box(4219,3887,6,6,0)]" test2.fits +</PRE> +Now run funcnts using the original filter on the derived file: +<PRE> + ./funcnts test2.fits "physical; box(4219,3887,6,6,0)" + + [... lot of processed output ...] + + # the following source and background components were used: + source region(s) + ---------------- + physical; box(4219,3887,6,6,0) + + reg counts pixels + ---- ------------ --------- + 1 7847.000 36 +</PRE> +There are 1044 rows (events) that pass the row filter in fundisp (or +funtable) but fail to make it through funcnts. Why? + +<P> +The reason can be traced to how analytic row filtering (fundisp, funtable) +differs from integerized pixel filtering(funcnts, funimage). Consider the +region: +<PRE> + box(4219,3887,6,6,0) +</PRE> +Analytically (i.e., using row filtering), positions will pass this +filter successfully if: +<PRE> + 4216 <= x <= 4222 + 3884 <= y <= 3890 +</PRE> +For example, photons with position values of x=4216.4 or y=3884.08 will pass. + +<P> +Integerized image filtering is different in that the pixels that will +pass this filter have centers at: +<PRE> + x = 4217, 4218, 4219, 4220, 4221, 4222 + y = 3885, 3886, 3887, 3888, 3889, 3890 +</PRE> +Note that there are 6 pixels in each direction, as specified by the region. +That means that positions will pass the filter successfully if: +<PRE> + 4217 <= (int)x <= 4222 + 3885 <= (int)y <= 3890 +</PRE> +Photons with position values of x=4216.4 or y=3884.08 will NOT pass. + +<P> +Note that the position values are integerized, in effect, binned into +image values. This means that x=4222.4 will pass this filter, but not +the analytic filter above. We do this to maintain the design goal that +either all counts in a pixel are included in an integerized filter, or +else none are included. + +<P> +[It could be argued that the correct photon limits for floating point +row data really should be: +<PRE> + 4216.5 <= x <= 4222.5 + 3884.5 <= y <= 3890.5 +</PRE> +since each pixel extends for .5 on either side of the center. We chose +to the maintain integerized algorithm for all image-style filtering so +that funcnts would give the exact same results regardless of whether +a table or a derived non-blocked binned image is used.] + +<!-- =section regbounds SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 16, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/regcoords.html b/funtools/doc/regcoords.html new file mode 100644 index 0000000..ff656c2 --- /dev/null +++ b/funtools/doc/regcoords.html @@ -0,0 +1,239 @@ +<!-- =defdoc regcoords regcoords n --> +<HTML> +<HEAD> +<TITLE>Spatial Region Coordinates</TITLE> +</HEAD> +<BODY> + +<!-- =section regcoords NAME --> +<H2><A NAME="regcoords">RegCoords: Spatial Region Coordinates</A></H2> + +<!-- =section regcoords SYNOPSIS --> +<H2>Summary</H2> +<P> +This document describes the specification of coordinate systems, and the +interpretation of coordinate values, for spatial region filtering. + +<!-- =section regcoords DESCRIPTION --> +<H2>Pixel coordinate systems</H2> +<P> +The default coordinate system for regions is PHYSICAL, which means +that region position and size values are taken from the original +data. (Note that this is a change from the original IRAF/PROS +implementation, in which the IMAGE coordinate system was the default.) +PHYSICAL coordinates always refer to pixel positions on the original +image (using IRAF LTM and LTV keywords). With PHYSICAL coordinates, +if a set of coordinates specifies the position of an object in an +original FITS file, the same coordinates will specify the same object +in any FITS derived from the original. Physical coordinates are +invariant with blocking of FITS files or taking sections of images, +even when a blocked section is written to a new file. + +<P> +Thus, although a value in pixels refers, by default, to the PHYSICAL +coordinate system, you may specify that position values refer to the +image coordinate system using the <B>global</B> or <B>local</B> +properties commands: + +<PRE> + global coordsys image + circle 512 512 100 +</PRE> + +The <B>global</B> command changes the coordinate system for all +regions that follow, while the <B>local</B> command changes the +coordinate system only for the region immediately following: +<PRE> + local coordsys image + circle 512 512 100 + circle 1024 1024 200 +</PRE> +This changes the coordinate system only for the region that follows. +In the above example, the second region uses the global coordinate +system (PHYSICAL by default). + +<P> +<H2>World Coordinate Systems</H2> + +If World Coordinate System information is contained in the data file +being filtered, it also is possible to define regions using a sky +coordinate system. Supported systems include: + +<PRE> + name description + ---- ----------- + PHYSICAL pixel coords of original file using LTM/LTV + IMAGE pixel coords of current file + FK4, B1950 sky coordinate systems + FK5, J2000 sky coordinate systems + GALACTIC sky coordinate systems + ECLIPTIC sky coordinate systems + ICRS currently same as J2000 + LINEAR linear wcs as defined in file +</PRE> + +In addition, two mosaic coordinate systems have been defined that +utilize the (evolving) IRAF mosaic keywords: + +<PRE> + name description + ---- ----------- + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV +</PRE> +Again, to use one of these coordinate systems, the <B>global</B> or +<B>local</B> properties commands are used: + +<PRE> + global coordsys galactic +</PRE> + +<H2>WCS Positions and Sizes</H2> + +In addition to pixels, positional values in a WCS-enabled region can +be specified using sexagesimal or degrees format: + +<PRE> + position arguments description + ------------------ ----------- + [num] context-dependent (see below) + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + [num]:[num]:[num] hms for 'odd' position arguments + [num]:[num]:[num] dms for 'even' position arguments + [num]h[num]m[num]s explicit hms + [num]d[num]m[num]s explicit dms +</PRE> + +If ':' is used as sexagesimal separator, the value is considered to be +specifying hours/minutes/seconds if it is the first argument of a +positional pair, and degrees/minutes/seconds for the second argument +of a pair (except for galactic coordinates, which always use degrees): + +<PRE> + argument description + ----------- ----------- + 10:20:30.0 10 hours, 20 minutes, 30 seconds for 1st positional argument + 10 degrees, 20 minutes, 30 seconds for 2nd positional argument + 10h20m30.0 10 hours, 20 minutes, 30 seconds + 10d20m30.0 10 degrees, 20 minutes, 30 seconds + 10.20d 10.2 degrees +</PRE> + +Similarly, the units of size values are defined by the formating +character(s) attached to a number: + +<PRE> + size arguments description + -------------- ----------- + [num] context-dependent (see below) + [num]" arc seconds + [num]' arc minutes + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels +</PRE> + +For example: +<PRE> + argument description + ----------- ----------- + 10 ten pixels + 10' ten minutes of arc + 10" ten seconds of arc + 10d ten degrees + 10p ten pixels + 0.5r half of a radian +</PRE> + +<P> +An example of using sky coordinate systems follows: + +<PRE> + global coordsys B1950 + -box 175.54d 20.01156d 10' 10' + local coordsys J2000 + pie 179.57d 22.4d 0 360 n=4 && annulus 179.57d 22.4d 3' 24' n=5 +</PRE> + +At the FK4 1950 coordinates 175.54d RA, 20.01156d DEC exclude a 10 +minute by 10 minute box. Then at the FK5 2000 coordinates 179.57d RA +22.4d DEC draw a radial profile regions pattern with 4 quadrants and 5 +annuli ranging from 3 minutes to 24 minutes in diameter. In this +example, the default coordinate system is overridden by the commands +in the regions spec. + +<H2>NB: The Meaning of Pure Numbers Are Context Sensitive</H2> + +<P> +When a "pure number" (i.e. one without a format directive such as 'd' +for 'degrees') is specified as a position or size, its interpretation +depends on the context defined by the 'coordsys' keyword. In general, +the rule is: + +<P> +<B>All pure numbers have implied units corresponding to the current +coordinate system.</B> + +<P> +If no coordinate system is explicitly specified, the default system is +implicitly assumed to be PHYSICAL. In practice this means that for +IMAGE and PHYSICAL systems, pure numbers are pixels. Otherwise, +for all systems other than LINEAR, pure numbers are degrees. For +LINEAR systems, pure numbers are in the units of the linear system. +This rule covers both positions and sizes. + +<P> +As a corollary, when a sky-formatted number is used with the IMAGE +or PHYSICAL coordinate system (which includes the default case of no +coordsys being specified), the formatted number is assumed to be in +the units of the WCS contained in the current file. If no sky WCS is +specified, an error results. + +<P> +Examples: + +<PRE> + circle(512,512,10) + ellipse 202.44382d 47.181656d 0.01d 0.02d +</PRE> + +<P> +In the absence of a specified coordinate system, the circle uses the +default PHYSICAL units of pixels, while the ellipse explicitly uses degrees, +presumably to go with the WCS in the current file. + +<PRE> + global coordsys=fk5 + global color=green font="system 10 normal" + circle 202.44382 47.181656 0.01 + circle 202.44382 47.181656 10p + ellipse(512p,512p,10p,15p,20) +</PRE> + + +<P> +Here, the circles use the FK5 units of degrees (except for the +explicit use of pixels in the second radius), while the ellipse +explicitly specifies pixels. The ellipse angle is in degrees. + +<P> +Note that Chandra data format appears to use "coordsys=physical" +implicitly. Therefore, for most Chandra applications, valid regions +can be generated safely by asking ds9 to save/display regions in +pixels using the PHYSICAL coordsys. + +<!-- =section regcoords SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 17, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/regdiff.html b/funtools/doc/regdiff.html new file mode 100644 index 0000000..29c9658 --- /dev/null +++ b/funtools/doc/regdiff.html @@ -0,0 +1,71 @@ +<!-- =defdoc regdiff regdiff n --> +<HTML> +<HEAD> +<TITLE>Differences Between Funtools and IRAF Regions</TITLE> +</HEAD> +<BODY> + +<!-- =section regdiff NAME --> +<H2><A NAME="regdiff">RegDiff:Differences Between Funtools and IRAF Regions</A></H2> + +<!-- =section regdiff SYNOPSIS --> +<H2>Summary</H2> +Describes the differences between Funtools/ds9 regions and the old IRAF/PROS +regions. + +<!-- =section regdiff DESCRIPTION --> +<H2>Description</H2> +<P> +We have tried to make Funtools regions compatible with their +predecessor, IRAF/PROS regions. For simple regions and simple boolean +algebra between regions, there should be no difference between the two +implementations. The following is a list of differences and +incompatibilities between the two: + +<UL> + +<P> +<LI>If a pixel is covered by two different regions expressions, +Funtools assigns the mask value of the <B>first</B> region that +contains that pixel. That is, successive regions <B>do not</b> +overwrite previous regions in the mask, as was the case with the +original PROS regions. This means that one must define overlapping +regions in the reverse order in which they were defined in PROS. If +region N is fully contained within region M, then N should be defined +<B>before</B> M, or else it will be "covered up" by the latter. This +change is necessitated by the use of optimized filter compilation, i.e., +Funtools only tests individual regions until a proper match is made. + +<P> +<LI> The <B>PANDA</B> region has replaced the old PROS syntax in which +a <B>PIE</B> accelerator was combined with an <B>ANNULUS</B> accelerator +using <B>AND</B>. That is, +<PRE> + ANNULUS(20,20,0,15,n=4) & PIE(20,20,0,360,n=3) +</PRE> +has been replaced by: +<PRE> + PANDA(20,20,0,360,3,0,15,4) +</PRE> +The PROS syntax was inconsistent with the meaning of the <B>AND</B> operator. + +<P> +<LI> The meaning of pure numbers (i.e., without format specifiers) in +regions has been clarified, as has the syntax for specifying coordinate +systems. See the general discussion on +<A HREF="./regions.html">Spatial Region Filtering</A> +for more information. + +</UL> + +<!-- =section regdiff SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 16, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/reggeometry.html b/funtools/doc/reggeometry.html new file mode 100644 index 0000000..faca53f --- /dev/null +++ b/funtools/doc/reggeometry.html @@ -0,0 +1,1148 @@ +<!-- =defdoc reggeometry reggeometry n --> +<HTML> +<HEAD> +<TITLE>Region Geometry</TITLE> +</HEAD> +<BODY> + +<!-- =section reggeometry NAME --> +<H2><A NAME="reggeometry">RegGeometry: Geometric Shapes in Spatial Region Filtering</A></H2> + +<!-- =section reggeometry SYNOPSIS --> +<H2>Summary</H2> +<P> +This document describes the geometry of regions available for spatial +filtering in IRAF/PROS analysis. + +<!-- =section reggeometry DESCRIPTION --> +<H2>Geometric shapes</H2> +<P> +Several geometric shapes are used to describe regions. The valid +shapes are: + +<PRE> + shape: arguments: + ----- ---------------------------------------- + ANNULUS xcenter ycenter inner_radius outer_radius + BOX xcenter ycenter xwidth yheight (angle) + CIRCLE xcenter ycenter radius + ELLIPSE xcenter ycenter xwidth yheight (angle) + FIELD none + LINE x1 y1 x2 y2 + PIE xcenter ycenter angle1 angle2 + POINT x1 y1 + POLYGON x1 y1 x2 y2 ... xn yn +</PRE> + + +All arguments are real values; integer values are automatically +converted to real where necessary. All angles are in degrees and +specify angles that run counter-clockwise from the positive y-axis. + +<P> +Shapes can be specified using "command" syntax: +<PRE> + [shape] arg1 arg2 ... +</PRE> +or using "routine" syntax: +<PRE> + [shape](arg1, arg2, ...) +</PRE> +or by any combination of the these. (Of course, the parentheses must +balance and there cannot be more commas than necessary.) The shape +keywords are case-insensitive. Furthermore, any shape can be +specified by a three-character unique abbreviation. For example, one +can specify three circular regions as: + +<PRE> + "foo.fits[CIRCLE 512 512 50;CIR(128 128, 10);cir(650,650,20)]" +</PRE> + +(Quotes generally are required to protect the region descriptor +from being processed by the Unix shell.) + +<HR> + +<P> +The <B>annulus</B> shape specifies annuli, centered at xcenter, +ycenter, with inner and outer radii (r1, r2). For example, +<PRE> + ANNULUS 25 25 5 10 +</PRE> +specifies an annulus centered at 25.0 25.0 with an inner radius of 5.0 and +an outer radius of 10. Assuming (as will be done for all examples in this +document, unless otherwise noted) this shape is used in a mask of size 40x40, +it will look like this: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:....................111111111........... + 33:...................11111111111.......... + 32:.................111111111111111........ + 31:.................111111111111111........ + 30:................11111111111111111....... + 29:...............1111111.....1111111...... + 28:...............111111.......111111...... + 27:...............11111.........11111...... + 26:...............11111.........11111...... + 25:...............11111.........11111...... + 24:...............11111.........11111...... + 23:...............11111.........11111...... + 22:...............111111.......111111...... + 21:...............1111111.....1111111...... + 20:................11111111111111111....... + 19:.................111111111111111........ + 18:.................111111111111111........ + 17:...................11111111111.......... + 16:....................111111111........... + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................ + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> + +<HR> + +<P> +The <B>box</B> shape specifies an orthogonally oriented box, +centered at xcenter, ycenter, of size xwidth, yheight. It requires four +arguments and accepts an optional fifth argument to specify a rotation angle. +When the rotation angle is specified (in degrees), the box is rotated by +an angle that runs counter-clockwise from the positive y-axis. + +<P> +The <B>box</B> shape specifies a rotated box, centered at +xcenter, ycenter, of size xwidth, yheight. The box is rotated by an angle +specified in degrees that runs counter-clockwise from the positive y-axis. +If the angle argument is omitted, it defaults to 0. + +<HR> + +<P> +The <B>circle</B> shape specifies a circle, centered at xcenter, +ycenter, of radius r. It requires three arguments. + +<HR> + +<P> +The <B>ellipse</B> shape specifies an ellipse, centered at +xcenter, ycenter, with y-axis width a and the y-axis length b defined such +that: +<PRE> + x**2/a**2 + y**2/b**2 = 1 +</PRE> +Note that a can be less than, equal to, or greater than b. The ellipse +is rotated the specified number of degrees. The rotation is done according +to astronomical convention, counter-clockwise from the positive y-axis. +An ellipse defined by: +<PRE> + ELLIPSE 20 20 5 10 45 +</PRE> +will look like this: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:............111111...................... + 26:............11111111.................... + 25:............111111111................... + 24:............11111111111................. + 23:............111111111111................ + 22:............111111111111................ + 21:.............111111111111............... + 20:.............1111111111111.............. + 19:..............111111111111.............. + 18:...............111111111111............. + 17:...............111111111111............. + 16:................11111111111............. + 15:..................111111111............. + 14:...................11111111............. + 13:.....................111111............. + 12:........................................ + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> + + +<HR> + +<P> +The <B>field</B> shape specifies the entire field as a +region. It is not usually specified explicitly, but is used implicitly in the +case where no regions are specified, that is, in cases where either a null +string or some abbreviation of the string "none" is input. +<B>Field</B> takes no arguments. + +<HR> + +<P> +The <B>pie</B> shape specifies an angular wedge of the entire field, +centered at xcenter, ycenter. The wedge runs between the two specified angles. +The angles are given in degrees, running counter-clockwise from the positive +x-axis. For example, +<PRE> + PIE 20 20 90 180 +</PRE> +defines a region from 90 degrees to 180 degrees, i.e., quadrant 2 of the +Cartesian plane. The display of such a region looks like this: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:11111111111111111111.................... + 39:11111111111111111111.................... + 38:11111111111111111111.................... + 37:11111111111111111111.................... + 36:11111111111111111111.................... + 35:11111111111111111111.................... + 34:11111111111111111111.................... + 33:11111111111111111111.................... + 32:11111111111111111111.................... + 31:11111111111111111111.................... + 30:11111111111111111111.................... + 29:11111111111111111111.................... + 28:11111111111111111111.................... + 27:11111111111111111111.................... + 26:11111111111111111111.................... + 25:11111111111111111111.................... + 24:11111111111111111111.................... + 23:11111111111111111111.................... + 22:11111111111111111111.................... + 21:11111111111111111111.................... + 20:........................................ + 19:........................................ + 18:........................................ + 17:........................................ + 16:........................................ + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................ + 11:........................................ + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> +The pie slice specified is always a counter-clockwise sweep between +the angles, starting at the first angle and ending at the second. Thus: +<PRE> + PIE 10 15 30 60 +</PRE> +describes a 30 degree sweep from 2 o'clock to 1 o'clock, while: +<PRE> + PIE 10 15 60 30 +</PRE> +describes a 330 degree counter-clockwise sweep from 1 o'clock to 2 o'clock +passing through 12 o'clock (0 degrees). Note in both of these examples that +the center of the slice can be anywhere on the plane. The second mask looks +like this: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:111111111111111111111111................ + 39:11111111111111111111111................. + 38:11111111111111111111111................. + 37:1111111111111111111111.................. + 36:1111111111111111111111.................. + 35:111111111111111111111................... + 34:11111111111111111111.................... + 33:11111111111111111111.................... + 32:1111111111111111111....................1 + 31:1111111111111111111..................111 + 30:111111111111111111.................11111 + 29:111111111111111111................111111 + 28:11111111111111111...............11111111 + 27:1111111111111111..............1111111111 + 26:1111111111111111.............11111111111 + 25:111111111111111............1111111111111 + 24:111111111111111..........111111111111111 + 23:11111111111111.........11111111111111111 + 22:11111111111111........111111111111111111 + 21:1111111111111.......11111111111111111111 + 20:111111111111......1111111111111111111111 + 19:111111111111....111111111111111111111111 + 18:11111111111....1111111111111111111111111 + 17:11111111111..111111111111111111111111111 + 16:1111111111.11111111111111111111111111111 + 15:1111111111111111111111111111111111111111 + 14:1111111111111111111111111111111111111111 + 13:1111111111111111111111111111111111111111 + 12:1111111111111111111111111111111111111111 + 11:1111111111111111111111111111111111111111 + 10:1111111111111111111111111111111111111111 + 9:1111111111111111111111111111111111111111 + 8:1111111111111111111111111111111111111111 + 7:1111111111111111111111111111111111111111 + 6:1111111111111111111111111111111111111111 + 5:1111111111111111111111111111111111111111 + 4:1111111111111111111111111111111111111111 + 3:1111111111111111111111111111111111111111 + 2:1111111111111111111111111111111111111111 + 1:1111111111111111111111111111111111111111 +</PRE> +The pie slice goes to the edge of the field. To limit its scope, pie +usually is is combined with other shapes, such as circles and annuli, +using boolean operations. (See below and in "help regalgebra"). + +<P> +Pie Performance Notes: +<P> +Pie region processing time is proportional to the size of the image, +and not the size of the region. This is because the pie shape is the +only infinite length shape, and we essentially must check all y rows +for inclusion (unlike other regions, where the y limits can be +calculated beforehand). Thus, pie can run very slowly on large images. +In particular, it will run MUCH more slowly than the panda shape in +image-based region operations (such as funcnts). We recommend use of +panda over pie where ever possible. + +<P> +If you must use pie, always try to put it last in a boolean && +expression. The reason for this is that the filter code is optimized +to exit as soon as the result is know. Since pie is the slowest +region, it is better to avoid executing it if another region can decide +the result. Consider, for example, the difference in time required to +process a Chandra ACIS file when a pie and circle are combined in +two different orders: + +<PRE> + time ./funcnts nacis.fits "circle 4096 4096 100 && pie 4096 4096 10 78" +2.87u 0.38s 0:35.08 9.2% + + time ./funcnts nacis.fits "pie 4096 4096 10 78 && circle 4096 4096 100 " +89.73u 0.36s 1:03.50 141.8% +</PRE> + +<P> +Black-magic performance note: + +<P> +Panda region processing uses a <B>quick test</B> pie region instead of +the normal pie region when combining its annulus and pie shapes. This +<B>qtpie</B> shape differs from the normal pie in that it utilizes the +y limits from the previous region with which it is combined. In a +panda shape, which is a series of annuli combined with pies, the +processing time is thus reduced to that of the annuli. + +<P> +You can use the qtpie shape instead of pie in cases where you are +combining pie with another shape using the && operator. This will +cause the pie limits to be set using limits from the other shape, and +will speed up the processing considerably. For example, the above +execution of funcnts can be improved considerably using this technique: + +<PRE> + time ./funcnts nacis.fits "circle 4096 4096 100 && qtpie 4096 4096 10 78" +4.66u 0.33s 0:05.87 85.0% +</PRE> + +<P> +We emphasize that this is a quasi-documented feature and might change in +the future. The qtpie shape is not recognized by ds9 or other programs. + +<HR> + +<P> +The <B>line</B> shape allows single pixels in a line between (x1,y1) and +(x2,y2) to be included or excluded. For example: +<PRE> + LINE (5,6, 24,25) +<PRE> +displays as: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:.......................1................ + 24:......................1................. + 23:.....................1.................. + 22:....................1................... + 21:...................1.................... + 20:..................1..................... + 19:.................1...................... + 18:................1....................... + 17:...............1........................ + 16:..............1......................... + 15:.............1.......................... + 14:............1........................... + 13:...........1............................ + 12:..........1............................. + 11:.........1.............................. + 10:........1............................... + 9:.......1................................ + 8:......1................................. + 7:.....1.................................. + 6:....1................................... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> + +<HR> + +<P> +The <B>point</B> shape allows single pixels to be included or +excluded. Although the (x,y) values are real numbers, they are truncated +to integer and the corresponding pixel is included or excluded, as specified. + +<P> +Several points can be put in one region declaration; unlike the +original IRAF implementation, each now is given a different region mask value. +This makes it easier, for example, for funcnts to determine the number of +photons in the individual pixels. For example, +<PRE> + POINT (5,6, 10,11, 20,20, 35,30) +</PRE> +will give the different region mask values to all four points, as shown below: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:..................................4..... + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:........................................ + 24:........................................ + 23:........................................ + 22:........................................ + 21:........................................ + 20:...................3.................... + 19:........................................ + 18:........................................ + 17:........................................ + 16:........................................ + 15:........................................ + 14:........................................ + 13:........................................ + 12:........................................ + 11:.........2.............................. + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:....1................................... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> + +<HR> + +<P> +The <B>polygon</B> shape specifies a polygon with vertices +(x1, y1) ... (xn, yn). The polygon is closed automatically: one should +not specify the last vertex to be the same as the first. Any number of +vertices are allowed. For example, the following polygon defines a +right triangle as shown below: +<PRE> + POLYGON (10,10, 10,30, 30,30) +</PRE> + +looks like this: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:..........11111111111111111111.......... + 29:..........1111111111111111111........... + 28:..........111111111111111111............ + 27:..........11111111111111111............. + 26:..........1111111111111111.............. + 25:..........111111111111111............... + 24:..........11111111111111................ + 23:..........1111111111111................. + 22:..........111111111111.................. + 21:..........11111111111................... + 20:..........1111111111.................... + 19:..........111111111..................... + 18:..........11111111...................... + 17:..........1111111....................... + 16:..........111111........................ + 15:..........11111......................... + 14:..........1111.......................... + 13:..........111........................... + 12:..........11............................ + 11:..........1............................. + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> +Note that polygons can get twisted upon themselves if edge lines +cross. Thus: +<PRE> + POL (10,10, 20,20, 20,10, 10,20) +</PRE> +will produce an area which is two triangles, like butterfly wings, as shown +below: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:........................................ + 33:........................................ + 32:........................................ + 31:........................................ + 30:........................................ + 29:........................................ + 28:........................................ + 27:........................................ + 26:........................................ + 25:........................................ + 24:........................................ + 23:........................................ + 22:........................................ + 21:........................................ + 20:........................................ + 19:..........1........1.................... + 18:..........11......11.................... + 17:..........111....111.................... + 16:..........1111..1111.................... + 15:..........1111111111.................... + 14:..........1111..1111.................... + 13:..........111....111.................... + 12:..........11......11.................... + 11:..........1........1.................... + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> + +<HR> + +<P> +The following are combinations of pie with different shapes +(called "panda" for "Pie AND Annulus") allow for easy specification of +radial sections: +<PRE> + shape: arguments: + ----- --------- + PANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + BPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # box + EPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # ellipse +</PRE> + +The <B>panda</B> (<B>P</B>ies <B>AND</B> <B>A</B>nnuli) shape can be +used to create combinations of pie and annuli markers. It is analogous +to a Cartesian product on those shapes, i.e., the result is several +shapes generated by performing a boolean AND between pies and +annuli. Thus, the panda and cpanda specify combinations of annulus and +circle with pie, respectively and give identical results. The bpanda +combines box and pie, while epanda combines ellipse and pie. + +<P> +Consider the example shown below: +<PRE> + PANDA(20,20, 0,360,3, 0,15,4) +</PRE> +Here, 3 pie slices centered at 20, 20 are combined with 4 annuli, also +centered at 20, 20. The result is a mask with 12 regions (displayed in +base 16 to save characters): +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:..............44444444444............... + 33:............444444444444444............. + 32:...........88444444444444444............ + 31:.........888844443333344444444.......... + 30:........88888833333333333444444......... + 29:........88888733333333333344444......... + 28:.......8888877733333333333344444........ + 27:......888887777332222233333344444....... + 26:......888877777622222222333334444....... + 25:.....88887777766622222222333334444...... + 24:.....88887777666622222222233334444...... + 23:.....88887777666651111222233334444...... + 22:.....88877776666551111122223333444...... + 21:.....88877776666555111122223333444...... + 20:.....888777766665559999aaaabbbbccc...... + 19:.....888777766665559999aaaabbbbccc...... + 18:.....888777766665599999aaaabbbbccc...... + 17:.....88887777666659999aaaabbbbcccc...... + 16:.....888877776666aaaaaaaaabbbbcccc...... + 15:.....888877777666aaaaaaaabbbbbcccc...... + 14:......8888777776aaaaaaaabbbbbcccc....... + 13:......888887777bbaaaaabbbbbbccccc....... + 12:.......88888777bbbbbbbbbbbbccccc........ + 11:........888887bbbbbbbbbbbbccccc......... + 10:........888888bbbbbbbbbbbcccccc......... + 9:.........8888ccccbbbbbcccccccc.......... + 8:...........88ccccccccccccccc............ + 7:............ccccccccccccccc............. + 6:..............ccccccccccc............... + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> + +<HR> + +<P> +Several regions with different mask values can be combined in the +same mask. This supports comparing data from the different regions. +(For information on how to combine different shapes into a single +region, see "help regalgebra".) For example, consider the following +set of regions: +<PRE> + ANNULUS 25 25 5 10 + ELLIPSE 20 20 5 10 315 + BOX 15 15 5 10 +</PRE> +The resulting mask will look as follows: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........................................ + 35:........................................ + 34:....................111111111........... + 33:...................11111111111.......... + 32:.................111111111111111........ + 31:.................111111111111111........ + 30:................11111111111111111....... + 29:...............1111111.....1111111...... + 28:...............111111.......111111...... + 27:...............11111.222222..11111...... + 26:...............111112222222..11111...... + 25:...............111112222222..11111...... + 24:...............111112222222..11111...... + 23:...............111112222222..11111...... + 22:...............111111222222.111111...... + 21:..............211111112222.1111111...... + 20:............322211111111111111111....... + 19:............32222111111111111111........ + 18:............22222111111111111111........ + 17:............222222211111111111.......... + 16:............22222222111111111........... + 15:............222222222................... + 14:............22222222.................... + 13:............222222...................... + 12:............33333....................... + 11:............33333....................... + 10:........................................ + 9:........................................ + 8:........................................ + 7:........................................ + 6:........................................ + 5:........................................ + 4:........................................ + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> +Note that when a pixel is in 2 or more regions, it is arbitrarily +assigned to a one of the regions in question (often based on how a +give C compiler optimizes boolean expressions). + +<P> +<H2>Region accelerators</H2> + +<P> +Two types of \fBaccelerators</B>, to simplify region specification, +are provided as natural extensions to the ways shapes are described. +These are: extended lists of parameters, specifying multiple regions, +valid for annulus, box, circle, ellipse, pie, and points; and +<B>n=<int></B>, valid for annulus, box, circle, ellipse, and pie (not +point). In both cases, one specification is used to define several +different regions, that is, to define shapes with different mask +values in the region mask. + +<P> +The following regions accept <B>accelerator</B> syntax: +<PRE> + shape arguments + ----- ------------------------------------------ + ANNULUS xcenter ycenter radius1 radius2 ... radiusn + ANNULUS xcenter ycenter inner_radius outer_radius n=[number] + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + BOX xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + ELLIPSE xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + PIE xcenter ycenter angle1 angle2 (angle3) (angle4) (angle5) ... + PIE xcenter ycenter angle1 angle2 (n=[number]) + POINT x1 y1 x2 y2 ... xn yn +</PRE> +Note that the circle accelerators are simply aliases for the annulus +accelerators. + +<P> +For example, several annuli at the same center can be specified in one +region expression by specifying more than two radii. If <B>N</B> +radii are specified, then <B>N</B>-1 annuli result, with the outer +radius of each preceding annulus being the inner radius of the +succeeding annulus. Each annulus is considered a separate region, and +is given a separate mask value. For example, +<PRE> + ANNULUS 20 20 0 2 5 10 15 20 +</PRE> +specifies five different annuli centered at 20 20, and is equivalent to: +<PRE> + ANNULUS 20.0 20.0 0 2 + ANNULUS 20.0 20.0 2 5 + ANNULUS 20.0 20.0 5 10 + ANNULUS 20.0 20.0 10 15 + ANNULUS 20.0 20.0 15 20 +</PRE> +The mask is shown below: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:.............5555555555555.............. + 38:...........55555555555555555............ + 37:.........555555555555555555555.......... + 36:........55555555555555555555555......... + 35:......555555555555555555555555555....... + 34:.....55555555544444444444555555555...... + 33:....5555555544444444444444455555555..... + 32:....5555555444444444444444445555555..... + 31:...555555444444444444444444444555555.... + 30:..55555544444444444444444444444555555... + 29:..55555544444443333333334444444555555... + 28:.5555554444444333333333334444444555555.. + 27:.5555544444433333333333333344444455555.. + 26:555555444444333333333333333444444555555. + 25:555554444443333333333333333344444455555. + 24:555554444433333332222233333334444455555. + 23:555554444433333322222223333334444455555. + 22:555554444433333222222222333334444455555. + 21:555554444433333222111222333334444455555. + 20:555554444433333222111222333334444455555. + 19:555554444433333222111222333334444455555. + 18:555554444433333222222222333334444455555. + 17:555554444433333322222223333334444455555. + 16:555554444433333332222233333334444455555. + 15:555554444443333333333333333344444455555. + 14:555555444444333333333333333444444555555. + 13:.5555544444433333333333333344444455555.. + 12:.5555554444444333333333334444444555555.. + 11:..55555544444443333333334444444555555... + 10:..55555544444444444444444444444555555... + 9:...555555444444444444444444444555555.... + 8:....5555555444444444444444445555555..... + 7:....5555555544444444444444455555555..... + 6:.....55555555544444444444555555555...... + 5:......555555555555555555555555555....... + 4:........55555555555555555555555......... + 3:.........555555555555555555555.......... + 2:...........55555555555555555............ + 1:.............5555555555555.............. +</PRE> + +<P> +For boxes and ellipses, if an odd number of arguments is specified, +then the last argument is assumed to be an angle. Otherwise, the +angle is assumed to be zero. For example: +<PRE> + ellipse 20 20 3 5 6 10 9 15 12 20 45 +</PRE> +specifies an 3 ellipses at a 45 degree angle: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:........................................ + 38:........................................ + 37:........................................ + 36:........33333333........................ + 35:......333333333333...................... + 34:.....3333333333333333................... + 33:....333333333333333333.................. + 32:....33333332222233333333................ + 31:...3333332222222222333333............... + 30:...33333222222222222233333.............. + 29:...333332222222222222223333............. + 28:...3333222222211112222223333............ + 27:...33332222211111111222223333........... + 26:...333322222111111111122223333.......... + 25:...3333222211111111111122223333......... + 24:....3332222111111..1111122223333........ + 23:....333322211111.....11112222333........ + 22:....33332222111.......11112223333....... + 21:.....33322221111.......11122223333...... + 20:.....33332221111.......11112223333...... + 19:.....33332222111.......11112222333...... + 18:......33332221111.......11122223333..... + 17:.......33322221111.....111112223333..... + 16:.......3333222211111..1111112222333..... + 15:........3333222211111111111122223333.... + 14:.........333322221111111111222223333.... + 13:..........33332222211111111222223333.... + 12:...........3333222222111122222223333.... + 11:............333322222222222222233333.... + 10:.............33333222222222222233333.... + 9:..............3333332222222222333333.... + 8:...............33333333222223333333..... + 7:.................333333333333333333..... + 6:..................3333333333333333...... + 5:.....................333333333333....... + 4:.......................33333333......... + 3:........................................ + 2:........................................ + 1:........................................ +</PRE> +Note in the above example that the lower limit is not part of the +region for boxes, circles, and ellipses. This makes circles and annuli +equivalent, i.e.: +<PRE> + circle 20 20 5 10 15 20 + annulus 20 20 5 10 15 20 +</PRE> +both give the following region mask: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........................................ + 39:.............3333333333333.............. + 38:...........33333333333333333............ + 37:.........333333333333333333333.......... + 36:........33333333333333333333333......... + 35:......333333333333333333333333333....... + 34:.....33333333322222222222333333333...... + 33:....3333333322222222222222233333333..... + 32:....3333333222222222222222223333333..... + 31:...333333222222222222222222222333333.... + 30:..33333322222222222222222222222333333... + 29:..33333322222221111111112222222333333... + 28:.3333332222222111111111112222222333333.. + 27:.3333322222211111111111111122222233333.. + 26:333333222222111111111111111222222333333. + 25:333332222221111111111111111122222233333. + 24:33333222221111111.....11111112222233333. + 23:3333322222111111.......1111112222233333. + 22:333332222211111.........111112222233333. + 21:333332222211111.........111112222233333. + 20:333332222211111.........111112222233333. + 19:333332222211111.........111112222233333. + 18:333332222211111.........111112222233333. + 17:3333322222111111.......1111112222233333. + 16:33333222221111111.....11111112222233333. + 15:333332222221111111111111111122222233333. + 14:333333222222111111111111111222222333333. + 13:.3333322222211111111111111122222233333.. + 12:.3333332222222111111111112222222333333.. + 11:..33333322222221111111112222222333333... + 10:..33333322222222222222222222222333333... + 9:...333333222222222222222222222333333.... + 8:....3333333222222222222222223333333..... + 7:....3333333322222222222222233333333..... + 6:.....33333333322222222222333333333...... + 5:......333333333333333333333333333....... + 4:........33333333333333333333333......... + 3:.........333333333333333333333.......... + 2:...........33333333333333333............ + 1:.............3333333333333.............. + +</PRE> + +<P> +As a final example, specifying several angles in one pie slice +expression is equivalent to specifying several separate slices with +the same center. As with the annulus, if <B>N</B> angles are +specified, then <B>N</B>-1 slices result, with the ending angle of +each preceding slice being the starting angle of the succeeding slice. +Each slice is considered a separate region, and is given a separate +mask value. For example, +<PRE> + PIE 12 12 315 45 115 270 +</PRE> +specifies three regions as shown below: +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:2222222222222222222222222222222222222222 + 39:2222222222222222222222222222222222222221 + 38:2222222222222222222222222222222222222211 + 37:2222222222222222222222222222222222222111 + 36:2222222222222222222222222222222222221111 + 35:3222222222222222222222222222222222211111 + 34:3222222222222222222222222222222222111111 + 33:3322222222222222222222222222222221111111 + 32:3322222222222222222222222222222211111111 + 31:3332222222222222222222222222222111111111 + 30:3332222222222222222222222222221111111111 + 29:3333222222222222222222222222211111111111 + 28:3333222222222222222222222222111111111111 + 27:3333322222222222222222222221111111111111 + 26:3333322222222222222222222211111111111111 + 25:3333322222222222222222222111111111111111 + 24:3333332222222222222222221111111111111111 + 23:3333332222222222222222211111111111111111 + 22:3333333222222222222222111111111111111111 + 21:3333333222222222222221111111111111111111 + 20:3333333322222222222211111111111111111111 + 19:3333333322222222222111111111111111111111 + 18:3333333332222222221111111111111111111111 + 17:3333333332222222211111111111111111111111 + 16:3333333333222222111111111111111111111111 + 15:3333333333222221111111111111111111111111 + 14:3333333333322211111111111111111111111111 + 13:3333333333322111111111111111111111111111 + 12:33333333333.1111111111111111111111111111 + 11:3333333333331111111111111111111111111111 + 10:333333333333.111111111111111111111111111 + 9:333333333333..11111111111111111111111111 + 8:333333333333...1111111111111111111111111 + 7:333333333333....111111111111111111111111 + 6:333333333333.....11111111111111111111111 + 5:333333333333......1111111111111111111111 + 4:333333333333.......111111111111111111111 + 3:333333333333........11111111111111111111 + 2:333333333333.........1111111111111111111 + 1:333333333333..........111111111111111111 +</PRE> + +<P> +The annulus, box, circle, ellipse, and pie shapes also accept an +<B>n=[int]</B> syntax for specifying multiple regions. The +<B>n=[int]</b>syntax interprets the previous (shape-dependent) +arguments as lower and upper limits for the region and creates n +shapes with evenly spaced boundaries. For example, if <B>n=[int]</B> +is specified in an annulus, the two immediately preceding radii +(<B>rn</B> and <B>rm</B>) are divided into <B>int</B> annuli, such +that the inner radius of the first is <B>rn</B> and the outer radius +of the last is <B>rm</B>. For example, +<PRE> + ANNULUS 20 20 5 20 n=3 +</PRE> +is equivalent to: +<PRE> + ANNULUS 20 20 5 10 15 20 +</PRE> +If this syntax is used with an ellipse or box, then the two preceding +pairs of values are taken to be lower and upper limits for a set of +ellipses or boxes. A circle uses the two preceding arguments for upper +and lower radii. For pie, the two preceding angles are divided into n +wedges such that the starting angle of the first is the lower bound +and the ending angle of the last is the upper bound. In all cases, +the <B>n=[int]</B> syntax allows any single alphabetic character +before the "=", i.e, i=3, z=3, etc. are all equivalent. + +<P> +Also note that for boxes and ellipses, the optional angle argument is +always specified after the <B>n=[int]</B> syntax. For example: +<PRE> + ellipse 20 20 4 6 16 24 n=3 45 +<PRE> +specifies 3 elliptical regions at an angle of 45 degrees: + +<PRE> + 1234567890123456789012345678901234567890 + ---------------------------------------- + 40:........33333333........................ + 39:.....33333333333333..................... + 38:....33333333333333333................... + 37:...33333333333333333333................. + 36:..33333333333333333333333............... + 35:.3333333333222223333333333.............. + 34:3333333322222222222233333333............ + 33:33333332222222222222223333333........... + 32:333333222222222222222222333333.......... + 31:3333322222222222222222222333333......... + 30:33333222222222111122222222333333........ + 29:333332222222111111112222222333333....... + 28:3333222222211111111111222222333333...... + 27:3333222222111111111111112222233333...... + 26:33332222221111111111111112222233333..... + 25:33332222211111111.111111112222233333.... + 24:333322222111111......111111222223333.... + 23:333322222111111.......111112222233333... + 22:33333222221111.........11111222223333... + 21:333332222211111.........11112222233333.. + 20:.33332222211111.........11111222223333.. + 19:.33333222221111.........111112222233333. + 18:..33332222211111.........11112222233333. + 17:..333332222211111.......111111222233333. + 16:...333322222111111......111111222223333. + 15:...333332222211111111.111111112222233333 + 14:....333332222211111111111111122222233333 + 13:.....33333222221111111111111122222233333 + 12:.....33333322222211111111111222222233333 + 11:......3333332222222111111112222222333333 + 10:.......333333222222221111222222222333333 + 9:........33333322222222222222222222333333 + 8:.........333333222222222222222222333333. + 7:..........33333332222222222222223333333. + 6:...........3333333322222222222233333333. + 5:.............3333333333222223333333333.. + 4:..............33333333333333333333333... + 3:................33333333333333333333.... + 2:..................33333333333333333..... + 1:....................33333333333333...... +</PRE> + +<P> +Both the variable argument syntax and the <B>n=[int]</B> syntax must +occur alone in a region descriptor (aside from the optional angle for +boxes and ellipses). They cannot be combined. Thus, it is not valid +to precede or follow an <B>n=[int]</B> accelerator with more angles or +radii, as in this example: +<PRE> + # INVALID -- one too many angles before a=5 ... + # and no angles are allowed after a=5 + PIE 12 12 10 25 50 a=5 85 135 +</PRE> +Instead, use three separate specifications, such as: +<PRE> + PIE 12 12 10 25 + PIE 12 12 25 50 a=5 + PIE 12 12 85 135 +</PRE> +The original (IRAF) implementation of region filtering permitted this +looser syntax, but we found it caused more confusion than it was worth +and therefore removed it. + +<P> +NB: Accelerators may be combined with other shapes in a boolean +expression in any order. (This is a change starting with funtools +v1.1.1. Prior to this release, the accelerator shape had to be +specified last). The actual region mask id values returned depend on the +order in which the shapes are specified, although the total number of +pixels or rows that pass the filter will be consistent. For this +reason, use of accelerators in boolean expressions is discouraged in +programs such as funcnts, where region mask id values are used +to count events or image pixels. + +<P> +[All region masks displayed in this document were generated using the +<B>fundisp</B> routine and the undocumented "mask=all" argument (with +spaced removed using sed ): +<PRE> + fundisp "funtools/funtest/test40.fits[ANNULUS 25 25 5 10]" mask=all |\ + sed 's/ //g' +</PRE> +Note that you must supply an image of the appropriate size -- in this case, +a FITS image of dimension 40x40 is used.] + +<!-- =section reggeometry SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: March 12, 2007</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/regions.html b/funtools/doc/regions.html new file mode 100644 index 0000000..33a8cde --- /dev/null +++ b/funtools/doc/regions.html @@ -0,0 +1,563 @@ +<!-- =defdoc funregions funregions n --> +<HTML> +<HEAD> +<TITLE>Spatial Region Filtering</TITLE> +</HEAD> +<BODY> + +<!-- =section funregions NAME --> +<H2><A NAME="regions">Regions: Spatial Region Filtering</A></H2> + +<!-- =section funregions SYNOPSIS --> +<H2>Summary</H2> +<P> +This document contains a summary of the user interface for spatial +region filtering images and tables. + +<!-- =section funregions DESCRIPTION --> +<H2>Description</H2> +<P> +Spatial region filtering allows a program to select regions of an +image or rows of a table (e.g., X-ray events) to process using +simple geometric shapes and boolean combinations of shapes. When an +image is filtered, only pixels found within these shapes are +processed. When a table is filtered, only rows found within these +shapes are processed. + +<P> +Spatial region filtering for images and tables is accomplished by +means of <B>region specifications</B>. A region specification +consists of one or more <B>region expressions</B>, which are geometric +shapes,combined according to the rules of boolean algebra. Region +specifications also can contain comments and local/global processing +directives. + +<P> +Typically, region specifications are specified using bracket notation +appended to the filename of the data being processed: +<PRE> + foo.fits[circle(512,512,100)] +</PRE> +It is also possible to put region specification inside a file and +then pass the filename in bracket notation: +<PRE> + foo.fits[@my.reg] +</PRE> + +<P> +When region filters are passed in bracket notation in this manner, the +filtering is set up automatically when the file is opened and all +processing occurs through the filter. Programs also can use the filter +library API to open filters explicitly. + +<H2>Region Expressions</H2> + +More specifically, region specifications consist of one or more lines +containing: +<PRE> + # comment until end of line + global keyword=value keyword=value ... # set global value(s) + # include the following file in the region descriptor + @file + # use the FITS image as a mask (cannot be used with other regions) + @fitsimage + # each region expression contains shapes separated by operators + [region_expression1], [region_expression2], ... + [region_expression], [region_expression], ... +</PRE> + +<P> +A single region expression consists of: +<PRE> + # parens and commas are optional, as is the + sign + [+-]shape(num , num , ...) OP1 shape num num num OP2 shape ... + +e.g.: + + ([+-]shape(num , num , ...) && shape num num || shape(num, num) + # a comment can come after a region -- reserved for local properties + [+-]shape(num , num , ...) # local properties go here, e.g. color=red +</PRE> + +<P> +Thus, a region descriptor consists of one or more <B>region +expressions</B> or <B>regions</B>, separated by comas, new-lines, or +semi-colons. Each <B>region</B> consists of one or more <B>geometric +shapes</B> combined using standard boolean operation. Several types +of shapes are supported, including: + +<PRE> + shape: arguments: + ----- ---------------------------------------- + ANNULUS xcenter ycenter inner_radius outer_radius + BOX xcenter ycenter xwidth yheight (angle) + CIRCLE xcenter ycenter radius + ELLIPSE xcenter ycenter xwidth yheight (angle) + FIELD none + LINE x1 y1 x2 y2 + PIE xcenter ycenter angle1 angle2 + POINT x1 y1 + POLYGON x1 y1 x2 y2 ... xn yn +</PRE> + +<P> +In addition, the following regions accept <B>accelerator</B> syntax: + +<PRE> + shape arguments + ----- ------------------------------------------ + ANNULUS xcenter ycenter radius1 radius2 ... radiusn + ANNULUS xcenter ycenter inner_radius outer_radius n=[number] + BOX xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + BOX xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + CIRCLE xcenter ycenter r1 r2 ... rn # same as annulus + CIRCLE xcenter ycenter rinner router n=[number] # same as annulus + ELLIPSE xcenter ycenter xw1 yh1 xw2 yh2 ... xwn yhn (angle) + ELLIPSE xcenter ycenter xwlo yhlo xwhi yhhi n=[number] (angle) + PIE xcenter ycenter angle1 angle2 (angle3) (angle4) (angle5) ... + PIE xcenter ycenter angle1 angle2 (n=[number]) + POINT x1 y1 x2 y2 ... xn yn +</PRE> +Note that the circle accelerators are simply aliases for the annulus +accelerators. See <A HREF="./reggeometry.html">region geometry</A> +for more information about accelerators. + +<P> +Finally, the following are combinations of pie with different shapes +(called "panda" for "Pie AND Annulus") allow for easy specification of +radial sections: + +<PRE> + shape: arguments: + ----- --------- + PANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + CPANDA xcen ycen ang1 ang2 nang irad orad nrad # circular + BPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # box + EPANDA xcen ycen ang1 ang2 nang xwlo yhlo xwhi yhhi nrad (ang) # ellipse +</PRE> + +The panda and cpanda specify combinations of annulus and circle with pie, +respectively and give identical results. The bpanda combines box and pie, +while epanda combines ellipse and pie. +See <A HREF="./reggeometry.html">region geometry</A> +for more information about pandas. + +<P> +The following "shapes" are ignored by funtools (generated by ds9): +<PRE> + shape: arguments: + ----- --------- + PROJECTION x1 y1 x2 y2 width # NB: ignored by funtools + RULER x1 y1 x2 y2 # NB: ignored by funtools + TEXT x y # NB: ignored by funtools + GRID # NB: ignored by funtools + TILE # NB: ignored by funtools + COMPASS # NB: ignored by funtools +</PRE> + +<P> +All arguments to regions are real values; integer values are +automatically converted to real where necessary. All angles are in +degrees and run from the positive image x-axis to the positive image +y-axis. If a rotation angle is part of the associated WCS header, that +angle is added implicitly as well. + +<P> +Note that 3-letter abbreviations are supported for all shapes, so that +you can specify "circle" or "cir". + +<P> +<H2>Columns Used in Region Filtering</H2> +<P> +By default, the x,y values in a region expression refer to the two +"image binning" columns, i.e. the columns that would be used to +bin the data into an image. For images, these are just the 2 dimensions +of the image. For tables, these usually default to x and y but +can be changed as required. For example, in Funtools, new binning +columns are specified using a bincols=(col1,col2) statement within +the bracket string on the command line. +<P> +Alternate columns for region filtering can be specified by the syntax: +<PRE> + (col1,col2)=region(...) +</PRE> +e.g.: +<PRE> + (X,Y)=annulus(x,y,ri,ro) + (PHA,PI)=circle(x,y,r) + (DX,DY)=ellipse(x,y,a,b[,angle]) +</PRE> + +<P> +<H2>Region Algebra</H2> + +(See also <A HREF="./regalgebra.html">Region Algebra</A> for more complete +information.) + +<P> +Region shapes can be combined together using Boolean operators: +<PRE> + Symbol Operation Use + -------- --------- ----------------------------------- + ! not Exclude this shape from this region + & or && and Include only the overlap of these shapes + | or || inclusive or Include all of both shapes + ^ exclusive or Include both shapes except their overlap +</PRE> +Note that the !region syntax must be combined with another region in order +that we be able to assign a region id properly. That is, +<PRE> + !circle(512,512,10) +</PRE> +is not a legal region because there is no valid region id to work with. +To get the full field without a circle, combine the above with field(), +as in: +<PRE> + field() && !circle(512,512,10) +</PRE> + +<H2> Region Separators Also Are Operators</H2> + +<P> +As mentioned previously, multiple region expressions can be specified +in a region descriptor, separated by commas, new-lines, or +semi-colons. When such a separator is used, the boolean OR operator +is automatically generated in its place but, unlike explicit use of +the OR operator, the region ID is incremented (starting from 1). + +<P> +For example, the two shapes specified in this example are given the +same region value: +<PRE> + foo.fits[circle(512,512,10)||circle(400,400,20)] +</PRE> +On the other hand, the two shapes defined in the following example are +given different region values: +<PRE> + foo.fits[circle(512,512,10),circle(400,400,20)] +</PRE> + +<P> +Of course these two examples will both mask the same table rows or +pixels. However, in programs that distinguish region id's (such as +<A HREF="programs.html#funcnts">funcnts</A> ), they will act +differently. The explicit OR operator will result in one region +expression consisting of two shapes having the same region id and +funcnts will report a single region. The comma operator will cause +funcnts to report two region expressions, each with one shape, in +its output. + +<P> +In general, commas are used to separate region expressions entered +in bracket notation on the command line: +<PRE> + # regions are added to the filename in bracket notation + foo.fits[circle(512,512,100),circle(400,400,20)] +</PRE> +New-lines are used to separate region +expressions in a file: +<PRE> + # regions usually are separated by new-lines in a file + # use @filename to include this file on the command line + circle(512,512,100) + circle(400,400,20) +</PRE> +Semi-colons are provided for backward compatibility with the original +IRAF/PROS implementation and can be used in either case. + +<P> +If a pixel is covered by two different regions expressions, it is +given the mask value of the <B>first</B> region that contains that +pixel. That is, successive regions <B>do not</b> overwrite previous +regions in the mask, as was the case with the original PROS regions. +In this way, an individual pixel is covered by one and only one +region. This means that one must sometimes be careful about the order +in which regions are defined. If region N is fully contained within +region M, then N should be defined <B>before</B> M, or else it will be +"covered up" by the latter. + +<H2>Region Exclusion</H2> +<P> +Shapes also can be globally excluded from all the region specifiers in +a region descriptor by using a minus sign before a region: + +<PRE> + operator arguments: + -------- ----------- + - Globally exclude the region expression following '-' sign + from ALL regions specified in this file +</PRE> +The global exclude region can be used by itself; in such a case, field() is +implied. + +<P> +A global exclude differs from the local exclude (i.e. a shape prefixed +by the logical not "!" symbol) in that global excludes are logically +performed last, so that no region will contain pixels from a globally +excluded shape. A local exclude is used in a boolean expression with +an include shape, and only excludes pixels from that include shape. +Global excludes cannot be used in boolean expressions. + +<H2>Include Files</H2> + +<P> +The <B>@filename</B> directive specifies an include file +containing region expressions. This file is processed as part of +the overall region descriptor: +<PRE> + foo.fits[circle(512,512,10),@foo] +</PRE> +A filter include file simply includes text without changing the state +of the filter. It therefore can be used in expression. That is, if the +file foo1 contains "pi==1" and foo2 contains "pha==2" then +the following expressions are equivalent: +<pre> + "[@foo1&&@foo2]" is equivalent to "[pi==1&&pha==2]" + "[pha==1||@foo2]" is equivalent to "[pi==1||pha==2]" + "[@foo1,@foo2]" is equivalent to "[pi==1,pha==2]" +</pre> +Be careful that you specify evaluation order properly using +parenthesis, especially if the include file contains multiple +filter statements. For example, consider a file containing two +regions such as: +<pre> + circle 512 512 10 + circle 520 520 10 +</pre> +If you want to include only events (or pixels) that are in these regions +and have a pi value of 4, then the correct syntax is: +<pre> + pi==4&&(@foo) +</pre> +since this is equivalent to: +<pre> + pi==4 && (circle 512 512 10 || circle 520 520 10) +</pre> +If you leave out the parenthesis, you are filtering this statement: +<pre> + pi==4 && circle 512 512 10 || circle 520 520 10) +</pre> +which is equivalent to: +<pre> + (pi==4 && circle 512 512 10) || circle 520 520 10) +</pre> +The latter syntax only applies the pi test to the first region. + +<P> +For image-style filtering, the <B>@filename</B> can specify an 8-bit +or 16-bit FITS image. In this case, the pixel values in the mask image +are used as the region mask. The valid pixels in the mask must have +positive values. Zero values are excluded from the mask and negative +values are not allowed. Moreover, the region id value is taken as +the image pixel value and the total number of regions is taken to be +the highest pixel value. The dimensions of the image mask must be less +than or equal to the image dimensions of the data. The mask will be +replicated as needed to match the size of the image. (Thus, best +results are obtained when the data dimensions are an even multiple of +the mask dimensions.) + +<P> +An image mask can be used in any image filtering operation, regardless +of whether the data is of type image or table. For example, the +<A HREF="programs.html#funcnts">funcnts</A> ) +program performs image filtering on images or tables, and so +FITS image masks are valid input for either type of data in this +program.. An image mask cannot be used in a program such as +<A HREF="programs.html#fundisp">fundisp</A> ) +when the input data is a table, because fundisp displays +rows of a table and processes these rows using event-style filtering. + +<H2>Global and Local Properties of Regions</H2> + +<P> +The ds9 image display program describes a host of properties such as +color, font, fix/free state, etc. Such properties can be specified +globally (for all regions) or locally (for an individual region). +The <B>global</B> keyword specifies properties and qualifiers for all +regions, while local properties are specified in comments on the same +line as the region: +<PRE> + global color=red + circle(10,10,2) + circle(20,20,3) # color=blue + circle(30,30,4) +</PRE> +The first and third circles will be red, which the second circle will +be blue. Note that funtools currently ignores region properties, as +they are used in display only. + +<H2> Coordinate Systems</H2> + +For each region, it is important to specify the coordinate system +used to interpret the region, i.e., to set the context in which position and +size values are interpreted. For this purpose, the following keywords +are recognized: + +<PRE> + name description + ---- ------------------------------------------ + PHYSICAL pixel coords of original file using LTM/LTV + IMAGE pixel coords of current file + FK4, B1950 sky coordinate systems + FK5, J2000 sky coordinate systems + GALACTIC sky coordinate systems + ECLIPTIC sky coordinate systems + ICRS currently same as J2000 + LINEAR linear wcs as defined in file + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV +</PRE> + +<P> +<H2>Specifying Positions, Sizes, and Angles</H2> + +The arguments to region shapes can be floats or integers describing +positions and sizes. They can be specified as pure numbers or using +explicit formatting directives: + +<PRE> + position arguments description + ------------------ ------------------------------ + [num] context-dependent (see below) + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels + [num]:[num]:[num] hms for 'odd' position arguments + [num]:[num]:[num] dms for 'even' position arguments + [num]h[num]m[num]s explicit hms + [num]d[num]m[num]s explicit dms + + size arguments description + -------------- ----------- + [num] context-dependent (see below) + [num]" arc seconds + [num]' arc minutes + [num]d degrees + [num]r radians + [num]p physical pixels + [num]i image pixels +</PRE> + +<!- this helps emacs close the open quote " > +When a "pure number" (i.e. one without a format directive such as 'd' +for 'degrees') is specified, its interpretation depends on the context +defined by the 'coordsys' keyword. In general, the rule is: + +<P> +<B>All pure numbers have implied units corresponding to the current +coordinate system.</B> + +<P> +If no such system is explicitly specified, the default system is +implicitly assumed to be PHYSICAL. + +<P> +In practice this means that for IMAGE and PHYSICAL systems, pure +numbers are pixels. Otherwise, for all systems other than linear, +pure numbers are degrees. For LINEAR systems, pure numbers are in the +units of the linear system. This rule covers both positions and +sizes. + +<P> +The input values to each shape can be specified in several coordinate +systems including: + +<PRE> + name description + ---- ---------------------------- + IMAGE pixel coords of current file + LINEAR linear wcs as defined in file + FK4, B1950 various sky coordinate systems + FK5, J2000 + GALACTIC + ECLIPTIC + ICRS + PHYSICAL pixel coords of original file using LTM/LTV + AMPLIFIER mosaic coords of original file using ATM/ATV + DETECTOR mosaic coords of original file using DTM/DTV +</PRE> + +<P> +If no coordinate system is specified, PHYSICAL is assumed. PHYSICAL or +a World Coordinate System such as J2000 is preferred and most general. +The coordinate system specifier should appear at the beginning of the +region description, on a separate line (in a file), or followed by a +new-line or semicolon; e.g., + +<PRE> + global coordsys physical + circle 6500 9320 200 +</PRE> + +The use of celestial input units automatically implies WORLD +coordinates of the reference image. Thus, if the world coordinate +system of the reference image is J2000, then + +<PRE> + circle 10:10:0 20:22:0 3' +</PRE> + +is equivalent to: + +<PRE> + circle 10:10:0 20:22:0 3' # j2000 +</PRE> + +</PRE> +Note that by using units as described above, you may mix coordinate +systems within a region specifier; e.g., + +<PRE> + circle 6500 9320 3' # physical +</PRE> + +<P> +Note that, for regions which accept a rotation angle: + +<PRE> +ellipse (x, y, r1, r2, angle) +box(x, y, w, h, angle) +</PRE> + +the angle is relative to the specified coordinate system. In +particular, if the region is specified in WCS coordinates, the angle +is related to the WCS system, not x/y image coordinate axis. For WCS +systems with no rotation, this obviously is not an issue. However, +some images do define an implicit rotation (e.g., by using a non-zero +CROTA value in the WCS parameters) and for these images, the angle +will be relative to the WCS axes. In such case, a region specification +such as: + +<PRE> +fk4;ellipse(22:59:43.985, +58:45:26.92,320", 160", 30) +</PRE> + +will not, in general, be the same region specified as: + +<PRE> +physical;ellipse(465, 578, 40, 20, 30) +</PRE> + +even when positions and sizes match. The angle is relative to WCS axes +in the first case, and relative to physical x,y axes in the second. + + +<P> +More detailed descriptions are available for: +<A HREF="./reggeometry.html">Region Geometry</A>, +<A HREF="./regalgebra.html">Region Algebra</A>, +<A HREF="./regcoords.html">Region Coordinates</A>, and +<A HREF="./regbounds.html">Region Boundaries</A>. + +<!-- =section funregions SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: November 17, 2005</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/sman/fun4.index b/funtools/doc/sman/fun4.index Binary files differnew file mode 100644 index 0000000..ce00526 --- /dev/null +++ b/funtools/doc/sman/fun4.index diff --git a/funtools/doc/sman/fun4.index.prop b/funtools/doc/sman/fun4.index.prop Binary files differnew file mode 100644 index 0000000..9b01bbd --- /dev/null +++ b/funtools/doc/sman/fun4.index.prop diff --git a/funtools/doc/sman/fun4.index.version b/funtools/doc/sman/fun4.index.version new file mode 100644 index 0000000..accb620 --- /dev/null +++ b/funtools/doc/sman/fun4.index.version @@ -0,0 +1,2 @@ +SMAN_DATA_VERSION 1.2 +VERSION 1.01 diff --git a/funtools/doc/sman/fun8.index b/funtools/doc/sman/fun8.index Binary files differnew file mode 100644 index 0000000..44fd94a --- /dev/null +++ b/funtools/doc/sman/fun8.index diff --git a/funtools/doc/sman/fun8.index.prop b/funtools/doc/sman/fun8.index.prop Binary files differnew file mode 100644 index 0000000..5fd8f14 --- /dev/null +++ b/funtools/doc/sman/fun8.index.prop diff --git a/funtools/doc/sman/fun8.index.version b/funtools/doc/sman/fun8.index.version new file mode 100644 index 0000000..accb620 --- /dev/null +++ b/funtools/doc/sman/fun8.index.version @@ -0,0 +1,2 @@ +SMAN_DATA_VERSION 1.2 +VERSION 1.01 diff --git a/funtools/doc/sman_conf.tmpl b/funtools/doc/sman_conf.tmpl new file mode 100644 index 0000000..abac8e4 --- /dev/null +++ b/funtools/doc/sman_conf.tmpl @@ -0,0 +1,96 @@ +# $Id: sman-defaults.conf,v 1.33 2006/05/02 02:09:18 joshr Exp $ +# by Josh Rabinowitz, 2005-2006. <joshr> +# this is the default sman configuration file. +# if you need to make changes, copy this file to +# /usr/local/etc/sman.conf and make changes to that file. +# NOTE: if you do use a custom sman.conf you may need to manually +# NOTE: merge changes from this file to your configuration file +# NOTE: after upgrading sman versions + +# See 'perldoc sman.conf' for more documentation + +# sman-defaults.conf +# holds the defaults for sman. +# the directive names are case-insensitive + +SWISHECMD @SW@ -v 0 + +# MANCMD specifies how to manually convert the manpages to +# ASCII. For 'manual' parsing. +# %F is the quoted full [F]ilename of the manpage (ie, '/usr/share/man/man1/ls.1.gz) +# %S is the quoted [S]ection of the manpage (ie, '1') +# %C is the quoted (apparent) [C]ommand of the manpage (ie, 'ls') + +# NOTE: lack of a MANCMD (or a value of AUTOCONFIG) causes sman-update +# to autoconfigure the value of MANCMD. It will most likely choose +# one of the below: + +# this works for most linuxes we've tested. ie 'man /man/man1/ls.1' +# MANCMD man -c %F +# this works for freebsd 4.4 and Mac OS X. ie 'man 1 ls' +#MANCMD man -c %S %C +# -c means reparse man page sources (for line re-wrapping) + +# works for Solaris and OS X +MANCMD man -s %S %C + +# the COL program is used to strip out backspaces and +# such from the MAN output. + +COLCMD col -b + +# all vars starting with ENV_ get set in the +# environment, sans the ENV_ prefix. + +# Try to make MAN wrap lines at 256 chars (not 80!) +ENV_MANWIDTH 256 + +# where to put sman's temp files. +# (Use SWISHE_TMPDIR to set affect SWISH-E at index time) + +TMPDIR /tmp + +# meta and property aliases. If your XML has different tags. +# refentrytitle, manvolnum and refpurpose are the names from rman +# each ALIASES line turns into 2 lines for SWISH-E like this: +#MetaNameAlias swishtitle refentrytitle +#PropertyNameAlias swishtitle refentrytitle + +TITLEALIASES RefEntryTitle +SECALIASES ManVolNum +DESCALIASES RefPurpose +#MANPAGEALIASES swishdefault + +# All parameters beginning with SWISHE_ have the SWISHE_ prefix stripped +# and are written into a tmp config file for SWISH-E at index time. + +# SWISHE_IndexFile is also used by sman as which index to search. + +# SWISHE_IndexFile specifies which index to create and search +# NOTE: if the containing dir (ie, /var/lib/sman) is owned by an unprivileged +# user, then sman-update can be run as that user. + +SWISHE_IndexFile /var/lib/sman/sman.index + +# these have 'SWISHE_' stripped off and are used +# when indexing the man pages. + +SWISHE_IndexComments no + # don't index text in comments + +# These are important! You probably don't want to mess with these. +SWISHE_IndexPointer "format=%V;" + # for future use. %V becomes $Sman::SMAN_DATA_VERSION +#SWISHE_FuzzyIndexingMode Stem + # above was deprecated in 2.4.3, does not work in 2.4.4 +SWISHE_FuzzyIndexingMode Stemming_en2 +SWISHE_MetaNames desc sec swishtitle manpage digest +SWISHE_PropertyNames desc sec manpage digest + +# SWISHE_IgnoreWords File: ./stopwords/english.txt +# allow _ and : but not / +# .'s in middle of word are word chars, for files like 'named.conf'. +SWISHE_WordCharacters 0123456789abcdefghijklmnopqrstuvwxyz_:. +SWISHE_BeginCharacters 0123456789abcdefghijklmnopqrstuvwxyz_: +SWISHE_EndCharacters 0123456789abcdefghijklmnopqrstuvwxyz_: +SWISHE_IgnoreTotalWordCountWhenRanking 0 diff --git a/funtools/doc/szlong.c b/funtools/doc/szlong.c new file mode 100644 index 0000000..b0e84bc --- /dev/null +++ b/funtools/doc/szlong.c @@ -0,0 +1,8 @@ +#include <stdio.h> + +int main(int argc, char **argv) +{ + fprintf(stdout, "%d\n", (int)sizeof(long)); + return 0; +} + diff --git a/funtools/doc/tabcalc.c b/funtools/doc/tabcalc.c new file mode 100644 index 0000000..5f1cb30 --- /dev/null +++ b/funtools/doc/tabcalc.c @@ -0,0 +1,196 @@ +/* + * + * This program was generated automatically by the funcalc program, + * by running the tabcalc.c template through the funcalc.l lexical analyzer. + * On this system, it was (or can be) built a command such as: + * + * $COMMAND + * + */ +#include <funtools.h> + +extern char *optarg; +extern int optind; + +/* define the types of event records we have to handle */ +#define REC_CUR 1 +#define REC_PREV 2 +#define REC_NEXT 4 + +/* default number of rows to read at a time */ +#define MAXROW 8192 +int maxrow=MAXROW; + +typedef struct rowstruct{ +$MEMBERS +} *Row, RowRec; + +/* global definitions and init calls go here */ +$GLOBAL + +/* main program */ +int main (int argc, char **argv) +{ + int c, i, got, total, rectype, start, stop, skip, args; + int del=0; + char *s; + Fun fun, ofun; + Row rowbuf, rowptr; + Row cur, prev, next; + $AUTO + + /* local definitions, followed by init calls go here */ + $LOCAL + + /* exit on gio errors */ + setgerror(1); + + /* avoid shared library problems by using "process" type for filtering */ + putenv("FILTER_PTYPE=process"); + + /* process switch arguments */ + while ((c = getopt(argc, argv, "d")) != -1){ + switch(c){ + case 'd': + del = 1; + break; + } + } + /* check for required arguments */ + args = argc - optind; + /* make sure we have minimal arguments */ + if( args < $ARGS ){ + if( $ARGS == 1 ) + fprintf(stderr, + "usage: [expr] | funcalc [-e expr] [-f file] iname\n"); + else + fprintf(stderr, + "usage: [expr] | funcalc [-e expr] [-f file] iname oname [cols]\n"); + goto error; + } + + /* set rectype: determine whether we need prev,next records */ + rectype=$RECTYPE; + + /* get maxrow,if user-specified */ + if( (s=(char *)getenv("FUN_MAXROW")) != NULL ) + maxrow = atoi(s); + /* make sure max row is large enough to handle prev, next */ + if( rectype & (REC_PREV|REC_NEXT) ) maxrow = MAX(3,maxrow); + + /* open input file */ + if( !(fun = FunOpen(argv[optind+0], "rc", NULL)) ){ + gerror(stderr, "could not FunOpen input file: %s\n", argv[optind+0]); + goto error; + } + + /* open the output FITS image, inheriting params from input */ + if( $ARGS > 1 ){ + if( !(ofun = FunOpen(argv[optind+1], "w", fun)) ){ + gerror(stderr, "could not FunOpen output file: %s\n", argv[optind+1]); + goto error; + } + } + + /* select columns */ + FunColumnSelect(fun, sizeof(RowRec), "merge=replace", + $SELECT + NULL); + + /* activate specified columns -- these will be written to the output file */ + if( args >= 3 ) + FunColumnActivate(fun, argv[optind+2], NULL); + + /* allocate space for rowbuf -- we will manage this buffer ourselves */ + rowbuf = (Row)calloc(maxrow, sizeof(RowRec)); + + /* no record read yet */ + total = 0; + + /* any user-defined calls before we enter the row loop go here */ + $BEFORE + + /* main loop -- get rows and process */ + while( 1 ){ + /* need prev record */ + if( rectype & REC_PREV ){ + /* no records yet: read new batch of records into start of rowbuf */ + if( total == 0 ){ + rowptr = rowbuf; + } + /* we have read records: move last record from last batch into first + new record and read new records after that */ + else{ + /* if we are accessing next, we did not actually process final record, + therefore we do it now */ + if( rectype & REC_NEXT ){ + memcpy(rowbuf, rowbuf+(got-2), sizeof(RowRec)*2); + rowptr = rowbuf+2; + } + /* not accessing next, so we just move last record processed */ + else{ + memcpy(rowbuf, rowbuf+(got-1), sizeof(RowRec)); + rowptr = rowbuf+1; + } + } + /* start at 2nd record so as to have a valid prev */ + start = 1; + } + /* processing that does not require prev */ + else{ + rowptr = rowbuf; + start = 0; + } + + /* adjust maxrow to account for handling of prev record */ + skip = (rowptr - rowbuf)/sizeof(RowRec); + + /* read new rows */ + if( !FunTableRowGet(fun, rowptr, maxrow-skip, NULL, &got) ) + break; + + /* if we need access to next record, don't process the last one we read */ + if( rectype & REC_NEXT ){ + stop = got - 1; + } + /* processing that does not require acces to next, process as cur */ + else{ + stop = got; + } + + /* process all rows */ + for(i=start; i<stop; i++){ + /* set up pointer to current row */ + cur = rowbuf+i; + /* set up pointer to prev and next as needed */ + if( rectype & REC_PREV ) prev = rowbuf+(i-1); + if( rectype & REC_NEXT ) next = rowbuf+(i+1); + /* execute the expression */ + $EXPR + /* write out this row with the new column */ + if( $ARGS > 1 ) + FunTableRowPut(ofun, (char *)cur, 1, i, NULL); + } + total += got; + } + + /* any user-defined calls after we finish the row loop go here */ + $AFTER + + /* free row data */ + if( rowbuf ) free(rowbuf); + + /* clean up -- close output before input to perform flush automatically */ + if( $ARGS > 1 ) + FunClose(ofun); + FunClose(fun); + + /* delete program if necessary */ + if( del ) unlink(argv[0]); + return(0); + +error: + /* delete program if necessary */ + if( del ) unlink(argv[0]); + return(1); +} diff --git a/funtools/doc/text.html b/funtools/doc/text.html new file mode 100644 index 0000000..f4fdac8 --- /dev/null +++ b/funtools/doc/text.html @@ -0,0 +1,564 @@ +<!-- =defdoc funtext funtext n --> +<HTML> +<HEAD> +<TITLE>Column-based Text Files</TITLE> +</HEAD> +<BODY> + +<!-- =section funtext NAME --> +<H2><A NAME="funtext">Funtext: Support for Column-based Text Files</A></H2> + +<!-- =section funtext SYNOPSIS --> +<H2>Summary</H2> +<P> +This document contains a summary of the options for processing column-based +text files. + +<!-- =section funtext DESCRIPTION --> +<H2>Description</H2> + +<P> +Funtools will automatically sense and process "standard" +column-based text files as if they were FITS binary tables without any +change in Funtools syntax. In particular, you can filter text files +using the same syntax as FITS binary tables: +<pre> + fundisp foo.txt'[cir 512 512 .1]' + fundisp -T foo.txt > foo.rdb + funtable foo.txt'[pha=1:10,cir 512 512 10]' foo.fits +</pre> + +<P> +The first example displays a filtered selection of a text file. The +second example converts a text file to an RDB file. The third example +converts a filtered selection of a text file to a FITS binary table. + +<P> +Text files can also be used in Funtools image programs. In this case, +you must provide binning parameters (as with raw event files), using +the bincols keyword specifier: + +<pre> + bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]] +</pre> + +For example: +<pre> + funcnts foo'[bincols=(x:1024,y:1024)]' "ann 512 512 0 10 n=10" +</pre> + +<H2>Standard Text Files</H2> + +<P> +Standard text files have the following characteristics: + +<ul> +<li> Optional comment lines start with # +<li> Optional blank lines are considered comments +<li> An optional table header consists of the following (in order): +<ul> + <li> a single line of alpha-numeric column names + <li> an optional line of unit strings containing the same number of cols + <li> an optional line of dashes containing the same number of cols +</ul> +<li> Data lines follow the optional header and (for the present) consist of + the same number of columns as the header. +<li> Standard delimiters such as space, tab, comma, semi-colon, and bar. +</ul> + +<P> +Examples: + +<pre> + # rdb file + foo1 foo2 foo3 foos + ---- ---- ---- ---- + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + # multiple consecutive whitespace and dashes + foo1 foo2 foo3 foos + --- ---- ---- ---- + 1 2.2 3 xxxx + 10 20.2 30 yyyy + + # comma delims and blank lines + foo1,foo2,foo3,foos + + 1,2.2,3,xxxx + 10,20.2,30,yyyy + + # bar delims with null values + foo1|foo2|foo3|foos + 1||3|xxxx + 10|20.2||yyyy + + # header-less data + 1 2.2 3 xxxx + 10 20.2 30 yyyy +</pre> + +<P> +The default set of token delimiters consists of spaces, tabs, commas, +semi-colons, and vertical bars. Several parsers are used +simultaneously to analyze a line of text in different ways. One way +of analyzing a line is to allow a combination of spaces, tabs, and +commas to be squashed into a single delimiter (no null values between +consecutive delimiters). Another way is to allow tab, semi-colon, and +vertical bar delimiters to support null values, i.e. two consecutive +delimiters implies a null value (e.g. RDB file). A successful parser +is one which returns a consistent number of columns for all rows, with +each column having a consistent data type. More than one parser can +be successful. For now, it is assumed that successful parsers all +return the same tokens for a given line. (Theoretically, there are +pathological cases, which will be taken care of as needed). Bad parsers +are discarded on the fly. + +<P> +If the header does not exist, then names "col1", "col2", etc. are +assigned to the columns to allow filtering. Furthermore, data types +for each column are determined by the data types found in the columns +of the first data line, and can be one of the following: string, int, +and double. Thus, all of the above examples return the following +display: +<pre> + fundisp foo'[foo1>5]' + FOO1 FOO2 FOO3 FOOS + ---------- --------------------- ---------- ------------ + 10 20.20000000 30 yyyy +</pre> + +<H2>Comments Convert to Header Params</H2> + +<P> +Comments which precede data rows are converted into header parameters and +will be written out as such using funimage or funhead. Two styles of comments +are recognized: + +<P> +1. FITS-style comments have an equal sign "=" between the keyword and +value and an optional slash "/" to signify a comment. The strict FITS +rules on column positions are not enforced. In addition, strings only +need to be quoted if they contain whitespace. For example, the following +are valid FITS-style comments: + +<pre> + # fits0 = 100 + # fits1 = /usr/local/bin + # fits2 = "/usr/local/bin /opt/local/bin" + # fits3c = /usr/local/bin /opt/local/bin /usr/bin + # fits4c = "/usr/local/bin /opt/local/bin" / path dir +</pre> + +Note that the fits3c comment is not quoted and therefore its value is the +single token "/usr/local/bin" and the comment is "opt/local/bin /usr/bin". +This is different from the quoted comment in fits4c. + +<P> +2. Free-form comments can have an optional colon separator between the +keyword and value. In the absence of quote, all tokens after the +keyword are part of the value, i.e. no comment is allowed. If a string +is quoted, then slash "/" after the string will signify a comment. +For example: + +<pre> + # com1 /usr/local/bin + # com2 "/usr/local/bin /opt/local/bin" + # com3 /usr/local/bin /opt/local/bin /usr/bin + # com4c "/usr/local/bin /opt/local/bin" / path dir + + # com11: /usr/local/bin + # com12: "/usr/local/bin /opt/local/bin" + # com13: /usr/local/bin /opt/local/bin /usr/bin + # com14c: "/usr/local/bin /opt/local/bin" / path dir +</pre> + +<P> +Note that com3 and com13 are not quoted, so the whole string is part of +the value, while comz4c and com14c are quoted and have comments following +the values. + +<P> +Some text files have column name and data type information in the header. +You can specify the format of column information contained in the +header using the "hcolfmt=" specification. See below for a detailed +description. + +<H2>Multiple Tables in a Single File</H2> + +<P> +Multiple tables are supported in a single file. If an RDB-style file +is sensed, then a ^L (vertical tab) will signify end of +table. Otherwise, an end of table is sensed when a new header (i.e., +all alphanumeric columns) is found. (Note that this heuristic does not +work for single column tables where the column type is ASCII and the +table that follows also has only one column.) You also can specify +characters that signal an end of table condition using the <b>eot=</b> +keyword. See below for details. + +<P> +You can access the nth table (starting from 1) in a multi-table file +by enclosing the table number in brackets, as with a FITS extension: + +<pre> + fundisp foo'[2]' +</pre> +The above example will display the second table in the file. +(Index values start at 1 in oder to maintain logical compatibility +with FITS files, where extension numbers also start at 1). + + +<H2>TEXT() Specifier</H2> + +<P> +As with ARRAY() and EVENTS() specifiers for raw image arrays and raw +event lists respectively, you can use TEXT() on text files to pass +key=value options to the parsers. An empty set of keywords is +equivalent to not having TEXT() at all, that is: + +<pre> + fundisp foo + fundisp foo'[TEXT()]' +</pre> + +are equivalent. A multi-table index number is placed before the TEXT() +specifier as the first token, when indexing into a multi-table: + + fundisp foo'[2,TEXT(...)]' + +<P> +The filter specification is placed after the TEXT() specifier, separated +by a comma, or in an entirely separate bracket: + +<pre> + fundisp foo'[TEXT(...),circle 512 512 .1]' + fundisp foo'[2,TEXT(...)][circle 512 512 .1]' +</pre> + +<H2>Text() Keyword Options</H2> + +<P> +The following is a list of keywords that can be used within the TEXT() +specifier (the first three are the most important): + +<DL> + +<P> +<DT> delims="[delims]" +<DD>Specify token delimiters for this file. Only a single parser having these +delimiters will be used to process the file. +<pre> + fundisp foo.fits'[TEXT(delims="!")]' + fundisp foo.fits'[TEXT(delims="\t%")]' +</pre> + +<P> +<DT> comchars="[comchars]" +<DD> Specify comment characters. You must include "\n" to allow blank lines. +These comment characters will be used for all standard parsers (unless delims +are also specified). +<pre> + fundisp foo.fits'[TEXT(comchars="!\n")]' +</pre> + +<P> +<DT> cols="[name1:type1 ...]" +<DD> Specify names and data type of columns. This overrides header +names and/or data types in the first data row or default names and +data types for header-less tables. +<pre> + fundisp foo.fits'[TEXT(cols="x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e")]' +</pre> +<P> +If the column specifier is the only keyword, then the cols= is not +required (in analogy with EVENTS()): +<pre> + fundisp foo.fits'[TEXT(x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e)]' +</pre> +Of course, an index is allowed in this case: +<pre> + fundisp foo.fits'[2,TEXT(x:I,y:I,pha:I,pi:I,time:D,dx:E,dy:e)]' +</pre> + +<P> +<DT> eot="[eot delim]" +<DD> Specify end of table string specifier for multi-table files. RDB +files support ^L. The end of table specifier is a string and the whole +string must be found alone on a line to signify EOT. For example: +<pre> + fundisp foo.fits'[TEXT(eot="END")]' +</pre> +will end the table when a line contains "END" is found. Multiple lines +are supported, so that: +<pre> + fundisp foo.fits'[TEXT(eot="END\nGAME")]' +</pre> +will end the table when a line contains "END" followed by a line +containing "GAME". +<P> +In the absence of an EOT delimiter, a new table will be sensed when a new +header (all alphanumeric columns) is found. + +<P> +<DT> null1="[datatype]" +<DD> Specify data type of a single null value in row 1. +Since column data types are determined by the first row, a null value +in that row will result in an error and a request to specify names and +data types using cols=. If you only have a one null in row 1, you don't +need to specify all names and columns. Instead, use null1="type" to +specify its data type. + +<P> +<DT> alen=[n] +<DD>Specify size in bytes for ASCII type columns. +FITS binary tables only support fixed length ASCII columns, so a +size value must be specified. The default is 16 bytes. + +<P> +<DT> nullvalues=["true"|"false"] +<DD>Specify whether to expect null values. +Give the parsers a hint as to whether null values should be allowed. The +default is to try to determine this from the data. + +<P> +<DT> whitespace=["true"|"false"] +<DD> Specify whether surrounding white space should be kept as part of +string tokens. By default surrounding white space is removed from +tokens. + +<P> +<DT> header=["true"|"false"] +<DD>Specify whether to require a header. This is needed by tables +containing all string columns (and with no row containing dashes), in +order to be able to tell whether the first row is a header or part of +the data. The default is false, meaning that the first row will be +data. If a row dashes are present, the previous row is considered the +column name row. + +<P> +<DT> units=["true"|"false"] +<DD>Specify whether to require a units line. +Give the parsers a hint as to whether a row specifying units should be +allowed. The default is to try to determine this from the data. + +<P> +<DT> i2f=["true"|"false"] +<DD>Specify whether to allow int to float conversions. +If a column in row 1 contains an integer value, the data type for that +column will be set to int. If a subsequent row contains a float in +that same column, an error will be signaled. This flag specifies that, +instead of an error, the float should be silently truncated to +int. Usually, you will want an error to be signaled, so that you can +specify the data type using cols= (or by changing the value of +the column in row 1). + +<P> +<DT> comeot=["true"|"false"|0|1|2] +<DD>Specify whether comment signifies end of table. +If comeot is 0 or false, then comments do not signify end of table and +can be interspersed with data rows. If the value is true or 1 (the +default for standard parsers), then non-blank lines (e.g. lines +beginning with '#') signify end of table but blanks are allowed +between rows. If the value is 2, then all comments, including blank +lines, signify end of table. + +<P> +<DT> lazyeot=["true"|"false"] +<DD>Specify whether "lazy" end of table should be permitted (default is +true for standard formats, except rdb format where explicit ^L is required +between tables). A lazy EOT can occur when a new table starts directly +after an old one, with no special EOT delimiter. A check for this EOT +condition is begun when a given row contains all string tokens. If, in +addition, there is a mismatch between the number of tokens in the +previous row and this row, or a mismatch between the number of string +tokens in the prev row and this row, a new table is assumed to have +been started. For example: +<PRE> + ival1 sval3 + ----- ----- + 1 two + 3 four + + jval1 jval2 tval3 + ----- ----- ------ + 10 20 thirty + 40 50 sixty +</PRE> +Here the line "jval1 ..." contains all string tokens. In addition, +the number of tokens in this line (3) differs from the number of +tokens in the previous line (2). Therefore a new table is assumed +to have started. Similarly: +<PRE> + ival1 ival2 sval3 + ----- ----- ----- + 1 2 three + 4 5 six + + jval1 jval2 tval3 + ----- ----- ------ + 10 20 thirty + 40 50 sixty +</PRE> +Again, the line "jval1 ..." contains all string tokens. The number of +string tokens in the previous row (1) differs from the number of +tokens in the current row(3). We therefore assume a new table as been +started. This lazy EOT test is not performed if lazyeot is explicitly +set to false. + +<P> +<DT> hcolfmt=[header column format] +<DD> Some text files have column name and data type information in the header. +For example, VizieR catalogs have headers containing both column names +and data types: +<PRE> + #Column e_Kmag (F6.3) ?(k_msigcom) K total magnitude uncertainty (4) [ucd=ERROR] + #Column Rflg (A3) (rd_flg) Source of JHK default mag (6) [ucd=REFER_CODE] + #Column Xflg (I1) [0,2] (gal_contam) Extended source contamination (10) [ucd=CODE_MISC] +</PRE> + +while Sextractor files have headers containing column names alone: + +<PRE> + # 1 X_IMAGE Object position along x [pixel] + # 2 Y_IMAGE Object position along y [pixel] + # 3 ALPHA_J2000 Right ascension of barycenter (J2000) [deg] + # 4 DELTA_J2000 Declination of barycenter (J2000) [deg] +</PRE> +The hcolfmt specification allows you to describe which header lines +contain column name and data type information. It consists of a string +defining the format of the column line, using "$col" (or "$name") to +specify placement of the column name, "$fmt" to specify placement of the +data format, and "$skip" to specify tokens to ignore. You also can +specify tokens explicitly (or, for those users familiar with how +sscanf works, you can specify scanf skip specifiers using "%*"). +For example, the VizieR hcolfmt above might be specified in several ways: +<PRE> + Column $col ($fmt) # explicit specification of "Column" string + $skip $col ($fmt) # skip one token + %*s $col ($fmt) # skip one string (using scanf format) +</PRE> +while the Sextractor format might be specified using: +<PRE> + $skip $col # skip one token + %*d $col # skip one int (using scanf format) +</PRE> +You must ensure that the hcolfmt statement only senses actual column +definitions, with no false positives or negatives. For example, the +first Sextractor specification, "$skip $col", will consider any header +line containing two tokens to be a column name specifier, while the +second one, "%*d $col", requires an integer to be the first token. In +general, it is preferable to specify formats as explicitly as +possible. + +<P> +Note that the VizieR-style header info is sensed automatically by the +funtools standard VizieR-like parser, using the hcolfmt "Column $col +($fmt)". There is no need for explicit use of hcolfmt in this case. + +<P> +<DT> debug=["true"|"false"] +<DD>Display debugging information during parsing. + +</DL> + +<H2>Environment Variables</H2> + +<P> +Environment variables are defined to allow many of these TEXT() values to be +set without having to include them in TEXT() every time a file is processed: + +<pre> + keyword environment variable + ------- -------------------- + delims TEXT_DELIMS + comchars TEXT_COMCHARS + cols TEXT_COLUMNS + eot TEXT_EOT + null1 TEXT_NULL1 + alen TEXT_ALEN + bincols TEXT_BINCOLS + hcolfmt TEXT_HCOLFMT +</pre> + +<H2>Restrictions and Problems</H2> + +<P> +As with raw event files, the '+' (copy extensions) specifier is not +supported for programs such as funtable. + +<P> +String to int and int to string data conversions are allowed by the +text parsers. This is done more by force of circumstance than by +conviction: these transitions often happens with VizieR catalogs, +which we want to support fully. One consequence of allowing these +transitions is that the text parsers can get confused by columns which +contain a valid integer in the first row and then switch to a +string. Consider the following table: +<PRE> + xxx yyy zzz + ---- ---- ---- + 111 aaa bbb + ccc 222 ddd +</PRE> +The xxx column has an integer value in row one a string in row two, +while the yyy column has the reverse. The parser will erroneously +treat the first column as having data type int: +<PRE> + fundisp foo.tab + XXX YYY ZZZ + ---------- ------------ ------------ + 111 'aaa' 'bbb' + 1667457792 '222' 'ddd' +</PRE> +while the second column is processed correctly. This situation can be avoided +in any number of ways, all of which force the data type of the first column +to be a string. For example, you can edit the file and explicitly quote the +first row of the column: +<PRE> + xxx yyy zzz + ---- ---- ---- + "111" aaa bbb + ccc 222 ddd + + [sh] fundisp foo.tab + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd' +</PRE> +You can edit the file and explicitly set the data type of the first column: +<PRE> + xxx:3A yyy zzz + ------ ---- ---- + 111 aaa bbb + ccc 222 ddd + + [sh] fundisp foo.tab + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd' +</PRE> +You also can explicitly set the column names and data types of all columns, +without editing the file: +<PRE> + [sh] fundisp foo.tab'[TEXT(xxx:3A,yyy:3A,zzz:3a)]' + XXX YYY ZZZ + ------------ ------------ ------------ + '111' 'aaa' 'bbb' + 'ccc' '222' 'ddd' +</PRE> +The issue of data type transitions (which to allow and which to disallow) +is still under discussion. + +<!-- =section funtext SEE ALSO --> +<!-- =text See funtools(n) for a list of Funtools help pages --> +<!-- =stop --> + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: August 3, 2007</H5> + +</BODY> +</HTML> diff --git a/funtools/doc/twcs.c b/funtools/doc/twcs.c new file mode 100644 index 0000000..2da9319 --- /dev/null +++ b/funtools/doc/twcs.c @@ -0,0 +1,61 @@ +/* + * + * twcs.c -- example using WCS Library + * + */ + +#include <funtools.h> + +int main(int argc, char **argv) +{ + int i; + Fun fun; + struct WorldCoor *wcs; /* WCS info */ + double x,y,ra,dec,xr,yr; + + if(argc == 1){ + fprintf(stderr, "usage: twcs iname\n"); + exit(1); + } + + /* open Funtools file */ + /* Funopen makes initial WCS library call: wcs = wcsinit(header_string) */ + if( !(fun = FunOpen(argv[1], "r ", NULL)) ){ + fprintf(stderr, "ERROR can't open file: %s\n", argv[1]); + exit(1); + } + + /* get wcs structure */ + FunInfoGet(fun,FUN_WCS,&wcs,0); + if( !wcs || !iswcs(wcs) ){ + fprintf(stderr,"No WCS data"); + return(1); + } + + /* read input, convert pixels to wcs and back */ + while(1){ + fprintf(stdout,"\nInput x y: "); + if(scanf("%lf %lf", &x, &y) != EOF){ + if(x <= -999) + break; + /* convert image pixels to sky coords */ + pix2wcs(wcs, x, y, &ra, &dec); + fprintf(stdout,"Convert from pixels to ra,dec using pix2wcs()\n"); + fprintf(stdout, "x=%.10g y=%.10g -> ra=%.10g dec=%.10g\n", + x, y, ra, dec); + /* convert sky coords to image pixels */ + fprintf(stdout,"Convert from ra,dec -> pixels using wcs2pix()\n"); + wcs2pix(wcs, ra, dec, &xr, &yr, &i); + fprintf(stdout, "ra=%.10g dec=%.10g -> x=%.10g y=%.10g offscale=%d\n", + ra, dec, xr, yr, i); + } + else + break; + } + + /* clean up */ + /* FunClose makes final WCS library call: wcsfree(wcs) */ + FunClose(fun); + return(0); +} + diff --git a/funtools/doc/view.html b/funtools/doc/view.html new file mode 100644 index 0000000..6f1faa8 --- /dev/null +++ b/funtools/doc/view.html @@ -0,0 +1,376 @@ +<!-- =defdoc funview funview n --> +<HTML> +<HEAD> +<TITLE>Database View Support for Tables</TITLE> +</HEAD> +<BODY> + +<!-- =section funview NAME --> +<H2><A NAME="funview">Funview: Database View Support for Tables</A></H2> + +<!-- =section funview SYNOPSIS --> +<H2>Summary</H2> +<P> +This document contains a summary of the options for utilizing +database-inspired Views of tables. + +<!-- =section funview DESCRIPTION --> +<H2>Description</H2> + +<H2>Database Views</H2> +<P> +In database parlance, a <B>View</B> defines a "virtual table", i.e., +a description of row and/or column selection filters (but with no +permanent storage space allocated). When used in place of a table, a +View selects the specified rows and/or columns from one or more real +tables. Views enable you to see complicated data tables in a more +convenient format. They also can be used as a security mechanism, by +restricting user access to specific columns and/or rows. [See: +<pre> +http://www.cs.unibo.it/~ciaccia/COURSES/RESOURCES/SQLTutorial/sqlch5.htm +</pre> +for a good discussion of SQL Views.] + +<P> +Funtools supports an expanded notion of Views for all tabular data +(FITS tables, raw binary tables, and ASCII column files). Funtools +Views allow you to pre-set values for the filter specification, the +columns to activate, and display format (though the latter is for +fundisp only). Setting the filter and column activation values +provides functionality equivalent to that of a classical database +View, while the ability to set the format is similar to classical +report writing capabilities. + +<H2>Funtools View Attributes</H2> +<P> +A Funtools View is a text file containing one or more of the following +columns: +<PRE> + column description + ------ ----------------------------- + view name of view + file data file name or template + filter filter specification + columns columns to activate + format fundisp format specification +</PRE> +All of the attribute columns are optional, including +the <B>view</B> name itself. This means that a View can be named or +unnamed. Unnamed Views can refer to a specific file or a template of +files (obviously if neither the view or the file column is specified, +the input View specification will never be used). You can specify any +combination of filter, column, and format parameters. (It also is +possible to apply file-specific View to other files; see the discussion +on <B>View Lists</B> below). Each column has a size limit of 1024 characters. + +<P> +For example, consider the following View file: +<PRE> + view file format columns filter + ---- ---------------------- ------ ------------ ------- + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 + x2 ${HOME}/data/snr.ev x y pi pha cir 512 512 .1 + x1 ${HOME}/data/snr.ev cir 512 512 .1 + x1a ${HOME}/data/snr.ev x y pi pha + x0 ${HOME}/data/snr.ev + xf I=%4d + xc x y pi pha + xr cir 512 512 .1 + *.ev x y pi pha + *.fit x y dx dy cir 400 400 3 + *.fits I=%3d x y dx dy cir 400 400 3 +</PRE> +This database example is in rdb format, i.e. using tab delimiters and +permitting null values. Any valid ASCII table format is acceptable, +but if you use a format that does not permit null values, it will be +necessary to quote the null strings. + +<P> +The first five entries (x3, x2, x1, x1a, x0) are named entries defining +default values specifically for the snr.ev data file. Typically, you +would use these Views by specifying View name, and the corresponding +file, filter, column, and format values would be used. Note that the x0 +View is essentially an alias for the pathname of this file. + +<P> +The next three entries define defaults that can be applied to any +file. You typically would use these View names in conjunction with +a specific file name (see <B>View Lists</B> below) so that the associated +parameter(s) were applied to that file. + +<P> +The last three entry in the database define unnamed Views that +pertains to all files ending with the specified templates. In these +cases, any View that specifies a file name matching the file template +would be processed with the associated parameter attributes. + +<H2>Invoking a Funtools View (in Place of an Input File)</H2> +<P> +To use a Funtools View, you simply pre-pend the "v:" prefix to a View name or +a file name where an input file name usually is specified. For example: +<PRE> + fundisp v:x3 +</PRE> +specifies that the View named x3 (with its file name and associated +parameters) is processed as the input file to fundisp. Using the +example database, above, this is equivalent to: +<PRE> + fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha" +</PRE> +That is, the format is used with fundisp's -f (format) switch, while the +filename and extension are composed of the x3 View's filename and +region filter. + +<P> +Similarly, executing a command such as: +<PRE> + fundisp v:foo.fit +</PRE> +will match the unnamed View associated with the template "*.fit". +This is equivalent to executing: +<PRE> + fundisp foo.fit'[cir 400 400 3]' "x y dx dy" +</PRE> +Of course, if you omit the "v:" prefix, then no View processing takes place: +<PRE> + fundisp foo.fit # process foo.fit without any View parameters + fundisp x3 # error (assuming there is no file named x3) +</PRE> + +<H2>Basic View Matching Rules</H2> + +<P> +When a "v:" prefix is recognized, Funtools searches for a View database +file in the following order: +<PRE> + location description + ------------ ------------------------------------ + FUN_VIEWFILE environment variable (any file name) + ./.funtools.vu hidden file, default name + $HOME/.funtools.vu hidden file, default name +</PRE> +The first View database file located is used to construct a new +filename, as well as an activation column specification and a format +specification. The following rules are used: + +<P> +1. An attempt is made to match the input name (i.e., the part of the +input View after the "v:" prefix) against the <B>view</B> column value +(if present) of each row in the database. If a match is found, the +values of all non-blank columns are saved for later use. Also note +that the first match terminates the search: i.e., <B>the order of the +database rows matters</B>. + +<P> +2. If no <B>view</B> match is made, an attempt is made to match the input +name against the <B>file</B> column value (if present). Matching is +performed on the full pathname of both the input name and the +database file name, and on the non-directory (root) part of these +files. This means that the root specification: +<PRE> + fundisp v:snr.ev +</PRE> +will match a row in the database that has a full pathname in the file, +allowing you to use a <B>file</B>-matched View without having to +specify the full pathname. In this example, the "v:snr.ev" View +specification will match the first row (v:x3) in the database: +<PRE> + x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1 +</PRE> +even though the row contains a fully qualified pathname as the file +value. Once again, values of all non-blank columns are saved, and the +first match terminates the search. + +<P> +3. If neither a <B>view</B> or a <B>view</B> match has been found, +then a simple template match is attempted against the <B>view</B> +values. Template matching supports a simplified version of file +globbing (not a regular expression), with support for a single "*" +(all characters), "?" (single character), or "[...]" (range) specification. + +<P> +4. If no template match was found on the <B>view</B> column, then a +simple template match is attempted against the <B>file</B> columns. + +<P> +5. If no match is found, then the filename (minus the "v:" prefix) is +returned. + +<H2>More on View Matching Rules: Single vs. Multiple Matches </H2> + +The matching rules described above stop after the first match, +regardless of whether that match provides values for all three +parameters (filter, columns, and format). In cases where a <B>view</B> +or <B>file</B> match does not provide all three values, it is possible +that a template match might do so. With regard to the example View +database above, the x1 View provides only a filter, while omitting +both the format and columns values. But note that the final rows in +the database could provide the values via a template match on the +filename. This sort of multiple matching is especially valuable in +order to provide "global" values to several Views. + +<P> +Obviously, multiple matching might not be wanted in every +case. Therefore, we support both multiple matching and single matching +according to the value of the FUN_VIEWMATCH environment variable. If +the FUN_VIEWMATCH environment variable exists and if its value begins +with "s", then a single match is used and missing parameters are not +filled in with subsequent template matches on the file name. That is, +matching rules above are followed exactly as explained above. If the +value of this environment variable begins with "m" (or does not exist), +then multiple matches are used to try to fill in missing parameters. +In this case, template matching always takes place and missing values are +taken from these template matches. + +<P> +Thus, in the example above, the View specification: +<PRE> + fundisp v:x1 +</PRE> +will take the file name and filter value from the x1 View: +<PRE> + x1 ${HOME}/data/snr.ev cir 512 512 .1 +</PRE> +The column value then will be taken from the "*.ev" file template match +against the x1 file name: +<PRE> + *.ev x y pi pha +</PRE> +Note once again that order is important: missing values are taken in the +order in which the template matches are processed. + +<H2>View Lists: Applying a View to Any File</H2> + +<P> +It is possible to apply a named View, or even several Views, to any +data file by appending a <B>viewlist</B> immediately after the standard "v:" +prefix. A viewlist takes the form: +<PRE> + :v1,v2,...vn: +</PRE> +where v1, v2, etc. are named Views. The two ":" colon characters surrounding +the list are required. Thus, the syntax for applying a viewlist to a file is: +<PRE> + v::view1,view2,...viewn:filename +</PRE> +Note that the name after the last ":" is assumed to be a file; it is +not permissible (or sensible) to use a View name. + +<P> +For example, the View specification: +<PRE> + fundisp v::x2:foo +</PRE> +applies the x2 View to the file foo (even if there is a View named foo) +and (in using our example database) is equivalent to: +<PRE> + ./fundisp foo'[cir 512 512 .1] "x y pi pha" +</PRE> +The same command can be effected using a list of Views: +<PRE> + fundisp v::x1,x1a:foo +</PRE> + +<P> +What happens if a viewlist is used and the file also matches a +template? Consider, for example, this View specification: +<PRE> + fundisp v::x2:foo.fit +</PRE> +Here, the x2 View will supply filter and column values, while the +template *.fit can also supply (different) filter and column +values. In this case, the explicitly specified Views of the viewlist +trump the matched view values. + +<P> +On the other hand, if a file template match can supply a View value +that is not supplied by the viewlist, then that value will be taken +from the file template match. For example: +<PRE> + fundisp v::x2:foo.fits +</PRE> +does not explicitly supply a format value, but the file match on *.fits +can and does. You can avoid supplying missing values using file template +matching by replacing the first ":" with a "-" in a viewlist +specification: +<PRE> + fundisp v:-x2:foo.fits +</PRE> +The use of ":+" to explicitly allow file template matching is also +supported, but is the same as the default case. Note that the nuances +of viewlist support are subject to change as our experience and +understanding grow. + +<H2>Overriding Values Associated with a View</H2> + +<P> +To override values associated with a View, simply supply the override +values in the correct place on the command line. Thus, given +the example database described above, the command: +<PRE> + fundisp v:x3 +</PRE> +specifies that the View named x3, along with its file name and +associated parameters, be processed as the input file to fundisp in +this way: +<PRE> + fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha" +</PRE> +To override one or more of these values, simply specify a new value +for the format, filter, or columns. For example, if your input View file +contains a filter, then the View will use that filter as an override +of the View filter: +<PRE> + fundisp v:x3'[cir 400 400 3]' +</PRE> +will use the columns and format of the x3 View but not the x3 filter. Further +examples are: +<PRE> + fundisp v:x3 "x y dx dy" # activate a different set of columns + fundisp -f "I=%3d" v:x3 # use a different format statement +</PRE> + +<P> +Note that extension names, extension index values, and other +non-filter specifications <B>do not</B> override the View +filter. Thus: +<PRE> + fundisp v:foo.fit[3] +</PRE> +will still use the filter associated with the .fit template (see above), since +the "3" is an extension index, not a filter. + +<H2>Environment Variables</H2> + +The following environment variables are used by Funtools Views: +<DL> +<DT><B>FUN_VIEWNAME</B> +<DD> The <B>FUN_VIEWNAME</B> environment variable specifies the +name and location of the View database file. If not present, the +files ./.funtools.vu and $HOME/.funtools.vu are searched for, in +that order. + +<DT><B>FUN_VIEWMATCH</B> +<DD> The <B>FUN_VIEWMATCH</B> environment variable specifies whether a +single match or multiple match algorithm is used to locate parameter +values. If the value of this environment variable begins with "s", +then a single match is used and missing parameters are not filled in +with subsequent template matches on the file name. If the value begins +with "m", then multiple matches are used to try to fill in missing +parameters. The default is to use multiple matches. +</DL> + +<H2>Restrictions and Problems</H2> + +Support for overriding a filter (while not overriding extension names, +extension indexes, etc.) requires that we can sense the presence of a +filter in a bracket specification. It is unclear yet whether our +algorithm is perfect. + +<P> +<A HREF="./help.html">Go to Funtools Help Index</A> + +<H5>Last updated: August 3, 2007</H5> + +</BODY> +</HTML> |