diff options
author | Albert Cheng <acheng@hdfgroup.org> | 2011-05-07 17:06:48 (GMT) |
---|---|---|
committer | Albert Cheng <acheng@hdfgroup.org> | 2011-05-07 17:06:48 (GMT) |
commit | bcde54c0031d29a56bea731303e0bd10d37b8bd7 (patch) | |
tree | 4eff4df18acd1ccc6fa569eb0dd1a2e598fa700e /bin/output_filter.sh | |
parent | 88a54b790d4c3fcf96161f97192be7bad44cf156 (diff) | |
download | hdf5-bcde54c0031d29a56bea731303e0bd10d37b8bd7.zip hdf5-bcde54c0031d29a56bea731303e0bd10d37b8bd7.tar.gz hdf5-bcde54c0031d29a56bea731303e0bd10d37b8bd7.tar.bz2 |
[svn-r20772] Problem:
Test scripts sometimes need to filter some system-specific messages from the
actual output so that it can match the correct expected output. These filtering
functions, ususally called "STDOUT_FILTER()" and "STDERR_FILTER()" were being
repeated in individual test scripts. This becomes a maintenance problem and
is error prone.
Solution:
Extract the two filter functions code to bin/output_filter.sh and then each
test script sources it in. This allows reuse of coding and is much easier to
maintain and to add new filtering.
Tested:
LLNL Zeus (linux64 cluster) and Dawndev (Blue-Gene cluster), both for serial
mode only.
Diffstat (limited to 'bin/output_filter.sh')
-rw-r--r-- | bin/output_filter.sh | 101 |
1 files changed, 101 insertions, 0 deletions
diff --git a/bin/output_filter.sh b/bin/output_filter.sh new file mode 100644 index 0000000..58f1c48 --- /dev/null +++ b/bin/output_filter.sh @@ -0,0 +1,101 @@ +## Copyright by The HDF Group. +## All rights reserved. +## +## This file is part of HDF5. The full HDF5 copyright notice, including +## terms governing use, modification, and redistribution, is contained in +## the files COPYING and Copyright.html. COPYING can be found at the root +## of the source code distribution tree; Copyright.html can be found at the +## root level of an installed copy of the electronic HDF5 document set and +## is linked from the top-level documents page. It can also be found at +## http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have +## access to either file, you may request a copy from help@hdfgroup.org. + +# This contains function definitions of output filtering. +# This file should only be sourced in by another shell script. +# +# Programmer: Albert Cheng +# Created Date: 2011/5/3 + + +# Some systems will dump some messages to stdout for various reasons. +# Remove them from the stdout result file. +# $1 is the file name of the file to be filtered. +# Cases of filter needed: +# 1. Sandia Red-Storm +# yod always prints these two lines at the beginning. +# LibLustre: NAL NID: 0004a605 (5) +# Lustre: OBD class driver Build Version: 1, info@clusterfs.com +# 2. LANL Lambda +# mpijob mirun -np always add an extra line at the end like: +# P4 procgroup file is /users/acheng/.lsbatch/host10524.l82 +STDOUT_FILTER() { + result_file=$1 + tmp_file=/tmp/h5test_tmp_$$ + # Filter Sandia Red-Storm yod messages. + cp $result_file $tmp_file + sed -e '/^LibLustre:/d' -e '/^Lustre:/d' \ + < $tmp_file > $result_file + # Filter LANL Lambda mpirun message. + cp $result_file $tmp_file + sed -e '/^P4 procgroup file is/d' \ + < $tmp_file > $result_file + # cleanup + rm -f $tmp_file +} + + +# Some systems will dump some messages to stderr for various reasons. +# Remove them from the stderr result file. +# $1 is the file name of the file to be filtered. +# Cases of filter needed: +# 1. MPE: +# In parallel mode and if MPE library is used, it prints the following +# two message lines whether the MPE tracing is used or not. +# Writing logfile. +# Finished writing logfile. +# 2. LANL MPI: +# The LANL MPI will print some messages like the following, +# LA-MPI: *** mpirun (1.5.10) +# LA-MPI: *** 3 process(es) on 2 host(s): 2*fln21 1*fln22 +# LA-MPI: *** libmpi (1.5.10) +# LA-MPI: *** Copyright 2001-2004, ACL, Los Alamos National Laboratory +# 3. h5diff debug output: +# Debug output all have prefix "h5diff debug: ". +# 4. AIX system prints messages like these when it is aborting: +# ERROR: 0031-300 Forcing all remote tasks to exit due to exit code 1 in task 0 +# ERROR: 0031-250 task 4: Terminated +# ERROR: 0031-250 task 3: Terminated +# ERROR: 0031-250 task 2: Terminated +# ERROR: 0031-250 task 1: Terminated +# 5. LLNL Blue-Gene mpirun prints messages like there when it exit non-zero: +# <Apr 12 15:01:49.075658> BE_MPI (ERROR): The error message in the job record is as follows: +# <Apr 12 15:01:49.075736> BE_MPI (ERROR): "killed by exit(1) on node 0" +STDERR_FILTER() { + result_file=$1 + tmp_file=/tmp/h5test_tmp_$$ + # Filter LLNL Blue-Gene error messages in both serial and parallel modes + # since mpirun is used in both modes. + cp $result_file $tmp_file + sed -e '/ BE_MPI (ERROR): /d' \ + < $tmp_file > $result_file + # Filter MPE messages + if test -n "$pmode"; then + cp $result_file $tmp_file + sed -e '/^Writing logfile./d' -e '/^Finished writing logfile./d' \ + < $tmp_file > $result_file + fi + # Filter LANL MPI messages + # and LLNL srun messages + # and AIX error messages + if test -n "$pmode"; then + cp $result_file $tmp_file + sed -e '/^LA-MPI:/d' -e '/^srun:/d' -e '/^ERROR:/d' \ + < $tmp_file > $result_file + fi + # Filter h5diff debug output + cp $result_file $tmp_file + sed -e '/^h5diff debug: /d' \ + < $tmp_file > $result_file + # clean up temporary files. + rm -f $tmp_file +} |