Skip to content
Aborted

Console Output

Started by user rhaas
Building on master in workspace /var/lib/jenkins/workspace/Cactus
Updating https://github.com/rhaas80/SWTools-BWCI/trunk/EinsteinToolkit@HEAD at revision HEAD
UU        2016_05/cle5.2_gnu5.1.0/retest
At revision 34

[Cactus] $ /bin/sh -xe /tmp/hudson6638497666052822264.sh
+ /usr/local/bin/jenkins_sync /var/lib/jenkins/workspace/Cactus/EinsteinToolkit
NCSA JYC System 
Access status: User Access UnRestricted.

Access by OTP or Two Factor Certificate Authority only.
Use myproxy-logon -s tfca.ncsa.illinois.edu -p 7512 for gsissh access.
gsissh or ssh -o PreferredAuthentications=keyboard-interactive for otp access.

executing script:
SW_CONFIG="/scratch/system/bwjenkins/sw_config"
PYTHONPATH="/sw/tools/bin:/sw/tools/bin:$PYTHONPATH"
SW_WORKDIR="/scratch/system/bwjenkins/sw_workdir"
SW_ROOT="/scratch/system/bwjenkins/tests"
PATH="/sw/tools/bin:/sw/tools/bin:$PATH"

echo $PATH $SW_CONFIG $PYTHONPATH > /dev/null
export SW_CONFIG=$SW_CONFIG

export SW_WORKDIR=$SW_WORKDIR

cd $SW_ROOT
cd EinsteinToolkit/2016_05/cle5.2_gnu5.1.0
pwd -P
swbuildwait
ls -l
/u/staff/rhaas/bin/swtestwait || true
ls -l
# echo -n "YVALUE=" >builderrorcount  && ( grep --count '^error:' <make.log >>builderrorcount || [ $? -eq 1 ] )
# echo -n "YVALUE=" >buildwarningcount  && ( grep --count '^warning:' <make.log >>buildwarningcount || [ $? -eq 1 ] )
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0
swbuild -s -a EinsteinToolkit -v 2016_05 -b cle5.2_gnu5.1.0
set -e # abort on errors
# clear out old installation to prevent potential libtool chmod 
# commands from failing when reinstalled by another person
rm -rf $ET_SRCDIR
#clear out status file since re-making
rm -f status

cd $SW_BLDDIR

# GNU tar exists with 2 for actual errors and 1 for "file changed while
# reading" yet 2 is treated specially by swtools
tar -xzf /sw/sources/EinsteinToolkit/${ET_VER}/${ET_SRCDIR}.tar.gz || exit 1

cd $ET_SRCDIR

simfactory/bin/sim setup-silent --machine bluewaters


Here we will define some necessary Simulation Factory defaults.


Determining local machine name: jyc1.ncsa.illinois.edu
Creating machine jyc1.ncsa.illinois.edu from generic: machine jyc1.ncsa.illinois.edu [/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/repos/simfactory2/mdb/machines/jyc1.ncsa.illinois.edu.ini] created successfully

--------------------SUMMARY--------------------:


[default]
user            = bwjenkins
email           = bwjenkins
allocation      = NO_ALLOCATION


------------------END SUMMARY------------------:

Contents successfully written to /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/repos/simfactory2/etc/defs.local.ini
cat >>simfactory/etc/defs.local.ini <<EOF
[bluewaters]
sourcebasedir = $SW_BLDDIR
basedir = $SW_WORKDIR
allocation = fyy
submitpattern=(\d+[.]jyc)
rsynccmd=rsync
EOF
# even when the build itself fails we still want to compute the number of error
# messages to plot them in Jenkins
#simfactory/bin/sim build --machine bluewaters --thornlist manifest/einsteintoolkit.th 2>&1 | tee make.log
simfactory/bin/sim build --machine bluewaters --thornlist /dev/null 2>&1 | tee make.log
Using configuration: sim
Reconfiguring sim
Writing configuration to: /scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/configs/sim/OptionList
Cactus - version: 4.2.3
Reconfiguring sim.
Using configuration options from configure line
  Setting fds to '4,5 -j --'
End of options from configure line
Adding configuration options from '/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/configs/sim/OptionList'...
  Setting VERSION to 'bluewaters-gnu-2016-01-26'
  Setting CPP to 'cpp'
  Setting FPP to 'cpp'
  Setting CC to 'cc'
  Setting CXX to 'CC'
  Setting CUCC to 'nvcc'
  Setting F77 to 'ftn'
  Setting F90 to 'ftn'
  Setting CPPFLAGS to '-g3 -DDISABLE_AVX'
  Setting FPPFLAGS to '-g3 -traditional'
  Setting CFLAGS to '-g3 -m128bit-long-double -std=gnu99 -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4/lib -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4_mp/lib'
  Setting CXXFLAGS to '-g3 -m128bit-long-double -std=gnu++11 -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4/lib -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4_mp/lib'
  Setting CUCCFLAGS to '-g -m64 -arch=sm_30 --ptxas-options=-v'
  Setting F90FLAGS to '-g3 -m128bit-long-double -fcray-pointer -ffixed-line-length-none'
  Setting F77FLAGS to '-g3 -m128bit-long-double -fcray-pointer -ffixed-line-length-none'
  Setting LIBS to 'gfortran   dl'
  Setting C_LINE_DIRECTIVES to 'yes'
  Setting F_LINE_DIRECTIVES to 'yes'
  Setting VECTORISE to 'yes'
  Setting VECTORISE_ALIGNED_ARRAYS to 'no'
  Setting VECTORISE_INLINE to 'yes'
  Setting DEBUG to 'no'
  Setting CPP_DEBUG_FLAGS to '-DCARPET_DEBUG'
  Setting FPP_DEBUG_FLAGS to '-DCARPET_DEBUG'
  Setting C_DEBUG_FLAGS to '-ftrapv -fstack-protector-all -fbounds-check'
  Setting CXX_DEBUG_FLAGS to '-ftrapv -fstack-protector-all -fbounds-check'
  Setting CUCC_DEBUG_FLAGS to ''
  Setting F77_DEBUG_FLAGS to '-ftrapv -fstack-protector-all -fcheck=bounds,do,mem,pointer,recursion -finit-character=65 -finit-integer=42424242 -finit-real=nan'
  Setting F90_DEBUG_FLAGS to '-ftrapv -fstack-protector-all -fcheck=bounds,do,mem,pointer,recursion -finit-character=65 -finit-integer=42424242 -finit-real=nan'
  Setting OPTIMISE to 'yes'
  Setting CPP_OPTIMISE_FLAGS to ''
  Setting FPP_OPTIMISE_FLAGS to ''
  Setting C_OPTIMISE_FLAGS to '-Ofast -funroll-loops'
  Setting CXX_OPTIMISE_FLAGS to '-Ofast -funroll-loops'
  Setting CUCC_OPTIMISE_FLAGS to '-O3 -use_fast_math'
  Setting F90_OPTIMISE_FLAGS to '-Ofast -funroll-loops'
  Setting F77_OPTIMISE_FLAGS to '-Ofast -funroll-loops'
  Setting PROFILE to 'no'
  Setting CPP_PROFILE_FLAGS to ''
  Setting FPP_PROFILE_FLAGS to ''
  Setting C_PROFILE_FLAGS to '-pg'
  Setting CXX_PROFILE_FLAGS to '-pg'
  Setting F77_PROFILE_FLAGS to '-pg'
  Setting F90_PROFILE_FLAGS to '-pg'
  Setting OPENMP to 'yes'
  Setting CPP_OPENMP_FLAGS to '-fopenmp'
  Setting FPP_OPENMP_FLAGS to '-fopenmp'
  Setting C_OPENMP_FLAGS to '-fopenmp'
  Setting CXX_OPENMP_FLAGS to '-fopenmp'
  Setting F77_OPENMP_FLAGS to '-fopenmp'
  Setting F90_OPENMP_FLAGS to '-fopenmp'
  Setting WARN to 'yes'
  Setting CPP_WARN_FLAGS to '-Wall'
  Setting FPP_WARN_FLAGS to '-Wall'
  Setting C_WARN_FLAGS to '-Wall'
  Setting CXX_WARN_FLAGS to '-Wall'
  Setting CUCC_WARN_FLAGS to '-Xcompiler -Wall'
  Setting F90_WARN_FLAGS to '-Wall'
  Setting F77_WARN_FLAGS to '-Wall'
  Setting BLAS_DIR to '/opt/acml/5.3.1/gfortran64_fma4_mp/lib'
  Setting BLAS_LIBS to 'acml_mp'
  Setting BOOST_DIR to '/sw/xe/boost/1.53.0/sles11.3_gnu4.8.2'
  Setting FFTW3_DIR to '/opt/fftw/3.3.4.6/interlagos'
  Setting GSL_DIR to '/sw/xe/gsl/1.15/cnl4.1_gnu4.7.2'
  Setting HDF5_DIR to '/opt/cray/hdf5/1.8.14/GNU/49'
  Setting HWLOC_DIR to 'BUILD'
  Setting HWLOC_EXTRA_LIBS to 'numa'
  Setting LAPACK_DIR to '/opt/acml/5.3.1/gfortran64_fma4_mp/lib'
  Setting LAPACK_LIBS to 'acml_mp'
  Setting MPI_DIR to 'NONE'
  Setting OPENBLAS_DIR to '/opt/acml/5.3.1/gfortran64_fma4_mp'
  Setting OPENBLAS_LIBS to 'acml_mp'
  Setting OPENCL_DIR to '/opt/nvidia/cudatoolkit/7.0.28-1.0502.10742.5.1'
  Setting OPENCL_INC_DIRS to '/opt/nvidia/cudatoolkit/7.0.28-1.0502.10742.5.1/include'
  Setting OPENCL_LIB_DIRS to '/opt/nvidia/cudatoolkit/7.0.28-1.0502.10742.5.1lib64'
  Setting OPENCL_LIBS to 'OpenCL'
  Setting PAPI_DIR to '/opt/cray/papi/5.4.1.3'
  Setting PAPI_LIB_DIRS to '/opt/cray/papi/5.4.1.3/lib64'
  Setting PETSC_DIR to '/opt/cray/petsc/3.6.1.0/real/GNU/49/interlagos'
  Setting PTHREADS_DIR to 'NO_BUILD'
End of options from '/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/configs/sim/OptionList'.
creating cache ./config.cache
checking host system type... x86_64-unknown-linux-gnu
checking whether make sets ${MAKE}... yes
checking whether the C compiler (cc -g3 -m128bit-long-double -std=gnu99 -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4/lib -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4_mp/lib ) works... yes
checking whether the C compiler (cc -g3 -m128bit-long-double -std=gnu99 -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4/lib -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4_mp/lib ) is a cross-compiler... no
checking whether we are using GNU C... yes
checking whether the C++ compiler (CC -g3 -m128bit-long-double -std=gnu++11 -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4/lib -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4_mp/lib ) works... yes
checking whether the C++ compiler (CC -g3 -m128bit-long-double -std=gnu++11 -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4/lib -Wl,-rpath,/opt/acml/5.3.1/gfortran64_fma4_mp/lib ) is a cross-compiler... no
checking whether we are using GNU C++... yes
checking for ranlib... ranlib
checking for perl... perl
checking for gtar... no
checking for tar... tar
checking for xargs... xargs
checking for gzip... gzip
checking for gpatch... no
checking for patch... patch
checking for git... git
checking for ar... ar
checking for mkdir... mkdir
checking for bash... /bin/sh
Setting CACHELINE_BYTES to 64
Setting CACHE_SIZE to 512*1024 bytes
checking how to run the C preprocessor... cpp
checking for ANSI C header files... yes
checking for C99 features... yes
checking for M_PI... yes
checking for C strdup... yes
checking for C++ strdup... yes
checking whether byte ordering is bigendian... no
checking size of long long... 8
checking size of long int... 8
checking size of int... 4
checking size of short int... 2
checking size of long double... 16
checking size of double... 8
checking size of float... 4
checking size of char *... 8
checking for the null device... /dev/null
checking for gethostbyname... yes
checking whether mode_t is defined... yes
checking for availability of gettimeofday timing... yes
checking if gettimeofday needs timezone... yes
checking for availability of getrusage timing... yes
checking for availability of _ftime timing... no
checking for time.h... yes
checking for sys/time.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for string.h... yes
checking for assert.h... yes
checking for tgmath.h... yes
checking for sys/stat.h... yes
checking for getopt.h... yes
checking for dirent.h... yes
checking for c_asm.h... no
checking for intrinsics.h... no
checking for mach/mach_time.h... no
checking for regex.h... yes
checking for sys/filio.h... no
checking for sys/ioctl.h... yes
checking for sys/socket.h... yes
checking for netinet/in.h... yes
checking for netdb.h... yes
checking for arpa/inet.h... yes
checking for winsock2.h... no
checking for crypt.h... yes
checking for signal.h... yes
checking for malloc.h... yes
checking for sched.h... yes
checking for execinfo.h... yes
checking whether time.h and sys/time.h may both be included... yes
checking whether socklen_t is defined... yes
checking whether SOCKET is defined... no
checking for hrtime_t... no
checking for gethrtime... no
checking for read_real_time... no
checking for time_base_to_time... no
checking for clock_gettime... yes
checking for mach_absolute_time... no
checking for _rtc intrinsic... no
checking for mallinfo... yes
checking for mallopt... yes
checking whether M_MMAP_THRESHOLD is defined... yes
checking for sched_getaffinity... yes
checking for getpid... yes
checking if __int64 is defined... no
checking if __int128 is defined... yes
checking for vector... yes
checking for vector.h... no
checking for getopt_long_only... yes
checking for working const... yes
checking for C inline... inline
checking for C static inline... static inline
checking for C restrict... restrict
checking for C++ restrict... __restrict__
checking for C++ copysign... std::copysign
checking for C++ fpclassify... std::fpclassify
checking for C++ isfinite... std::isfinite
checking for C++ isinf... std::isinf
checking for C++ isnan... std::isnan
checking for C++ isnormal... std::isnormal
checking for C++ signbit... std::signbit
checking for C _Pragma... yes
checking for C function __attribute__((__noinline__))... yes
checking for C++ function __attribute__((__noinline__))... yes
checking for C++ member function __attribute__((__noinline__))... yes
checking for C function __attribute__((__always_inline__))... yes
checking for C++ function __attribute__((__always_inline__))... yes
checking for C++ member function __attribute__((__always_inline__))... yes
checking for C __attribute__((__unused__))... yes
checking for C++ __attribute__((__unused__))... yes
checking for C __attribute__((__aligned__(...)))... yes
checking for C++ __attribute__((__aligned__(...)))... yes
checking for C __attribute__((__cold__))... yes
checking for C++ __attribute__((__cold__))... yes
checking for C __attribute__((__hot__))... yes
checking for C++ __attribute__((__hot__))... yes
checking for C __attribute__((__format__(printf, 1, 2)))... yes
checking for C++ __attribute__((__format__(printf, 1, 2)))... yes
checking for C __attribute__((__noreturn__))... yes
checking for C++ __attribute__((__noreturn__))... yes
checking for C __attribute__((__nonnull__))... yes
checking for C++ __attribute__((__nonnull__))... yes
checking for C __attribute__((__returns_nonnull__))... yes
checking for C++ __attribute__((__returns_nonnull__))... yes
checking for C __builtin_expect... yes
checking for C++ __builtin_expect... yes
checking for C __builtin_unreachable... yes
checking for C++ __builtin_unreachable... yes
checking for C __builtin_assume_aligned... yes
checking for C++ __builtin_assume_aligned... yes
checking for C++ static_assert... yes
checking for C++ auto specifier... yes
checking for C++ lambda expressions... yes
checking for C++ range-based for statements... yes
checking for Fortran REAL*4... yes
checking for Fortran REAL*8... yes
checking for Fortran REAL*16... yes
checking for Fortran COMPLEX*8... yes
checking for Fortran COMPLEX*16... yes
checking for Fortran COMPLEX*32... yes
checking for cxxabi.h... yes
checking for __cxa_demangle... yes
checking for Dl_info.dli_sname... yes
checking for dladdr... yes
checking for backtrace... yes
checking for backtrace_symbols... yes
checking for crypt... no
checking for crypt in library crypt... yes
checking for finite in header math.h and library m... yes
checking for copysign in header math.h and library m... yes
checking for fpclassify in header math.h and library m... yes
checking for isfinite in header math.h and library m... yes
checking for isinf in header math.h and library m... yes
checking for isnan in header math.h and library m... yes
checking for isnormal in header math.h and library m... yes
checking for signbit in header math.h and library m... yes
checking for mkstemp... yes
checking for va_copy... yes
creating cctk_Archdefs.h
creating make.arch.defn
creating cctk_Extradefs.h
creating make.extra.defn
updating cache ./config.cache
creating ./config.status
creating make.config.defn
creating make.config.deps
creating make.config.rules
creating cctk_Config.h
Determining number of fortran underscores...
Compiling test file with ftn -g3 -m128bit-long-double -fcray-pointer -ffixed-line-length-none ...
Lowercase - One trailing underscore
Compiling test file with ftn -g3 -m128bit-long-double -fcray-pointer -ffixed-line-length-none ...
Lowercase - One trailing underscore
Use make sim to build the configuration.
Complete rebuild required
Cleaning sim
Cleaning configuration sim
Deleting all built files in /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/configs/sim
Warning: default submit script contents have changed
Updated script file for configuration sim
Warning: default run script contents have changed
Updated runscript file for configuration sim
Updated thorn list for configuration sim
Building sim
Cactus - version: 4.2.3
Building configuration sim
Processing CCL files
Reading ThornList...
Parsing configuration files...
Parsing interface files...
   Cactus
           Implements: Cactus
           0 Public, 0 Protected, 0 Private variable groups
Parsing parameter files...
   Cactus
          0 Global, 6 Restricted, 13 Private parameters
Parsing schedule files...
   Cactus
          0 schedule blocks.
Running any thorn-provided configuration scripts...
Checking consistency...
Creating Thorn-Flesh bindings...
   Creating implementation bindings...
   Creating parameter bindings...
   Creating variable bindings...
   Creating schedule bindings...
   Creating function bindings...
CST finished.
Checking status of thorn Cactus
Checking status of thorn CactusBindings
COMPILING configs/sim/bindings/Implementations/ImplementationBindings.c
COMPILING configs/sim/bindings/Functions/AliasedFunctions.c
COMPILING configs/sim/bindings/Variables/BindingsVariables.c
COMPILING configs/sim/bindings/Variables/Cactus.c
COMPILING configs/sim/bindings/Functions/IsFunctionAliased.c
COMPILING src/schedule/ScheduleSorter.c
COMPILING configs/sim/bindings/Parameters/BindingsParameters.c
COMPILING src/schedule/ScheduleCreater.c
COMPILING src/schedule/ScheduleTraverse.c
COMPILING configs/sim/bindings/Functions/RegisterThornFunctions.c
COMPILING configs/sim/bindings/Schedule/ScheduleCactus.c
COMPILING configs/sim/bindings/Parameters/Global.c
COMPILING configs/sim/bindings/Schedule/BindingsSchedule.c
COMPILING src/IO/IOMethods.c
COMPILING src/comm/CactusSync.c
COMPILING src/IO/OverloadIO.c
COMPILING configs/sim/bindings/Parameters/Cactus_Parameters.c
COMPILING configs/sim/bindings/Schedule/BindingsParameterRecovery.c
COMPILING src/comm/CactusDefaultComm.c
COMPILING src/comm/OverloadComm.c
COMPILING src/comm/Interp.c
COMPILING src/comm/Reduction.c
COMPILING src/IO/FortranBindingsIO.c
Creating /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/configs/sim/lib/libthorn_CactusBindings.a
In file included from /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/comm/OverloadComm.c:20:0:
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h: In function 'CCTKi_DummyExit':
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:92:3: warning: function declared 'noreturn' has a 'return' statement
   return 0;                                                     \
   ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:84:59: note: in expansion of macro '__OVERLOADABLE_DUMMY'
 #define _OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)   __OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)
                                                           ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:83:36: note: in expansion of macro '_OVERLOADABLE_DUMMY'
 #define OVERLOADABLE_DUMMY(name)   _OVERLOADABLE_DUMMY(OVERLOADABLE_PREFIX, OVERLOADABLE_DUMMY_PREFIX, name)
                                    ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/comm/OverloadComm.c:151:28: note: in expansion of macro 'OVERLOADABLE_DUMMY'
 #define OVERLOADABLE(name) OVERLOADABLE_DUMMY(name)
                            ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h:87:1: note: in expansion of macro 'OVERLOADABLE'
 OVERLOADABLE(Exit)
 ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h: In function 'CCTKi_DummyAbort':
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:92:3: warning: function declared 'noreturn' has a 'return' statement
   return 0;                                                     \
   ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:84:59: note: in expansion of macro '__OVERLOADABLE_DUMMY'
 #define _OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)   __OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)
                                                           ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:83:36: note: in expansion of macro '_OVERLOADABLE_DUMMY'
 #define OVERLOADABLE_DUMMY(name)   _OVERLOADABLE_DUMMY(OVERLOADABLE_PREFIX, OVERLOADABLE_DUMMY_PREFIX, name)
                                    ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/comm/OverloadComm.c:151:28: note: in expansion of macro 'OVERLOADABLE_DUMMY'
 #define OVERLOADABLE(name) OVERLOADABLE_DUMMY(name)
                            ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h:88:1: note: in expansion of macro 'OVERLOADABLE'
 OVERLOADABLE(Abort)
 ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h: In function 'CCTKi_DummyExit':
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:92:3: warning: 'noreturn' function does return
   return 0;                                                     \
   ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:84:59: note: in expansion of macro '__OVERLOADABLE_DUMMY'
 #define _OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)   __OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)
                                                           ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:83:36: note: in expansion of macro '_OVERLOADABLE_DUMMY'
 #define OVERLOADABLE_DUMMY(name)   _OVERLOADABLE_DUMMY(OVERLOADABLE_PREFIX, OVERLOADABLE_DUMMY_PREFIX, name)
                                    ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/comm/OverloadComm.c:151:28: note: in expansion of macro 'OVERLOADABLE_DUMMY'
 #define OVERLOADABLE(name) OVERLOADABLE_DUMMY(name)
                            ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h:87:1: note: in expansion of macro 'OVERLOADABLE'
 OVERLOADABLE(Exit)
 ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h: In function 'CCTKi_DummyAbort':
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:92:3: warning: 'noreturn' function does return
   return 0;                                                     \
   ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:84:59: note: in expansion of macro '__OVERLOADABLE_DUMMY'
 #define _OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)   __OVERLOADABLE_DUMMY(prefix, dummy_prefix, name)
                                                           ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/OverloadMacros.h:83:36: note: in expansion of macro '_OVERLOADABLE_DUMMY'
 #define OVERLOADABLE_DUMMY(name)   _OVERLOADABLE_DUMMY(OVERLOADABLE_PREFIX, OVERLOADABLE_DUMMY_PREFIX, name)
                                    ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/comm/OverloadComm.c:151:28: note: in expansion of macro 'OVERLOADABLE_DUMMY'
 #define OVERLOADABLE(name) OVERLOADABLE_DUMMY(name)
                            ^
/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/src/include/CommOverloadables.h:88:1: note: in expansion of macro 'OVERLOADABLE'
 OVERLOADABLE(Abort)
 ^
COMPILING configs/sim/bindings/build/Cactus/cctk_ThornBindings.c
COMPILING src/util/CactusTimers.c
COMPILING src/util/File.c
COMPILING src/util/ParseFile.c
COMPILING src/util/RegisterKeyedFunction.c
COMPILING src/util/StoreKeyedData.c
COMPILING src/util/StoreNamedData.c
COMPILING src/util/Misc.c
COMPILING src/util/StoreHandledData.c
COMPILING src/util/SKBinTree.c
COMPILING src/util/Hash.c
COMPILING src/util/Malloc.c
COMPILING src/util/Math.c
COMPILING src/util/String.c
COMPILING src/util/StringList.c
COMPILING src/util/Time.c
COMPILING src/util/Table.c
COMPILING src/util/Network.c
COMPILING src/main/ActiveThorns.c
COMPILING src/main/Banner.c
COMPILING src/main/CactusDefaultInitialise.c
COMPILING src/main/CactusDefaultEvolve.c
COMPILING src/main/CactusDefaultShutdown.c
COMPILING src/piraha/AutoGrammar.cc
COMPILING src/main/CactusDefaultMainLoopIndex.c
COMPILING src/piraha/Bracket.cc
COMPILING src/main/CommandLine.c
COMPILING src/main/CallStartupFunctions.c
COMPILING src/piraha/Dot.cc
COMPILING src/main/Complex.c
COMPILING src/piraha/End.cc
COMPILING src/piraha/Grammar.cc
COMPILING src/main/ConfigData.c
COMPILING src/main/Coord.c
COMPILING src/main/DebugDefines.c
COMPILING src/main/DefaultTimers.c
COMPILING src/main/Dummies.c
COMPILING src/piraha/Group.cc
COMPILING src/main/flesh.cc
COMPILING src/piraha/ILiteral.cc
COMPILING src/piraha/Literal.cc
COMPILING src/main/FortranWrappers.c
COMPILING src/main/GHExtensions.c
COMPILING src/main/Groups.c
COMPILING src/main/GroupsOnGH.c
COMPILING src/piraha/Lookup.cc
COMPILING src/piraha/Matcher.cc
COMPILING src/piraha/Multi.cc
COMPILING src/main/InitialiseCactus.c
COMPILING src/piraha/Or.cc
COMPILING src/main/InitialiseDataStructures.c
COMPILING src/piraha/ReParse.cc
COMPILING src/main/MainUtils.c
COMPILING src/piraha/Seq.cc
COMPILING src/piraha/Start.cc
COMPILING src/main/Names.c
COMPILING src/main/OverloadMain.c
COMPILING src/main/Parameters.c
COMPILING src/main/ProcessCommandLine.c
COMPILING src/piraha/Call.cc
COMPILING src/piraha/smart_ptr.cc
COMPILING src/piraha/Boundary.cc
COMPILING src/main/ProcessEnvironment.c
COMPILING src/main/ProcessParameterDatabase.c
COMPILING src/piraha/NegLookAhead.cc
COMPILING src/main/ScheduleInterface.c
COMPILING src/main/SetParams.c
COMPILING src/main/ShutdownCactus.c
COMPILING src/main/Subsystems.c
COMPILING src/main/Termination.c
COMPILING src/main/Traverse.c
COMPILING src/main/WarnLevel.c
Creating /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/configs/sim/lib/libthorn_Cactus.a
Creating cactus_sim in /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05/exe from
Done creating cactus_sim.
All done !
Building utilities for sim
Building utilities for sim
Done.

# HACK: make it possible to execute some commands once run finishes
# the nicer way is of course to define out own SubmitScript or at least to make
# one *before* building the configuration but that is actually harder to do
mv configs/sim/SubmitScript configs/sim/SubmitScript.orig
gawk '/^[^#]/&&!wrotetrap { 
  print "trap \"rm @SW_BLDDIR@/.running\" EXIT"
  wrotetrap=1
}
{print}' configs/sim/SubmitScript.orig > configs/sim/SubmitScript
cat <<EOF >>configs/sim/SubmitScript
if grep --silent 'Number failed *-> 0' @RUNDIR@/TEST/sim/summary.log ; then
  echo verified >@SW_BLDDIR@/status
else
  echo unverified >@SW_BLDDIR@/status
if 
"
EOF

############################### if this far, return 0
exit 0
++ readlink -f .
+ realpath=/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0
++ basename /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0
+ build=cle5.2_gnu5.1.0
++ dirname /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0
+ version_dir=/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05
++ basename /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05
+ version=2016_05
++ dirname /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05
+ app_dir=/mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit
++ basename /mnt/abc/scratch/system/bwjenkins/tests/EinsteinToolkit
+ app=EinsteinToolkit
+ pwd
+ swtest -a EinsteinToolkit -v 2016_05 -b cle5.2_gnu5.1.0 -s --debug
+ tee test.log
['/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05']
/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/rebuild : passed (0)
total 48
-rw-rw-r-- 1 bwjenkins BW_System 26181 Sep  1 19:14 build.log
-rw-rw-r-- 1 bwjenkins BW_System   391 Aug 23 09:20 build-notes
drwxrwxr-x 9 bwjenkins BW_System  4096 Sep  1 19:13 ET_2016_05
-rwxrwxr-x 1 bwjenkins BW_System  2550 Aug 30 19:26 rebuild
lrwxrwxrwx 1 bwjenkins BW_System     7 Aug 23 09:20 relink -> rebuild
-rw-rw-r-- 1 bwjenkins BW_System   447 Aug 30 17:18 remodule
-rwxrwxr-x 1 bwjenkins BW_System  2065 Sep  1 19:11 retest
/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0
#clear out status file since re-testing
rm -f status

#cd ${SW_WORKDIR}
cd $ET_SRCDIR
: ${TESTJOBNAME=$ET_VER-swretest-$(date +%s)-$$}
date +%s
simfactory/bin/sim submit --machine bluewaters --testsuite --select-test=CactusWave/WaveToyC --walltime 4:0:0 --procs 8 --num-threads 4 --define SW_BLDDIR $PWD $TESTJOBNAME >submit.log && \
gawk -vrc=1 '/^Submit finished, job id is/{print $6;rc=0;exit} END{exit rc}' <submit.log >../.running
Warning: Current Working directory does not match Cactus sourcetree, changing to /scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/ET_2016_05
Warning: simulation "2016_05-swretest-1472775246-16309" does not exist or is not readable
Warning: Total number of threads and number of cores per node are inconsistent: procs=8, ppn-used=32 (procs must be an integer multiple of ppn-used)
cd ..

# qsub returns 0 on successful job launch, so if failure return 1
if [ $? -ne 0 ]; then
  echo "Error submitting job"
  rm -f .running
  exit 1
else
  echo "Job submitted"
  cat submit.log
  exit 2
fi
Job submitted
cat: submit.log: No such file or directory
+ ls -l
+ '[' '!' -e .running ']'
++ grep '^\w[0-9]' .running
+ jobid=170343.jyc
+ true
+ qstat 170343.jyc
+ '[' 0 -ne 0 ']'
++ qstat -f 170343.jyc
++ grep job_state
++ awk '{print $3}'
+ job_state=R
+ '[' R == R ']'
+ echo 'Submitted job 170343.jyc is running. Following .running ... '
+ break
+ trackPID=25078
+ true
+ tail -f .running --pid=25078
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
['/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05']
/scratch/system/bwjenkins/tests/EinsteinToolkit/2016_05/cle5.2_gnu5.1.0/retest : batch submitted (2)
total 49
-rw-rw-r-- 1 bwjenkins BW_System 26181 Sep  1 19:14 build.log
-rw-rw-r-- 1 bwjenkins BW_System   391 Aug 23 09:20 build-notes
drwxrwxr-x 9 bwjenkins BW_System  4096 Sep  1 19:14 ET_2016_05
-rwxrwxr-x 1 bwjenkins BW_System  2550 Aug 30 19:26 rebuild
lrwxrwxrwx 1 bwjenkins BW_System     7 Aug 23 09:20 relink -> rebuild
-rw-rw-r-- 1 bwjenkins BW_System   447 Aug 30 17:18 remodule
-rwxrwxr-x 1 bwjenkins BW_System  2065 Sep  1 19:11 retest
-rw-rw-r-- 1 bwjenkins BW_System  1265 Sep  1 19:14 test.log
Submitted job 170343.jyc is running. Following .running ... 
170343.jyc
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
+ true
+ qstat 170343.jyc
+ '[' -e .running ']'
+ '[' 0 -eq 0 ']'
+ sleep 30
Build was aborted
Aborted by rhaas
Recording plot data
Saving plot series data from: /var/lib/jenkins/workspace/Cactus/builderrorcount
Saving plot series data from: /var/lib/jenkins/workspace/Cactus/buildwarningcount
Finished: ABORTED