[ET Trac] [Einstein Toolkit] #2096: SphericalHarmonicRecon and SphericalHarmonicReconGen tests fail on Jenkins build machine

[Einstein Toolkit]

#2096: SphericalHarmonicRecon and SphericalHarmonicReconGen tests fail on Jenkins
build machine
------------------------------------+---------------------------------------
  Reporter:  hinder                 |       Owner:                     
      Type:  defect                 |      Status:  confirmed          
  Priority:  major                  |   Milestone:  ET_2018_02         
 Component:  EinsteinToolkit thorn  |     Version:  development version
Resolution:                         |    Keywords:                     
------------------------------------+---------------------------------------
Description changed by hinder:

Old description:

> (Adapted from
> http://lists.einsteintoolkit.org/pipermail/users/2017-July/005671.html)
>
> The following three tests fail on the Jenkins build machine:
>
> SphericalHarmonicRecon.regression_test/2procs
> SphericalHarmonicReconGen.SpEC-dat-test/2procs
> SphericalHarmonicReconGen.SpEC-h5-test/2procs
>
> These tests all pass on one process but fail on two processes, using the
> ubuntu.cfg optionlist.  They all seem to pass on multiple processes on
> all other machines, including my laptop with gcc.
>
> The first, SphericalHarmonicRecon.regression_test, fails like this:
>
> {{{
> WARNING level 0 from host 7ce14e5707a0 process 0
>   while executing schedule bin NullEvol_Initial, routine
> NullEvolve::NullEvol_InitialSlice
>   in thorn NullEvolve, file NullEvol_InitialSlice.F90:42:
>   ->  Error
> }}}
>
> The second, SphericalHarmonicReconGen.SpEC-dat-test, fails like this:
>
> {{{
> NewsB_scri.L02Mm01.asc: substantial differences
>       significant differences on 1 (out of 2) lines
>       maximum absolute difference in column 1 is 963
>       maximum absolute difference in column 2 is 0.000185770963653907
>       maximum absolute difference in column 3 is 0.000142466608463344
>       maximum relative difference in column 1 is 1
>       maximum relative difference in column 2 is 1
>       maximum relative difference in column 3 is 1
>       ...
> }}}
>
> The third, SphericalHarmonicReconGen.SpEC-h5-test, fails like this:
>
> {{{
> NewsB_scri.L02Mm01.asc: substantial differences
>       significant differences on 1 (out of 2) lines
>       maximum absolute difference in column 1 is 963
>       maximum absolute difference in column 2 is 0.000185770963653907
>       maximum absolute difference in column 3 is 0.000142466608463344
>       maximum relative difference in column 1 is 1
>       maximum relative difference in column 2 is 1
>       maximum relative difference in column 3 is 1
>       ...
> }}}
>
> I suspect the second and third failures have the same cause.  These tests
> don't seem to fail on any other machines.
>
> See the discussion thread
> http://lists.einsteintoolkit.org/pipermail/users/2017-July/thread.html#5671
> for more information.

New description:

 (Adapted from
 http://lists.einsteintoolkit.org/pipermail/users/2017-July/005671.html)

 The following three tests fail on the Jenkins build machine:

 SphericalHarmonicRecon.regression_test/2procs
 SphericalHarmonicReconGen.SpEC-dat-test/2procs
 SphericalHarmonicReconGen.SpEC-h5-test/2procs

 See https://build-
 test.barrywardell.net/job/EinsteinToolkit/1271/testReport/.

 These tests all pass on one process but fail on two processes, using the
 ubuntu.cfg optionlist.  They all seem to pass on multiple processes on all
 other machines, including my laptop with gcc.

 The first, SphericalHarmonicRecon.regression_test, fails like this:

 {{{
 WARNING level 0 from host 7ce14e5707a0 process 0
   while executing schedule bin NullEvol_Initial, routine
 NullEvolve::NullEvol_InitialSlice
   in thorn NullEvolve, file NullEvol_InitialSlice.F90:42:
   ->  Error
 }}}

 The second, SphericalHarmonicReconGen.SpEC-dat-test, fails like this:

 {{{
 NewsB_scri.L02Mm01.asc: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 1 is 963
       maximum absolute difference in column 2 is 0.000185770963653907
       maximum absolute difference in column 3 is 0.000142466608463344
       maximum relative difference in column 1 is 1
       maximum relative difference in column 2 is 1
       maximum relative difference in column 3 is 1
       ...
 }}}

 The third, SphericalHarmonicReconGen.SpEC-h5-test, fails like this:

 {{{
 NewsB_scri.L02Mm01.asc: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 1 is 963
       maximum absolute difference in column 2 is 0.000185770963653907
       maximum absolute difference in column 3 is 0.000142466608463344
       maximum relative difference in column 1 is 1
       maximum relative difference in column 2 is 1
       maximum relative difference in column 3 is 1
       ...
 }}}

 I suspect the second and third failures have the same cause.  These tests
 don't seem to fail on any other machines.

 See the discussion thread
 http://lists.einsteintoolkit.org/pipermail/users/2017-July/thread.html#5671
 for more information.

--

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/2096#comment:2>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit