[ET Trac] [Einstein Toolkit] #824: Update tests which rely on BSSN_MoL to use ML_BSSN instead

[Einstein Toolkit]

#824: Update tests which rely on BSSN_MoL to use ML_BSSN instead
---------------------+------------------------------------------------------
  Reporter:  hinder  |       Owner:  rhaas     
      Type:  task    |      Status:  reopened  
  Priority:  major   |   Milestone:  ET_2012_05
 Component:  Other   |     Version:            
Resolution:          |    Keywords:  testsuites
---------------------+------------------------------------------------------

Comment (by hinder):

 With the standard optionlist, I get:

 {{{
   Tests failed:

     schw-0050 (from Cartoon2D)
     test_cartoon_3 (from Cartoon2D)
     KS-tilted (from Exact)
     Kerr (from RotatingSymmetry180)
     Kerr-rotating-180 (from RotatingSymmetry180)
     Kerr-rotating-90 (from RotatingSymmetry90)
     Kerr-rotating-90-staggered (from RotatingSymmetry90)
     regression_test (from SphericalHarmonicRecon)
 }}}

 with  "-fp-model precise" I get:

 {{{
   Tests failed:

     test_cartoon_3 (from Cartoon2D)
     KS-tilted (from Exact)
     Schwarzschild_EF (from Exact)
     Kerr (from RotatingSymmetry180)
     Kerr-rotating-180 (from RotatingSymmetry180)
     Kerr-rotating-90 (from RotatingSymmetry90)
     Kerr-rotating-90-staggered (from RotatingSymmetry90)
     regression_test (from SphericalHarmonicRecon)
 }}}

 So it makes schw-0050 pass, and Schwarzschild_EF, which has always passed
 before, fail!  Exact is very sensitive to roundoff-differences, so I'm not
 going to spend time on that.  Looking at test_cartoon_3, the differences
 before were:

 {{{
    grr_3D_diagonal.xg: substantial differences
       significant differences on 1 (out of 14) lines
       maximum absolute difference in column 2 is 1.36424205265939e-12
       maximum relative difference in column 2 is 5.71414569021309e-16
       (insignificant differences on 2 lines)
    grr_maximum.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.81898940354586e-12
       maximum relative difference in column 2 is 1.41519696318623e-16
    grr_minimum.xg: differences below tolerance on 2 lines
    grr_norm1.xg: differences below tolerance on 2 lines
    grr_norm2.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.02318153949454e-12
       maximum relative difference in column 2 is 1.78066858765207e-15
       (insignificant differences on 1 lines)
    grr_x_[2][3].xg: substantial differences
       significant differences on 3 (out of 42) lines
       maximum absolute difference in column 2 is 3.63797880709171e-12
       maximum relative difference in column 2 is 2.83039392637247e-16
       (insignificant differences on 1 lines)
    grr_y_[2][3].xg: substantial differences
       significant differences on 1 (out of 14) lines
       maximum absolute difference in column 2 is 1.81898940354586e-12
       maximum relative difference in column 2 is 3.37736718006668e-15
       (insignificant differences on 3 lines)
    grr_z_[2][2].xg: substantial differences
       significant differences on 2 (out of 72) lines
       maximum absolute difference in column 2 is 3.63797880709171e-12
       maximum relative difference in column 2 is 3.89684464464154e-14
       (insignificant differences on 1 lines)
    gxx_3D_diagonal.xg: differences below tolerance on 2 lines
    gxx_minimum.xg: differences below tolerance on 1 lines
    gxx_norm1.xg: differences below tolerance on 2 lines
    gxx_norm2.xg: substantial differences
       significant differences on 2 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97852014937682e-15
    gxx_x_[2][3].xg: substantial differences
       significant differences on 2 (out of 42) lines
       maximum absolute difference in column 2 is 1.81898940354586e-12
       maximum relative difference in column 2 is 5.22440810962943e-15
       (insignificant differences on 1 lines)
    gxx_y_[2][3].xg: differences below tolerance on 4 lines
    gxy_3D_diagonal.xg: differences below tolerance on 1 lines
    gxy_maximum.xg: differences below tolerance on 1 lines
    gxy_minimum.xg: differences below tolerance on 1 lines
    gxy_y_[2][3].xg: differences below tolerance on 4 lines
    gyy_3D_diagonal.xg: differences below tolerance on 3 lines
    gyy_minimum.xg: differences below tolerance on 1 lines
    gyy_norm1.xg: differences below tolerance on 2 lines
    gyy_norm2.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97851928863973e-15
       (insignificant differences on 1 lines)
    gyy_x_[2][3].xg: substantial differences
       significant differences on 3 (out of 42) lines
       maximum absolute difference in column 2 is 1.81898940354586e-12
       maximum relative difference in column 2 is 5.2244263577245e-15
    gyy_y_[2][3].xg: differences below tolerance on 2 lines
    gzz_3D_diagonal.xg: substantial differences
       significant differences on 1 (out of 14) lines
       maximum absolute difference in column 2 is 1.36424205265939e-12
       maximum relative difference in column 2 is 5.71410366136695e-16
       (insignificant differences on 2 lines)
    gzz_norm1.xg: differences below tolerance on 2 lines
    gzz_norm2.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97852013904379e-15
    gzz_x_[2][3].xg: substantial differences
       significant differences on 2 (out of 42) lines
       maximum absolute difference in column 2 is 1.81898940354586e-12
       maximum relative difference in column 2 is 1.41519740783568e-16
    gzz_y_[2][3].xg: differences below tolerance on 2 lines
    ham_z_[2][2].xg: differences below tolerance on 1 lines

 }}}

 and after they are:

 {{{
    grr_norm1.xg: differences below tolerance on 2 lines
    grr_norm2.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97852065294675e-15
       (insignificant differences on 1 lines)
    gxx_norm1.xg: differences below tolerance on 2 lines
    gxx_norm2.xg: substantial differences
       significant differences on 2 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97852014937682e-15
    gyy_norm1.xg: differences below tolerance on 2 lines
    gyy_norm2.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97851928863973e-15
       (insignificant differences on 1 lines)
    gzz_norm1.xg: differences below tolerance on 2 lines
    gzz_norm2.xg: substantial differences
       significant differences on 1 (out of 2) lines
       maximum absolute difference in column 2 is 1.13686837721616e-12
       maximum relative difference in column 2 is 1.97852013904379e-15
 }}}

 i.e. the only quantities which are now significantly different are the
 norms.  Note that only 14 digits are output.  The relative differences are
 of the order of 1e-15, but the absolute differences in the norms are of
 the order of 1e-12 (the default tolerance for the tests is 1e-12).  If we
 accept an absolute tolerance of 1e-12 for the variables, then shouldn't we
 should multiply this by the volume of the domain for the norms?  In any
 case, I think that differences of the order of 1e-9 would be perfectly
 fine, and I think the tolerances could just be increased.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/824#comment:10>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit