#824: Update tests which rely on BSSN_MoL to use ML_BSSN instead ---------------------+------------------------------------------------------ Reporter: hinder | Owner: rhaas Type: task | Status: reopened Priority: major | Milestone: ET_2012_05 Component: Other | Version: Resolution: | Keywords: testsuites ---------------------+------------------------------------------------------
Comment (by hinder):
With the standard optionlist, I get:
{{{ Tests failed:
schw-0050 (from Cartoon2D) test_cartoon_3 (from Cartoon2D) KS-tilted (from Exact) Kerr (from RotatingSymmetry180) Kerr-rotating-180 (from RotatingSymmetry180) Kerr-rotating-90 (from RotatingSymmetry90) Kerr-rotating-90-staggered (from RotatingSymmetry90) regression_test (from SphericalHarmonicRecon) }}}
with "-fp-model precise" I get:
{{{ Tests failed:
test_cartoon_3 (from Cartoon2D) KS-tilted (from Exact) Schwarzschild_EF (from Exact) Kerr (from RotatingSymmetry180) Kerr-rotating-180 (from RotatingSymmetry180) Kerr-rotating-90 (from RotatingSymmetry90) Kerr-rotating-90-staggered (from RotatingSymmetry90) regression_test (from SphericalHarmonicRecon) }}}
So it makes schw-0050 pass, and Schwarzschild_EF, which has always passed before, fail! Exact is very sensitive to roundoff-differences, so I'm not going to spend time on that. Looking at test_cartoon_3, the differences before were:
{{{ grr_3D_diagonal.xg: substantial differences significant differences on 1 (out of 14) lines maximum absolute difference in column 2 is 1.36424205265939e-12 maximum relative difference in column 2 is 5.71414569021309e-16 (insignificant differences on 2 lines) grr_maximum.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.81898940354586e-12 maximum relative difference in column 2 is 1.41519696318623e-16 grr_minimum.xg: differences below tolerance on 2 lines grr_norm1.xg: differences below tolerance on 2 lines grr_norm2.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.02318153949454e-12 maximum relative difference in column 2 is 1.78066858765207e-15 (insignificant differences on 1 lines) grr_x_[2][3].xg: substantial differences significant differences on 3 (out of 42) lines maximum absolute difference in column 2 is 3.63797880709171e-12 maximum relative difference in column 2 is 2.83039392637247e-16 (insignificant differences on 1 lines) grr_y_[2][3].xg: substantial differences significant differences on 1 (out of 14) lines maximum absolute difference in column 2 is 1.81898940354586e-12 maximum relative difference in column 2 is 3.37736718006668e-15 (insignificant differences on 3 lines) grr_z_[2][2].xg: substantial differences significant differences on 2 (out of 72) lines maximum absolute difference in column 2 is 3.63797880709171e-12 maximum relative difference in column 2 is 3.89684464464154e-14 (insignificant differences on 1 lines) gxx_3D_diagonal.xg: differences below tolerance on 2 lines gxx_minimum.xg: differences below tolerance on 1 lines gxx_norm1.xg: differences below tolerance on 2 lines gxx_norm2.xg: substantial differences significant differences on 2 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97852014937682e-15 gxx_x_[2][3].xg: substantial differences significant differences on 2 (out of 42) lines maximum absolute difference in column 2 is 1.81898940354586e-12 maximum relative difference in column 2 is 5.22440810962943e-15 (insignificant differences on 1 lines) gxx_y_[2][3].xg: differences below tolerance on 4 lines gxy_3D_diagonal.xg: differences below tolerance on 1 lines gxy_maximum.xg: differences below tolerance on 1 lines gxy_minimum.xg: differences below tolerance on 1 lines gxy_y_[2][3].xg: differences below tolerance on 4 lines gyy_3D_diagonal.xg: differences below tolerance on 3 lines gyy_minimum.xg: differences below tolerance on 1 lines gyy_norm1.xg: differences below tolerance on 2 lines gyy_norm2.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97851928863973e-15 (insignificant differences on 1 lines) gyy_x_[2][3].xg: substantial differences significant differences on 3 (out of 42) lines maximum absolute difference in column 2 is 1.81898940354586e-12 maximum relative difference in column 2 is 5.2244263577245e-15 gyy_y_[2][3].xg: differences below tolerance on 2 lines gzz_3D_diagonal.xg: substantial differences significant differences on 1 (out of 14) lines maximum absolute difference in column 2 is 1.36424205265939e-12 maximum relative difference in column 2 is 5.71410366136695e-16 (insignificant differences on 2 lines) gzz_norm1.xg: differences below tolerance on 2 lines gzz_norm2.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97852013904379e-15 gzz_x_[2][3].xg: substantial differences significant differences on 2 (out of 42) lines maximum absolute difference in column 2 is 1.81898940354586e-12 maximum relative difference in column 2 is 1.41519740783568e-16 gzz_y_[2][3].xg: differences below tolerance on 2 lines ham_z_[2][2].xg: differences below tolerance on 1 lines
}}}
and after they are:
{{{ grr_norm1.xg: differences below tolerance on 2 lines grr_norm2.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97852065294675e-15 (insignificant differences on 1 lines) gxx_norm1.xg: differences below tolerance on 2 lines gxx_norm2.xg: substantial differences significant differences on 2 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97852014937682e-15 gyy_norm1.xg: differences below tolerance on 2 lines gyy_norm2.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97851928863973e-15 (insignificant differences on 1 lines) gzz_norm1.xg: differences below tolerance on 2 lines gzz_norm2.xg: substantial differences significant differences on 1 (out of 2) lines maximum absolute difference in column 2 is 1.13686837721616e-12 maximum relative difference in column 2 is 1.97852013904379e-15 }}}
i.e. the only quantities which are now significantly different are the norms. Note that only 14 digits are output. The relative differences are of the order of 1e-15, but the absolute differences in the norms are of the order of 1e-12 (the default tolerance for the tests is 1e-12). If we accept an absolute tolerance of 1e-12 for the variables, then shouldn't we should multiply this by the volume of the domain for the norms? In any case, I think that differences of the order of 1e-9 would be perfectly fine, and I think the tolerances could just be increased.