[ET Trac] [Einstein Toolkit] #747: 9th order prolongation test fails with REAL_PRECISION=4

[Einstein Toolkit]

#747: 9th order prolongation test fails with REAL_PRECISION=4
---------------------+------------------------------------------------------
  Reporter:  hinder  |       Owner:  eschnett
      Type:  defect  |      Status:  new     
  Priority:  major   |   Milestone:          
 Component:  Carpet  |     Version:          
Resolution:          |    Keywords:          
---------------------+------------------------------------------------------

Comment (by hinder):

 I looked up numeric_limits, and according to
 http://www.cplusplus.com/reference/std/limits/numeric_limits,
 numeric_limits<T>::min() is of type T and is the minimum positive value
 that T can take.  i.e. it would be 1.17549435e-38F for single precision
 and 2.2250738585072014e-308 for double precision.  Do you mean
 numeric_limits<T>::epsilon() instead?  This would be 1.19209290e-07F for
 single precision and  2.2204460492503131e-16 for double precision.
 eps^3/4^ would be correspondingly 6.4e-6 and 1.8e-12, which sound more
 like what we want.  In the code, with single precision, I see:

 INFO (CarpetLib): numeric_limits<RT>::min() = 1.1920928955078125e-07
 INFO (CarpetLib): good::abs (res - y0) = 1.52587890625e-05

 Is this what you mean by borderline?  Should we use eps^4/7^?  This gives
 (1.19209290e-07)^(4/7)^ = 1.1e-4 as the tolerance for single precision and
 1.13e-9 for double precision.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/747#comment:2>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit