[Users] Test case requiring strict floating point operation ordering

[Ian Hinder]

On 13 May 2015, at 07:23, David Radice <david.e.pi.3.14 at gmail.com> wrote:

> Hello Ian,
> 
>> The test case test_local_interp_2 in LocalInterp2 is only supposed to pass if the compiler generates code strictly conforming to the ordering dictated by the parentheses in the source code.
> 
>> 1. Force the compiler to use strict ordering on that section of the code or that file (I don't know how to do this)
>> 2. Disable the test when strict ordering is not available
>> 3. Remove the test entirely
>> 4. Allow the test to fail on machines whose optionlists do not respect IEEE semantics
>> 
>> in decreasing order of my own preference.
> 
> Users that have applications relying on symmetric interpolation to work property would probably prefer this test to fail unless strict ordering has been preserved by the compiler (and no bugs have been introduced in the relevant code) than having the code silently produce wrong results while in production. So I would recommend against removing it. In any case, users that do not need to have symmetric interpolation would probably prefer to use the AEILocalInterp or the LocalInterp thorns and remove the *LocalInterp2 thorns from their thornlist.

The way I see it, the tests are supposed to check that the code is working as intended on the current machine.  This test doesn't make sense unless the compiler has been told to respect strict ordering of floating point operations, so "as-intended" is ambiguous.  The author of the optionlist intends that strict floating point is not required, whereas the author of the test intends that it is. The current situation, where some tests fail out of the box even though there is "nothing wrong", probably leads over time to people accepting test failures as routine, and new users who run the tests getting confused.

Here is a proposal, which unfortunately requires several bits of work:

1. Allow the user to set CCTK_STRICT_MATH in the optionlist.  This would indicate that strict ordering of floating point operations, and IEEE compliance, is required in this build.  This could potentially then be used to select compiler optimisation settings automatically.  We could then introduce additional Cactus tests, only enabled when CCTK_STRICT_MATH was set, which checked that certain operations give exactly the correct results.

2. Allow some tests to be disabled automatically if the features they require are not available.  We already silently omit tests which cannot be run due to not having all the required thorns, or not being run on the right number of processes.  I think it would be meaningful to omit this test because strict floating point compliance is "not available" in the current build.  Thus, the test would appear neither as a pass nor a fail, so would not be silently giving wrong results.  It would appear as "Tests disabled due to lack of required features" or similar, and the test would somehow indicate a one-line string which said "Test requires strict floating point conformance, but this is not available in this build".  A similar solution would help with tests requiring OpenCL, and a handful of other cases that have the same problem.  This could be implemented by adding an optional entry to test.ccl containing the name of a shell script which determined whether the test should be disabled in the current environment.  If this won't work because this is only parsed at test run time, and the information is only available at build time, then this could go into a configuration script instead.

Thoughts?

-- 
Ian Hinder
http://members.aei.mpg.de/ianhin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20150513/b43c2e71/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20150513/b43c2e71/attachment.bin