On 13 May 2015, at 07:23, David Radice david.e.pi.3.14@gmail.com wrote:

Hello Ian,

...

The test case test_local_interp_2 in LocalInterp2 is only supposed to pass if the compiler generates code strictly conforming to the ordering dictated by the parentheses in the source code.

...

1. Force the compiler to use strict ordering on that section of the code or that file (I don't know how to do this)
2. Disable the test when strict ordering is not available
3. Remove the test entirely
4. Allow the test to fail on machines whose optionlists do not respect IEEE semantics

in decreasing order of my own preference.

Users that have applications relying on symmetric interpolation to work property would probably prefer this test to fail unless strict ordering has been preserved by the compiler (and no bugs have been introduced in the relevant code) than having the code silently produce wrong results while in production. So I would recommend against removing it. In any case, users that do not need to have symmetric interpolation would probably prefer to use the AEILocalInterp or the LocalInterp thorns and remove the *LocalInterp2 thorns from their thornlist.

The way I see it, the tests are supposed to check that the code is working as intended on the current machine. This test doesn't make sense unless the compiler has been told to respect strict ordering of floating point operations, so "as-intended" is ambiguous. The author of the optionlist intends that strict floating point is not required, whereas the author of the test intends that it is. The current situation, where some tests fail out of the box even though there is "nothing wrong", probably leads over time to people accepting test failures as routine, and new users who run the tests getting confused.

Here is a proposal, which unfortunately requires several bits of work:

1. Allow the user to set CCTK_STRICT_MATH in the optionlist. This would indicate that strict ordering of floating point operations, and IEEE compliance, is required in this build. This could potentially then be used to select compiler optimisation settings automatically. We could then introduce additional Cactus tests, only enabled when CCTK_STRICT_MATH was set, which checked that certain operations give exactly the correct results.

2. Allow some tests to be disabled automatically if the features they require are not available. We already silently omit tests which cannot be run due to not having all the required thorns, or not being run on the right number of processes. I think it would be meaningful to omit this test because strict floating point compliance is "not available" in the current build. Thus, the test would appear neither as a pass nor a fail, so would not be silently giving wrong results. It would appear as "Tests disabled due to lack of required features" or similar, and the test would somehow indicate a one-line string which said "Test requires strict floating point conformance, but this is not available in this build". A similar solution would help with tests requiring OpenCL, and a handful of other cases that have the same problem. This could be implemented by adding an optional entry to test.ccl containing the name of a shell script which determined whether the test should be disabled in the current environment. If this won't work because this is only parsed at test run time, and the information is only available at build time, then this could go into a configuration script instead.

Thoughts?

On Wed, May 13, 2015 at 10:28 AM, Bruno Giacomazzo < bruno.giacomazzo@unitn.it> wrote:

On 13 May 2015, at 11:17, Ian Hinder ian.hinder@aei.mpg.de wrote:

Allow the user to set CCTK_STRICT_MATH in the optionlist. This would indicate that strict ordering of floating point operations, and IEEE compliance, is required in this build. This could potentially then be used to select compiler optimisation settings automatically. We could then introduce additional Cactus tests, only enabled when CCTK_STRICT_MATH was set, which checked that certain operations give exactly the correct results.

I like this idea.

Assuming this is too big a change so close to the release, would it be a good idea to disable the test for now and then implement this after the release?

Hello all,

...

...

Allow the user to set CCTK_STRICT_MATH in the optionlist. This would indicate that strict ordering of floating point operations, and IEEE compliance, is required in this build. This could potentially then be used to select compiler optimisation settings automatically. We could then introduce additional Cactus tests, only enabled when CCTK_STRICT_MATH was set, which checked that certain operations give exactly the correct results.

That is probably doable but requires changes to the script that parses test.ccl and we need to check that make makes the option list variables accessible to its daughter processes (such as that script).

I added the required pragmas to LocalInterp2 to actually get IEEE conforming arithmetic in the file in question (there is a simimlar one for gcc as well in the no-associative-math) to demonstrate how this could be done. This lets the test pass on datura (where it failed before). This can be found here:

https://bitbucket.org/cactuscode/cactusnumerical/pull-request/4/enforce-ieee...

This is too late for the release I think, and for the release we should disable the test altogether I fear (not liking it though) unless we are happy with the C++ code change (and the corresponding gcc options for those option lists where -Ofast is used).

...

I like this idea.

I agree -- provided the fact that the test did not run is shown in the same manner that tests that are not run for other reasons is shown. Otherwise there is a high chance of someone trusting the fact that "all tests passed" that the interpolation is actually symmetric. The C++ code must then be changed to depend on the same flag and not its own LOCALINTERP_SYMMETRIC preprocessor define anymore. Basically unless CCTK_STRICT_MATH is set, there may not be C++ code that claims to preserve the symmetry.

Yours, Roland