Ian

In several places, our OpenMP parallelization is not deterministic. For example, LoopControl (and some other thorns) allocate thread work items dynamically. However, this should not influence the results.

Carpet's reduction algorithm is not deterministic. The order in which the partial results are summed up may lead to different round-off errors in norms. However, here the grid points values differ, which can't be explained by this.

Maybe there was a change to this thorn after I ran this test?

-erik

On Nov 22, 2013, at 14:49 , Ian Hinder ian.hinder@aei.mpg.de wrote:

Hi Erik,

For me, the test

RotatingDBHIVP/E2xeon_test_rdbh fails on stampede, but for you it passes on stampede. The optionlists seem to be the same. Can you think of a possible explanation? The results should be the same.

Your results: http://git.barrywardell.net/?p=EinsteinToolkitTestResults-sandbox.git;a=blob...

My results: http://git.barrywardell.net/?p=EinsteinToolkitDaturaTestResults.git;a=blob;f...

(the second repo is badly-named; the result is from stampede not datura)

I have diffed the output log files, and there is nothing suspicious. Same number of OpenMP threads, at least for the 2 proc version. I'm wondering if the differences could come down to memory alignment differences between your executable and mine, due to different strings (dates, home directories etc) being present in the executable. See http://software.intel.com/en-us/articles/run-to-run-reproducibility-of-float....

The largest absolute difference measured is 2.92969914994501e-09.

-- Ian Hinder http://numrel.aei.mpg.de/people/hinder