On Fri, Feb 22, 2013 at 12:01 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 22 Feb 2013, at 17:43, Frank Loeffler knarf@cct.lsu.edu wrote:
Hi Kentaro,
On Fri, Feb 22, 2013 at 04:48:48PM +0100, Kentaro Takami wrote:
I encountered the problem in which the simulation result is randomly
different
at each run( of course I used same executable, par file, HPC and so
on.).
I see the same for some of my runs, but I wouldn't expect otherwise.
In my case the simulation depends on 'global' quantities, e.g., quantities obtained by reductions. Reductions cannot be done exactly, at least not efficiently. Thus, values obtained via reductions (can) have always an error that is different even when running the same simulation twice, at least when done in parallel. If your simulation depends on this, then these (small) differences can quickly grow, especially in iterative schemes, e.g., hydro.
I disagree. Reductions should be deterministic; assuming the same number of MPI processes, the contributions to the reduction should always be added in the same order. If you change the number of MPI processes, then I agree that the result of reductions can change, as the order of the sum over points will change, and floating point addition is not associative.
MPI assumes that reduction operations are associative. So does OpenMP, and Carpet. So does SimFactory it its option settings. So does Kranc when it generates code...
-erik