First of all, I should give you more information.
In my test, I don't use OpenMP. So I use only MPI. I put dissipation only for spacetime variable such as gij, kij and so on. I check these difference for GF in checkpoint files. The difference appear from 2nd step of RK at 1st iteration, where the difference are for last a few digit, but grow up as increasing iterations.
In the following case we don't see difference. *don't activate Dissipation thorn. *using only 1 refinement level. *using certain special dx=dy=dz.
Further strange behavior: When I insert non-essential line such as "WRITE(*,*)" statement in "apply_dissipation.F77", sometimes the difference is disappeared. So, I doubted incorrect pointer position.
Hi, Frank,
Finally I'm doubting the origin of the problem is "CactusNumerical/Dissipation", because when I don't activate this thorn, the problem can be disappeared.
This is interesting. So, you say you don't see random changes when Dissipation isn't active, but you do when you activate it, and everything else is the same? Which variables do you apply dissipation to?
Yes, if we don't use Dissipation, HDF5 checkpoint files can become exactly same, at least my test case. I added dissipation for the following variables: ML_BSSN::ML_log_confac ML_BSSN::ML_metric ML_BSSN::ML_trace_curv ML_BSSN::ML_curv ML_BSSN::ML_Gamma ML_BSSN::ML_lapse ML_BSSN::ML_shift ML_BSSN::ML_dtlapse ML_BSSN::ML_dtshift
Kentaro