<div dir="ltr">If this double problem occurs only on a particular machine, then this is likely an MPI problem. If things are misconfigured, you are running two identical serial computations instead of one parallel simulation. The output will be correct, but it will be doubled, and the run will take twice as long.<div><br></div><div>-erik</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 13, 2017 at 10:27 AM, Ian Hinder <span dir="ltr"><<a href="mailto:ian.hinder@aei.mpg.de" target="_blank">ian.hinder@aei.mpg.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><span class=""><div>On 13 Jul 2017, at 10:48, Miguel Zilhão <<a href="mailto:mzilhao@ffn.ub.es" target="_blank">mzilhao@ffn.ub.es</a>> wrote:</div><br class="m_1220587943877014261Apple-interchange-newline"><blockquote type="cite">hi Ian,<br><br><blockquote type="cite">NewsB_scri.L02Mm01.asc: substantial differences<br> significant differences on 1 (out of 2) lines<br> maximum absolute difference in column 1 is 963<br> maximum absolute difference in column 2 is 0.000185770963653907<br> maximum absolute difference in column 3 is 0.000142466608463344<br> maximum relative difference in column 1 is 1<br> maximum relative difference in column 2 is 1<br> maximum relative difference in column 3 is 1<br> ...<br>The third, SphericalHarmonicReconGen.<wbr>SpEC-h5-test, fails like this:<br>NewsB_scri.L02Mm01.asc: substantial differences<br> significant differences on 1 (out of 2) lines<br> maximum absolute difference in column 1 is 963<br> maximum absolute difference in column 2 is 0.000185770963653907<br> maximum absolute difference in column 3 is 0.000142466608463344<br> maximum relative difference in column 1 is 1<br> maximum relative difference in column 2 is 1<br> maximum relative difference in column 3 is 1<br> ...<br>I suspect the second and third failures have the same cause. Do we have any idea why these tests fail? They don't seem to fail on any other machines (<a href="http://einsteintoolkit.org/testsuite_results/index.php" target="_blank">http://einsteintoolkit.org/<wbr>testsuite_results/index.php</a>).<br></blockquote><br>i'm not sure whether it's related, but i've had a similar issue with a testsuite of a local thorn i have, which was passing just fine with 1 proc but not with 2 procs (and similar errors). the root of the problem turned out to be that on this particular machine, for whatever reason, when running the parfile with 2 procs the output was "doubled". as if each processor was writing the same thing on the same file. the output itself was the same, but the files that were written were obviously not. so diff was signalling differences in the files where in fact the numbers themselves were (nearly) the same. could you be seeing something like this as well?<br></blockquote><div><br></div></span><div>Hi,</div><div><br></div><div>This is in general a problem; Cactus output is not guaranteed to be the same when you change the number of processes. There are ways to work around this in specific cases, which are used for the tests. Specifically, you can use the options</div><div><br></div><div><div>CarpetIOASCII::compact_format = yes</div><div>CarpetIOASCII::output_ghost_<wbr>points = no</div><div><br></div><div>and look at output only in the z direction. If Carpet splits the grid in the z direction, then I think that for small numbers of processes at least, you are guaranteed that the output will be independent of the number of processes, as the processes will write in component order to the file, and the component ordering is the same as the z-ordering in this case.</div><div><br></div><div>In your case, you are probably seeing duplicate points because some points (the ghost points) exist on more than one process, and each process outputs all the points it has. Using output_ghost_points = no might help in that case.</div><div><br></div><div>For SphericalHarmonicRecon, the tests use PUGH, not Carpet, and compact_format is not available. I will have to check if this is a diffing issue, but I would be surprised, since it works on other machines on 2 processes.</div></div></div><span class=""><br><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div>-- </div><div>Ian Hinder</div><div><a href="http://members.aei.mpg.de/ianhin" target="_blank">http://members.aei.mpg.de/<wbr>ianhin</a></div></div></div></div></div>
</div>
<br></span></div><br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@einsteintoolkit.org">Users@einsteintoolkit.org</a><br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.<wbr>org/mailman/listinfo/users</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>Erik Schnetter <<a href="mailto:schnetter@cct.lsu.edu" target="_blank">schnetter@cct.lsu.edu</a>><br><a href="http://www.perimeterinstitute.ca/personal/eschnetter/" target="_blank">http://www.perimeterinstitute.ca/personal/eschnetter/</a></div><div><br></div></div></div>
</div>