<div dir="ltr">Bernard<div><br></div><div>Did you set the parameters that Ian Hinder suggested? This will make interpreting the timing output much easier.</div><div><br></div><div style>Time is not only spent in routines, but also in infrastructure tasks not listed here (regridding etc.). Ian Hinder implemented a tree-form timer output that makes it much easier to see how much time is spent on what task. I don't recall the options to select this -- Ian, is this Carpet::output_timer_tree_every and friends?</div>
<div><br></div><div style>-erik</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Feb 18, 2013 at 2:59 PM, Kelly, Bernard J. (GSFC-660.0)[UNIVERSITY OF MARYLAND BALTIMORE COUNTY] <span dir="ltr"><<a href="mailto:bernard.j.kelly@nasa.gov" target="_blank">bernard.j.kelly@nasa.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">Hi Roland, and thanks for your reply. I'm still a bit confused, I confess<br>
(see below) ...<br>
<br>
On 2/16/13 12:38 AM, "Roland Haas" <<a href="mailto:roland.haas@physics.gatech.edu">roland.haas@physics.gatech.edu</a>> wrote:<br>
<br>
</div><div class="im">>Hello Bernard,<br>
><br>
>> Hi. Can someone explain to me why ML_BSSN calls SelectBoundConds within<br>
>> MoL_PostStep? It seems like the kind of once-off routine that would<br>
>>happen<br>
>> near the start of a simulation, rather than something that has to be<br>
>> performed every single timestep.<br>
>In the "new" boundary/symmetry interface (ie using thorn Boundary and<br>
>Symbase) one has to Select the variables for boundaries each time before<br>
>ApplyBCs is scheduled. There is a routine in Boundaries that clears the<br>
>selection.<br>
<br>
</div><div class="im">OK; but can you tell me *why* this is? Why should we ever have to<br>
re-specify what kind of boundary conditions we use during a simulation,<br>
any more than we re-specify the evolution equations? Perhaps I don't<br>
really understand what "select the variables" means here.<br>
<br>
><br>
</div><div class="im">>> I wouldn't mind, but while trying to understand why ML_BSSN was evolving<br>
>> so slowly on one of our machines, I looked at the TimerReport files, and<br>
>> saw that SelectBoundConds was taking *much* more time (like 20 times as<br>
>> long) than the actual RHS calculation routines.<br>
>The long time is most likely caused by the fact that the boundary<br>
>selection routine tends to be the one calling SYNC which means it is the<br>
>one that does an MPI wait (if there is load imbalance) and communicates<br>
>data for buffer zone prolongation etc.<br>
<br>
</div><div class="im">So it might be spending most of the time waiting for other cores to catch<br>
</div>up? But if it's really waiting for prior routines to finish on other<br>
<div class="im">processors, then on the handful of cores where SBC appears significantly<br>
*quicker* than usual (e.g. ~50,000 seconds instead of ~100,000) I should<br>
see earlier routines taking correspondingly *longer*, right? But I don't.<br>
<br>
</div><div class="im">I'm attaching TimerReport files for two cores on the same (128-core)<br>
</div>evolution. Core 000 is typical. Line 3591 (the most up-to-date instance of<br>
<div class="HOEnZb"><div class="h5">"large" SBC behaviour) shows about 100K seconds spent cumulatively over<br>
the simulation so far. Core 052 shows only about half as much time used in<br>
the same routine, but I can't see what other EVOL routines might be taking<br>
up the slack.<br>
<br>
(Note, BTW, that what I'm running isn't vanilla ML_BSSN, but a locally<br>
modified version called MH_BSSN. The scheduling and most routines are<br>
almost identical to McLachlan)<br>
<br>
Bernard<br>
<br>
><br>
>Yours,<br>
>Roland<br>
<br>
</div></div><br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@einsteintoolkit.org">Users@einsteintoolkit.org</a><br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Erik Schnetter <<a href="mailto:schnetter@cct.lsu.edu" target="_blank">schnetter@cct.lsu.edu</a>><br><a href="http://www.perimeterinstitute.ca/personal/eschnetter/" target="_blank">http://www.perimeterinstitute.ca/personal/eschnetter/</a>
</div>