<div dir="ltr">On Wed, Oct 21, 2015 at 3:25 AM, Ian Hinder <span dir="ltr"><<a href="mailto:ian.hinder@aei.mpg.de" target="_blank">ian.hinder@aei.mpg.de</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><span class=""><div>On 20 Oct 2015, at 22:13, Geraint Pratten <<a href="mailto:g.pratten@sussex.ac.uk" target="_blank">g.pratten@sussex.ac.uk</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif">Thanks Ian, I think that did the trick! I rebuilt Cactus with a different MPI infrastructure and made sure that the appropriate MPI environment was loaded. Seems to be working now, e.g. <br></div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif"><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): MPI is enabled</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): Carpet is running on 12 processes</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): This is process 0</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): OpenMP is enabled</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): This process contains 1 threads, this is thread 0</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): There are 12 threads in total</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): There are 1 threads per process</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): This process runs on host node203, pid=28712</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): This process runs on 24 cores: 0-23</div><div style="margin:0px;font-size:11px;font-family:Menlo;color:rgb(245,245,245);background-color:rgb(0,0,0)">INFO (Carpet): Thread 0 runs on 24 cores: 0-23</div></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div style="font-family:arial,helvetica,sans-serif">Thanks!</div><div style="font-family:arial,helvetica,sans-serif">Geraint</div></div></div></blockquote><div><br></div></span><div>Glad to hear it's working!</div><div><br></div><div>The last line indicates that the threads are not being pinned to specific cores. Usually, we recommend to pin them. You should be able to do this by activating thorns hwloc and SystemTopology in your parameter file.</div><div><br></div><div>Note to others: with a change in the last few months, some of the functionality of hwloc was moved into SystemTopology, which is not automatically activated. If you want to get automatic thread pinning, you now need to edit your parameter files to explicitly activate SystemTopology in addition to hwloc.</div><div><br></div><div>Erik, could SystemTopology be automatically activated somehow?</div></div></div></blockquote><div><br></div><div>Ian</div><div><br></div><div>Yes, the way to do this is to have another thorn (that is already activated) optionally require this thorn. Obvious choices are the flesh, or Carpet (if you don't care about PUGH users).</div><div><br></div><div>-erik</div></div><div><br></div>-- <br><div class="gmail_signature">Erik Schnetter <<a href="mailto:schnetter@cct.lsu.edu" target="_blank">schnetter@cct.lsu.edu</a>><br><a href="http://www.perimeterinstitute.ca/personal/eschnetter/" target="_blank">http://www.perimeterinstitute.ca/personal/eschnetter/</a></div>
</div></div>