<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Cool. If you have anyone else using your computer, you can submit
your ini/cfg/runscript/submitscript to Simfactory. Thanks!<br>
</p>
<div class="moz-cite-prefix">On 5/10/2024 5:41 AM, Shamim Haque
1910511 wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAHrQTHXeg-aVti11N6L2J_4C4wbOPHKX=DoZGca=ghJyp-a02g@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div class="gmail_default" style="color:#000000">Hello Steve,</div>
<div class="gmail_default" style="color:#000000"><br>
</div>
<div class="gmail_default" style="color:#000000">My Runscript is
now working fine after removing -npernode. Thanks for the
help.</div>
<div class="gmail_default" style="color:#000000"><br>
</div>
<div class="gmail_default" style="color:#000000">Regards</div>
<div>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><font color="#666666">Shamim
Haque</font></div>
<div dir="ltr"><font color="#666666">Senior
Research Fellow (SRF)<br>
</font>
<div><font color="#666666">Department of
Physics</font></div>
<div><font color="#666666">IISER Bhopal</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<div hspace="streak-pt-mark" style="max-height:1px"><img alt=""
style="width:0px;max-height:0px;overflow:hidden"
src="https://mailfoogae.appspot.com/t?sender=ac2hhbWltc0BpaXNlcmIuYWMuaW4%3D&type=zerocontent&guid=74f5099b-ca29-4bc6-8a5e-1d256fbb22e0"
moz-do-not-send="true"><font size="1" color="#ffffff">ᐧ</font></div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, May 2, 2024 at 1:05 AM
Shamim Haque 1910511 <<a href="mailto:shamims@iiserb.ac.in"
moz-do-not-send="true" class="moz-txt-link-freetext">shamims@iiserb.ac.in</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_default" style="color:rgb(0,0,0)">Hello
Steve,</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">Thanks
for pointing this out. I'll try to write a fresh runscipt
by looking at example runscripts.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default"><font color="#a64d79">Since
you're using slurm, MPI should be smart enough that you
don't need to pass -n, -npernode,<br>
</font></div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">I don't
need to pass -n as well? I can see -n @NUM_PROCS@ in the
SBATCH runscripts that uses openmpi (example - graham,
expanse). Can you please explain a little bit about what
should be the simplest/safest mpi execution command to
start with? And how can we build it further to optimise it
more?</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default"><font color="#a64d79">How did you
get a Runscript and Submitscript for this machine. Did
you create yourself?</font><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">My first
attempt was at an HPC at my home institute IISER Bhopal,
which had PBS. Then, I installed ETK in the NSM facility
(India), which has SLURM. I changed most of the stuff in
machinefile and submitscipt as per SLRUM (by looking at
available example scripts) and copied the runscript from
PBS, which was working fine. </div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">The
current HPC also has SLURM, so I copied all the scripts
from the NSM facility. It always worked alright, so I was
never quite sceptical about the runscipt, especially
because the error has only shown up in the current HPC
only and quite occasionally.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">Now, I
can see from example runscripts that mpi execution
commands for SBATCH look very different from PBS ones.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">Regards</div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><font color="#666666">Shamim
Haque</font></div>
<div dir="ltr"><font color="#666666">Senior
Research Fellow (SRF)<br>
</font>
<div><font color="#666666">Department of
Physics</font></div>
<div><font color="#666666">IISER Bhopal</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<div hspace="streak-pt-mark" style="max-height:1px"><img
alt=""
style="width: 0px; max-height: 0px; overflow: hidden;"
src="https://mailfoogae.appspot.com/t?sender=ac2hhbWltc0BpaXNlcmIuYWMuaW4%3D&type=zerocontent&guid=ee6109f4-a5db-4cb7-b4fc-e71c32d45e20"
moz-do-not-send="true"><font size="1" color="#ffffff">ᐧ</font></div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, May 1, 2024 at
11:05 PM Steven Brandt <<a
href="mailto:sbrandt@cct.lsu.edu" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">sbrandt@cct.lsu.edu</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello Shamim,</p>
<p>The error says that you're calling MPI with the wrong
parameters, specificall -npernode. Since you're using
slurm, MPI should be smart enough that you don't need
to pass -n, -npernode, How did you get a Runscript
and Submitscript for this machine. Did you create
yourself?</p>
<p>--Steve<br>
</p>
<div>On 5/1/2024 6:54 AM, Shamim Haque 1910511 wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default" style="color:rgb(0,0,0)">Hi
all,</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">I
am attempting ETK installation in KALINGA Cluster
at NISER, India. This cluster has 40 procs per
node and SLURM workload manager.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">I
compiled ETK with gcc-7.5 and openmpi-4.0.5
(attached the machinefile, optionlist,
submitscript and runscript). The installation is
mostly alright, as I can run parfiles for test TOV
and BNS mergers.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">I
tried to run a simulation with procs=160 (nodes 4)
and num-threads=1 but landed with this error
(error file also attached):</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default"><i><font color="#a64d79">+
mpiexec -n 640 -npernode 40.0
/home/kamal/simulations/dx25_r500_rg7_t30_p640-1_2/SIMFACTORY/exe/cactus_sim
-L 3
/home/kamal/simulations/dx25_r500_rg7_t30_p640-1_2/output-0000/eos20_dx25_r500_rg7.par<br>
----------------------------------------------------------------------------<br>
Open MPI has detected that a parameter given
to a command line<br>
option does not match the expected format:<br>
<br>
Option: npernode<br>
Param: 40.0<br>
<br>
This is frequently caused by omitting to
provide the parameter<br>
to an option that requires one. Please check
the command line and try again.<br>
----------------------------------------------------------------------------</font><br>
</i></div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">Strangely,
this error is not at all regular. Mostly, the
error won't appear, and the simulation works just
fine (with no changes being made in the scripts or
simfactory command). In fact, this exact
simulation has worked fine before. Since I am
unable to find the source of this issue, I am also
unable to recreate the error on my own. But it
does kick in occasionally.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">My
command for mpi execution in runscript looks like
this:</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default"><i><font color="#3d85c6">time
mpiexec -n @NUM_PROCS@ -npernode @(@PPN_USED@
/ @NUM_THREADS@)@ @EXECUTABLE@ -L 3 @PARFILE@</font></i><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">If
I replace <i style="color:rgb(34,34,34)"><font
color="#3d85c6"> @(@PPN_USED@ /
@NUM_THREADS@)@ </font></i>with a desired
value, then the script always works. My simfactory
command looks like this:</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default"><i><font color="#3d85c6">./simfactory/bin/sim
create-submit dx25_r500_rg7_t30_p640-1_2
--parfile=par-smooth/scale_test/eos20_dx25_r500_rg7.par
--queue=large1 --procs=640 --num-threads=1
--walltime=00:45:00<br>
</font></i></div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">I
am unable to understand how to solve this issue.
Any help with this issue is appreciated. Please
let me know if you need more information. Thank
you.</div>
<div class="gmail_default" style="color:rgb(0,0,0)"><br>
</div>
<div class="gmail_default" style="color:rgb(0,0,0)">Regards</div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><font color="#666666">Shamim
Haque</font></div>
<div dir="ltr"><font color="#666666">Senior
Research Fellow (SRF)<br>
</font>
<div><font color="#666666">Department
of Physics</font></div>
<div><font color="#666666">IISER
Bhopal</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div hspace="streak-pt-mark" style="max-height:1px"><img
alt=""
style="width: 0px; max-height: 0px; overflow: hidden;"
src="https://mailfoogae.appspot.com/t?sender=ac2hhbWltc0BpaXNlcmIuYWMuaW4%3D&type=zerocontent&guid=e2f8d473-349d-4337-ba32-7ab03cfbfabd"
moz-do-not-send="true"><font size="1"
color="#ffffff">ᐧ</font></div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Users mailing list
<a href="mailto:Users@einsteintoolkit.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">Users@einsteintoolkit.org</a>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://lists.einsteintoolkit.org/mailman/listinfo/users</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@einsteintoolkit.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">Users@einsteintoolkit.org</a><br>
<a
href="http://lists.einsteintoolkit.org/mailman/listinfo/users"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>