<html>#2482: Inconsistency in Simfactory submission scripts
<table style='border-spacing: 1ex 0pt; '>
<tr><td style='text-align:right'> Reporter:</td><td>Steven R. Brandt</td></tr>
<tr><td style='text-align:right'> Status:</td><td>new</td></tr>
<tr><td style='text-align:right'>Milestone:</td><td></td></tr>
<tr><td style='text-align:right'> Version:</td><td></td></tr>
<tr><td style='text-align:right'> Type:</td><td>bug</td></tr>
<tr><td style='text-align:right'> Priority:</td><td>major</td></tr>
<tr><td style='text-align:right'>Component:</td><td>SimFactory</td></tr>
</table>
<p>Comment (by Steven R. Brandt):</p>
<p>I’m a little unclear about what is different about the clusters. In particular, I’d be interested in knowing if the behavior of this script is different. I expect that Slurm sets its environment variables the same way everywhere and that mpirun (I used mpich) will interpret those variables in a consistent way. The script below prints a hostname 8 times, 4 from one node and 4 from another. Does this basic behavior change by cluster? </p>
<div class="codehilite language-bash"><pre><span></span><span class="ch">#!/bin/bash</span>
<span class="c1">#SBATCH -N 2 -n 6</span>
<span class="c1">#SBATCH --partition=checkpt</span>
<span class="c1">#SBATCH --ntasks-per-node=4</span>
env <span class="p">|</span> grep SLURM > slurm-env.txt
<span class="c1"># SLURM_TASKS_PER_NODE has format NUM1(xNUM2)</span>
<span class="c1"># SLURM_NTASKS is the product of NUM1 and NUM2</span>
<span class="c1"># SLURM_NNODES is NUM2</span>
<span class="c1"># SLURM_JOB_NUM_NODES is NUM2</span>
<span class="nv">TASKS_PER_NODE</span><span class="o">=</span><span class="k">$(</span><span class="nb">echo</span> <span class="si">${</span><span class="nv">SLURM_TASKS_PER_NODE</span><span class="si">}</span> <span class="p">|</span> cut -f1 -d<span class="se">\(</span><span class="k">)</span>
<span class="nb">echo</span> <span class="s2">"SLURM_TASKS_PER_NODE=</span><span class="nv">$SLURM_TASKS_PER_NODE</span><span class="s2">"</span>
<span class="nb">echo</span> <span class="s2">"TASKS_PER_NODE=</span><span class="nv">$TASKS_PER_NODE</span><span class="s2">"</span>
<span class="nb">echo</span> <span class="s2">"SLURM_NTASKS=</span><span class="nv">$SLURM_NTASKS</span><span class="s2">"</span>
<span class="nb">echo</span> <span class="s2">"SLURM_JOB_NUM_NODES=</span><span class="nv">$SLURM_JOB_NUM_NODES</span><span class="s2">"</span>
mpirun hostname
</pre></div>
<p></p>
<p>--<br/>
Ticket URL: <a href='https://bitbucket.org/einsteintoolkit/tickets/issues/2482/inconsistency-in-simfactory-submission'>https://bitbucket.org/einsteintoolkit/tickets/issues/2482/inconsistency-in-simfactory-submission</a></p>
</html>