<html>#2483: Simfactory job parameters are not consistent
<table style='border-spacing: 1ex 0pt; '>
<tr><td style='text-align:right'> Reporter:</td><td>Steven R. Brandt</td></tr>
<tr><td style='text-align:right'> Status:</td><td>new</td></tr>
<tr><td style='text-align:right'>Milestone:</td><td></td></tr>
<tr><td style='text-align:right'> Version:</td><td></td></tr>
<tr><td style='text-align:right'> Type:</td><td>bug</td></tr>
<tr><td style='text-align:right'> Priority:</td><td>major</td></tr>
<tr><td style='text-align:right'>Component:</td><td>SimFactory</td></tr>
</table>
<p>According to simlib.py:</p>
<p>NUM_THREADS = threads per mpi proc (thread/mpi_proc)</p>
<p>PPN is supposed to be the number of processors, or cores requested from the scheduler per node. (core/node)</p>
<p>PPN_USED is supposed to be the number of cores actually used per node. (core/node)</p>
<p>NUM_SMT is supposed to be threads per core, and has a value of either 1 or 2 on all machines. (thread/core)</p>
<p>Thus</p>
<p>NODE_PROCS := PPNUSED * NUM_SMT/ NUM_THREADS</p>
<p>This follows since: NODE_PROCS = (cores/node)*(threads/core)/(threads/mpi proc) = mpi procs/node</p>
<p>Now here’s the problem.</p>
<p>NUM_PROCS = PROCS / NUM_THREADS</p>
<p>Now both --procs and --cores are two options for the same thing in simfactory. Thus “procs” is “processors” and “num_procs” is “number of processes.” That’s confusing, but that’s not the problem this ticket is about.</p>
<p>NUM PROCS is supposed to be the number of mpi processes. However, since --procs and --cores are the same thing:</p>
<p>NUM_PROCS = CORES / NUM_THREADS</p>
<p>= cores / (threads / mpi proc)</p>
<p>This is inconsistent. One would expect:</p>
<p>NUM_PROCS = NUM_SMT*CORES/NUM_THREADS</p>
<p> = (threads/core)*cores/(threads/mpi proc).</p>
<p>What if we define NUM_THREADS as cores/mpi proc? Well, apart from being confusing, that makes the NODE_PROCS calculation wrong.</p>
<p>So, unless I’m missing something, these parameters are not consistent, regardless of how you define them. They only work if NUM_SMT is one and cores and threads are interchangeable.</p>
<p>Is that always true?</p>
<p>The following machines have: max-num-smt = 2 are bethe, cori, philip, and supermucng. Looking at simlib.py, this parameter is <em>not</em> accessed! Instead, simlib.py only attempts to get ‘num-smt’, a parameter no ini file ever sets. Thus, the NUM_SMT is, essentially, always 1.</p>
<p>What to do?</p>
<p>My suggestion is that the definition of NUM_PROCS be ammended to be</p>
<p>NUM_PROCS = CORES * NUM_SMT / NUM_THREADS</p>
<p> so that cores*(threads/core)/(threads/mpi
<p>--<br/>
Ticket URL: <a href='https://bitbucket.org/einsteintoolkit/tickets/issues/2483/simfactory-job-parameters-are-not'>https://bitbucket.org/einsteintoolkit/tickets/issues/2483/simfactory-job-parameters-are-not</a></p>
</html>