<html>#2281: OpenMPI in tutorial containers binds multiple user's code to the same cores
<table style='border-spacing: 1ex 0pt; '>
<tr><td style='text-align:right'> Reporter:</td><td>Roland Haas</td></tr>
<tr><td style='text-align:right'> Status:</td><td>new</td></tr>
<tr><td style='text-align:right'>Milestone:</td><td></td></tr>
<tr><td style='text-align:right'> Version:</td><td></td></tr>
<tr><td style='text-align:right'> Type:</td><td>bug</td></tr>
<tr><td style='text-align:right'> Priority:</td><td>major</td></tr>
<tr><td style='text-align:right'>Component:</td><td>EinsteinToolkit Virtual Machine</td></tr>
</table>
<p>OpenMPI by default binds MPI ranks to specific cores and on a shared system like the tutorial server it binds multiple users' MPI ranks to the same core. Eg</p>
<div class="codehilite language-plaintext"><pre><span></span>16625 ubuntu 20 0 7588 1012 928 R 50.0 0.0 0:38.28 awk
16670 ubuntu 20 0 7588 888 804 R 50.0 0.0 0:31.65 awk
16671 ubuntu 20 0 7588 968 884 R 50.0 0.0 0:31.66 awk
16624 ubuntu 20 0 7588 1064 980 R 43.8 0.0 0:38.32 awk
</pre></div>
<p>when running (in two logins in jupyter notebooks)</p>
<div class="codehilite language-shell"><pre><span></span>!mpirun -n <span class="m">2</span> /usr/bin/awk <span class="s1">'BEGIN{while(1);}'</span> /dev/null
</pre></div>
<p>A fix is to add the following environment variable to the tutorial notebook:</p>
<div class="codehilite language-shell"><pre><span></span><span class="nv">OMPI_MCA_hwloc_base_binding_policy</span><span class="o">=</span>none
</pre></div>
<p></p>
<p>--<br/>
Ticket URL: <a href='https://bitbucket.org/einsteintoolkit/tickets/issues/2281/openmpi-in-tutorial-containers-binds'>https://bitbucket.org/einsteintoolkit/tickets/issues/2281/openmpi-in-tutorial-containers-binds</a></p>
</html>