<html>#2194: Memory increase during regridding
<table style='border-spacing: 1ex 0pt; '>
<tr><td style='text-align:right'> Reporter:</td><td>Wolfgang Kastaun</td></tr>
<tr><td style='text-align:right'> Status:</td><td>open</td></tr>
<tr><td style='text-align:right'>Milestone:</td><td>ET_2019_02</td></tr>
<tr><td style='text-align:right'> Version:</td><td>development version</td></tr>
<tr><td style='text-align:right'> Type:</td><td>bug</td></tr>
<tr><td style='text-align:right'> Priority:</td><td>minor</td></tr>
<tr><td style='text-align:right'>Component:</td><td>Other</td></tr>
</table>
<p>Comment (by Roland Haas):</p>
<p>I gave ML_QC0.par a test run using 6 MPI ranks. I used either the Proca release code or Proca with Carpet set to rhaas/commstate_free_buffers instead. </p>
<p>With this I observe a change in allocated memory at the regrid in iteration 2048.</p>
<p><img alt="" src="https://bitbucket.org/repo/8zBLME5/images/4070923915-ML_QC0-process_memory_MB.png" />
In the plot there are two fields of systemstatistics' output shown. <code>max_rss</code> which is the amount of memory currently claimed from the OS and which includes memory already free()ed but not yet returned to the OS. <code>uordblks</code> instead (see <code>man mallinfo</code>) is the amount of memory currently allocated via malloc (and new etc). The plot shows that applying the patch does (a) reduce the amount of memory malloc’ed on the rank with the largest such value decreases which is what should happen once one frees communication buffers and (b) that this apparently is enough to convince free() to return memory to the OS (drop in RSS). (b) is not something one can guarantee since memory fragmentation can prevent this since memory must be returned to the OS as hole pages (or worse as a whole chunk if mmap was not used to request it but setbrk).</p>
<p>I will try and see if there are differences i n ML_QC3 as well and maybe if there are differences late on in the ML_QC0 run as well.</p>
<p>I have not tested <span class="ap-mention" data-atlassian-id="557058:4b0c8315-696b-4d2e-92da-2da5fea9e8d4">@Gabriele Bozzola</span> 's files (yet) because they require an actual run with a number of nodes on Blue Waters. </p>
<p><span class="ap-mention" data-atlassian-id="557058:65c5f18c-c3ad-4126-bb2b-be0e766b31ef">@Miguel Zilhão</span> do you see a drop in rss after it=2048 in your test run or no change at all? Note that <code>rss</code> is not really controllable by user code (since it depends on when <code>free()</code> returns memory to the OS) and that all one can really do is control <code>uordblks</code> by calling <code>free()</code>.</p>
<p>--<br/>
Ticket URL: <a href='https://bitbucket.org/einsteintoolkit/tickets/issues/2194/memory-increase-during-regridding'>https://bitbucket.org/einsteintoolkit/tickets/issues/2194/memory-increase-during-regridding</a></p>
</html>