<html>#2288: Lean's boundary condition setting for BSSN constraints only fills in a single ghostzone
<table style='border-spacing: 1ex 0pt; '>
<tr><td style='text-align:right'> Reporter:</td><td>Zach Etienne</td></tr>
<tr><td style='text-align:right'> Status:</td><td>new</td></tr>
<tr><td style='text-align:right'>Milestone:</td><td></td></tr>
<tr><td style='text-align:right'> Version:</td><td></td></tr>
<tr><td style='text-align:right'> Type:</td><td>bug</td></tr>
<tr><td style='text-align:right'> Priority:</td><td>minor</td></tr>
<tr><td style='text-align:right'>Component:</td><td>EinsteinToolkit thorn</td></tr>
</table>
<p>Changes (by Zach Etienne):</p>
<p>In developing BaikalETK (a NRPy+-based BSSN thorn), I set up a schedule.ccl file that follows almost exactly the same pattern as Lean.</p>
<p>I noticed that when BaikalETK evaluated the constraints without initializing the variables to zero (as is done in Lean), I would get uninitialized on the boundaries **after ApplyBCs was called**. This confused me since BaikalETK, like Lean, calls for "flat" boundary conditions on the constraints. Sure enough when I performed the same run with Lean, most boundary points were filled with zeros (unlike BaikalETK, the constraints in Lean are initialized to zero). I spoke to Roland (Haas) about this, and he was also confused until he noticed that when BaikalETK/Lean calls Boundary_SelectVarForBC/Boundary_SelectGroupForBC <em>a la</em>:</p>
<p>Boundary_SelectVarForBC(cctkGH, CCTK_ALL_FACES, +1, -1, "BaikalETK::somegridfunction", "flat") ,</p>
<p>the "+1" indicates that only one ghost zone point should be filled in. You might then wonder what would happen if we just replaced that "+1" with "cctk_nghostzones[0]". Well, since this function in Lean & BaikalETK is called in "LEVEL" mode, "cctk_nghostzones[0]" does not exist (actually it compiles and gives a garbage value).</p>
<p>Update (Jun 16, 2020): The cctk_nghostzones[0] idea in the previous paragraph (combined with calling the function in LOCAL) mode results in issues when certain numbers of MPI processes are used (thanks to Tanmayee Gupte for pointing this out!) I ended up using Erik’s suggestion (run in LEVEL mode) below. It's currently implemented in BaikalETK (here's the schedule.ccl: <a data-is-external-link="true" href="https://nbviewer.jupyter.org/github/zachetienne/nrpytutorial/blob/master/Tutorial-BaikalETK.ipynb#scheduleccl" rel="nofollow">https://nbviewer.jupyter.org/github/zachetienne/nrpytutorial/blob/master/Tutorial-BaikalETK.ipynb#scheduleccl</a>) and seems to work great.</p>
<p>--<br/>
Ticket URL: <a href='https://bitbucket.org/einsteintoolkit/tickets/issues/2288/leans-boundary-condition-setting-for-bssn'>https://bitbucket.org/einsteintoolkit/tickets/issues/2288/leans-boundary-condition-setting-for-bssn</a></p>
</html>