<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Thanks, Roland, for looking at this! Very good point, that seems likely to be the culprit. I’ll give that a go and hopefully it’ll solve the issue.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best,<o:p></o:p></p>
<p class="MsoNormal">Deborah<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="color:black">--------------------------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Deborah Ferguson<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Post Doc<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black">University of Texas at Austin </span>
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Roland Haas <rhaas@illinois.edu><br>
<b>Date: </b>Thursday, March 25, 2021 at 9:07 AM<br>
<b>To: </b>Ferguson, Deborah <deborah.ferguson@austin.utexas.edu>, users@einsteintoolkit.org <users@einsteintoolkit.org><br>
<b>Subject: </b>Re: [Users] The following grid structure consistency check failed: Synchronisation and boundary prolongation<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Hello Deborah,<br>
<br>
looking at your parfile you are regridding every 288 but the coarsest<br>
level steps at every 64 and with the 2 freezing options you will have<br>
the regrid only happen at the smallest common multiple which would be<br>
9*288.<br>
<br>
You may want to try setting this to 256 instead.<br>
<br>
Yours,<br>
Roland<br>
<br>
> I submitted a simulation on Stampede that ran for several days (1024.4 M) before failing with multiple errors of the form:<br>
> <br>
> <br>
> [ml=0 rl=10 c=13] The following grid structure consistency check failed:<br>
> <br>
> Synchronisation and boundary prolongation: All points must have been received<br>
> <br>
> needrecv.empty()<br>
> <br>
> <br>
> Does anyone know why this may have happened after running for so long? I’ve attached for the most recent output, the .err file, the .out file, the parameter file, grid.asc, and the ShiftTracker files. Let me know if anything else would be useful.<br>
> <br>
> Thanks for any thoughts you might have!<br>
> <br>
> Best,<br>
> Deborah<br>
> <br>
> --------------------------------------<br>
> Deborah Ferguson<br>
> Post Doc<br>
> University of Texas at Austin<br>
<br>
<br>
-- <br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://pgp.mit.edu">http://pgp.mit.edu</a> .<o:p></o:p></p>
</div>
</div>
</body>
</html>