Thanks, Roland, for looking at this! Very good point, that seems likely to be the culprit. I’ll give that a go and hopefully it’ll solve the issue.
Best, Deborah
-------------------------------------- Deborah Ferguson Post Doc University of Texas at Austin
From: Roland Haas rhaas@illinois.edu Date: Thursday, March 25, 2021 at 9:07 AM To: Ferguson, Deborah deborah.ferguson@austin.utexas.edu, users@einsteintoolkit.org users@einsteintoolkit.org Subject: Re: [Users] The following grid structure consistency check failed: Synchronisation and boundary prolongation Hello Deborah,
looking at your parfile you are regridding every 288 but the coarsest level steps at every 64 and with the 2 freezing options you will have the regrid only happen at the smallest common multiple which would be 9*288.
You may want to try setting this to 256 instead.
Yours, Roland
I submitted a simulation on Stampede that ran for several days (1024.4 M) before failing with multiple errors of the form:
[ml=0 rl=10 c=13] The following grid structure consistency check failed:
Synchronisation and boundary prolongation: All points must have been received
needrecv.empty()
Does anyone know why this may have happened after running for so long? I’ve attached for the most recent output, the .err file, the .out file, the parameter file, grid.asc, and the ShiftTracker files. Let me know if anything else would be useful.
Thanks for any thoughts you might have!
Best, Deborah
Deborah Ferguson Post Doc University of Texas at Austin
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .