[Users] More Bizarre Scheduling Behavior in a Carpet Run (Bug?!)

[Ian Hinder]

On 2 Jun 2015, at 17:40, Zach Etienne <zachetie at gmail.com> wrote:

> Hi Frank,
> 
> Thanks for your feedback. 
> > You use the ADMBase variables, right? 
> My ADMBaseMcLachlanTester thorn (math.wvu.edu/~zetienne/ADMBaseMcLachlanTester.tar.gz) reproduces this issue, and uses *only* ADMBase::alp as input. I would encourage everyone interested to take a look at this thorn, as
> 1) there's nothing complicated about the ADMBaseMcLachlanTester thorn at all (67 lines of code, including ccl files, includes, and whitespace)
> 2) the included parfiles run on a single desktop computer needing only 5GB of RAM 
> 3) it takes only ~15 mins to evolve to timestep 192
> 
> > Could you output the variables your analysis depends on, and see if the nan values come from there?
> Absolutely! I just performed a run with ADMBaseMcLachlanTester, setting myadmbaselapse to alp every 64 iterations, and confirmed that there are *no* undefined (nan) values in the (IOASCII 2D output of) ADMBase::alp (at any iteration, upto and including iteration 192), while again, there are a very large number of undefined values in myadmbaselapse at iteration 192...
> 
> So myadmbaselapse is not undefined because ADMBase::alp is undefined.

That tells you that alp is defined at the point it is output, but it doesn't tell you what its state was at the time myadmbaselapse used it.  I think you can add a call to CCTK_OutputVarAs or something in the routine, and you will get an output at that point.  http://einsteintoolkit.org/documentation/ReferenceManual/ReferenceManualch2.html#x4-139000A2

So maybe something like

CCTK_OutputVarAs(cctkGH, "ADMBaseMcLachlanTester::myadmbaselapse", "myadmbaselapse_when_set")

and you should get your 2D output at the point the variable is set.  You could do the same for alp, to see what it is being set from.  Or you could use printf.

-- 
Ian Hinder
http://members.aei.mpg.de/ianhin