[Users] More Bizarre Scheduling Behavior in a Carpet Run (Bug?!)

[Zach Etienne]

Hi Ian,

> Can you experiment to see what happens if you schedule your copying
routine in POSTRESTRICT?  And in POSTREGRID?

Yes. I just performed the following experiments:
1) Schedule myadmbaselapse=alp function in POSTRESTRICT (GLOBAL,
LOOP-LOCAL), every 64 iterations
2) Schedule myadmbaselapse=alp function in POSTREGRID (GLOBAL, LOOP-LOCAL),
every 64 iterations
3) Schedule myadmbaselapse=alp function in both POSTRESTRICT and POSTRESTRICT
(both GLOBAL, LOOP-LOCAL), every 64 iterations

(1) and (3) resulted in no undefined values! In (2), undefined values
remained.

There remains the mystery of why calling my function in CCTK_ANALYSIS at
*every* iteration results in no undefined values...

-Zach

*     *     *
Zachariah Etienne
Assistant Professor of Mathematics
West Virginia University

On Tue, Jun 2, 2015 at 12:30 PM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:

>
> On 2 Jun 2015, at 18:19, Zach Etienne <zachetie at gmail.com> wrote:
>
> Thanks for getting back to me, Ian, and thanks for agreeing this does not
> appear to be a trivial problem. (I've been battling it on and off for days.)
>
> > That tells you that alp is defined at the point it is output, but it
> doesn't tell you what its state was at the time myadmbaselapse used it.
>
> You are absolutely right. I just added a printf() statement outputting
> ADMBase::alp at iteration 192, at all points on the z=0 plane, inside the
> ADMBaseMcLachlanTester loop that sets myadmbaselapse=alp. Then I re-did
> the run with ADMBaseMcLachlanTester. After analyzing the output, I can
> confirm that there are no undefined (nan) values at iteration 192, and
> alp values from this printf() statement lie in the very reasonable range of
> 0.25 < alp < 0.995 for all points. Again, this is inconsistent with the
> data I observe in the myadmbaselapse IOASCII 2D file in the z=0 plane,
> where many undefined values are seen.
>
> Thus I conclude that something bad is happening to myadmbaselapse after
> being set at CCTK_ANALYSIS in GLOBAL,LOOP-LOCAL mode at iteration 192, but
> before the data are written to file at the same iteration, causing
> perfectly reasonable values to suddenly become undefined. For some reason,
> the same behavior is not observed in ADMBase::alp...
>
>
> Could it be that restriction is happening after the variable is set in
> ANALYSIS?
>
> Can you experiment to see what happens if you schedule your copying
> routine in POSTRESTRICT?  And in POSTREGRID?
>
>
>
> -Zach
>
> *     *     *
> Zachariah Etienne
> Assistant Professor of Mathematics
> West Virginia University
>
> On Tue, Jun 2, 2015 at 11:46 AM, Ian Hinder <ian.hinder at aei.mpg.de> wrote:
>
>>
>> On 2 Jun 2015, at 17:40, Zach Etienne <zachetie at gmail.com> wrote:
>>
>> > Hi Frank,
>> >
>> > Thanks for your feedback.
>> > > You use the ADMBase variables, right?
>> > My ADMBaseMcLachlanTester thorn (
>> math.wvu.edu/~zetienne/ADMBaseMcLachlanTester.tar.gz) reproduces this
>> issue, and uses *only* ADMBase::alp as input. I would encourage everyone
>> interested to take a look at this thorn, as
>> > 1) there's nothing complicated about the ADMBaseMcLachlanTester thorn
>> at all (67 lines of code, including ccl files, includes, and whitespace)
>> > 2) the included parfiles run on a single desktop computer needing only
>> 5GB of RAM
>> > 3) it takes only ~15 mins to evolve to timestep 192
>> >
>> > > Could you output the variables your analysis depends on, and see if
>> the nan values come from there?
>> > Absolutely! I just performed a run with ADMBaseMcLachlanTester, setting
>> myadmbaselapse to alp every 64 iterations, and confirmed that there are
>> *no* undefined (nan) values in the (IOASCII 2D output of) ADMBase::alp (at
>> any iteration, upto and including iteration 192), while again, there are a
>> very large number of undefined values in myadmbaselapse at iteration 192...
>> >
>> > So myadmbaselapse is not undefined because ADMBase::alp is undefined.
>>
>> That tells you that alp is defined at the point it is output, but it
>> doesn't tell you what its state was at the time myadmbaselapse used it.  I
>> think you can add a call to CCTK_OutputVarAs or something in the routine,
>> and you will get an output at that point.
>> http://einsteintoolkit.org/documentation/ReferenceManual/ReferenceManualch2.html#x4-139000A2
>>
>> So maybe something like
>>
>> CCTK_OutputVarAs(cctkGH, "ADMBaseMcLachlanTester::myadmbaselapse",
>> "myadmbaselapse_when_set")
>>
>> and you should get your 2D output at the point the variable is set.  You
>> could do the same for alp, to see what it is being set from.  Or you could
>> use printf.
>>
>> --
>> Ian Hinder
>> http://members.aei.mpg.de/ianhin
>>
>>
>
> --
> Ian Hinder
> http://members.aei.mpg.de/ianhin
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20150602/eb431fa6/attachment.html