[Users] changing CarpetRegrid2 parameters for ETK Maxwell + mercurial Carpet?

Thu Nov 17 11:33:30 CST 2011

Hi all.

I followed Erik's suggestion with the extra diagnostic information. The
resulting screen output is about 200 MB in size, but zips down to 20MB.

I've placed it on Goddard's "webdrive" for anyone to download. Here's the
link:

https://webdrive.gsfc.nasa.gov/shortauth/bernard.j.kelly/jvXqUYC

You'll need the temporary username (beanykelly) and password
(helpmercurial) to access it.

All suggestions (especially the blindingly obvious) are appreciated.

Thanks, Bernard

---------------------------------------------------------------------------
-- 
Bernard Kelly -- CRESST Research Associate, NASA/GSFC

Phone: +1 (301) 286-7243
E-Mail: bernard.j.kelly at nasa.gov
Web: 
http://science.gsfc.nasa.gov/sed/index.cfm?fuseAction=people.jumpBio&iphone
bookid=13052
---------------------------------------------------------------------------
--

On 11/10/11 8:18 PM, "Erik Schnetter" <schnetter at cct.lsu.edu> wrote:

>Could you run with Carpet::veryverbose=yes,
>CarpetLib::output_bboxes=yes, and make the resulting stdout/stderr
>available somehow? It will balloon in size compared to what you have
>now.
>
>-erik
>
>On Thu, Nov 10, 2011 at 4:49 PM, Kelly, Bernard J.
>(GSFC-660.0)[UNIVERSITY OF MARYLAND BALTIMORE COUNTY]
><bernard.j.kelly at nasa.gov> wrote:
>> Hi all.
>> At Erik's suggestion (off-list), I resolved some checkpoint recovery
>>issues
>> by moving from "Curie" to the new ETK release. However, my old grid
>> specifications seem no longer to work.
>> The parameters are for a 2:1 mass-ratio BHB evolution. I've used three
>> "centres" -- one each for the punctures, and one centred at the origin.
>> Below is what I used to have in CarpetRegrid2, which worked with the
>>last
>> "Curie" release of ETK (the full grid is [-2048,2048] in size).
>> -------------------------------------------------------------------
>> # Parameters of thorn CarpetRegrid2 (implementing CarpetRegrid2)
>> CarpetRegrid2::freeze_unaligned_levels = "yes"
>> CarpetRegrid2::num_centres             = 3
>> CarpetRegrid2::num_levels_1            = 12
>> CarpetRegrid2::position_x_1            = -3.3333333333333333333
>> CarpetRegrid2::movement_threshold_1    = 0.10
>> CarpetRegrid2::radius_1[5]             = 40
>> CarpetRegrid2::radius_1[6]             = 20
>> CarpetRegrid2::radius_1[7]             = 10
>> CarpetRegrid2::radius_1[8]             = 5
>> CarpetRegrid2::radius_1[9]             = 2.5
>> CarpetRegrid2::radius_1[10]            = 1.25
>> CarpetRegrid2::radius_1[11]            = 0.625
>> CarpetRegrid2::num_levels_2            = 13
>> CarpetRegrid2::position_x_2            =  6.6666666666666666667
>> CarpetRegrid2::movement_threshold_2    = 0.05
>> CarpetRegrid2::radius_2[5]             = 40
>> CarpetRegrid2::radius_2[6]             = 20
>> CarpetRegrid2::radius_2[7]             = 10
>> CarpetRegrid2::radius_2[8]             = 5
>> CarpetRegrid2::radius_2[9]             = 2.5
>> CarpetRegrid2::radius_2[10]            = 1.25
>> CarpetRegrid2::radius_2[11]            = 0.625
>> CarpetRegrid2::radius_2[12]            = 0.3125
>> CarpetRegrid2::num_levels_3            = 6
>> CarpetRegrid2::radius_3[1]             = 1024
>> CarpetRegrid2::radius_3[2]             = 512
>> CarpetRegrid2::radius_3[3]             = 256
>> CarpetRegrid2::radius_3[4]             = 160
>> CarpetRegrid2::radius_3[5]             = 96
>> CarpetRegrid2::regrid_every            = 60
>> CarpetRegrid2::verbose                 = "yes"
>> -------------------------------------------------------------------
>> This doesn't work now, however. Current ETK complains that some entries
>>in
>> radius_1 and radius_2 are unspecified, and it sets them to zero with
>> catastrophic results.
>> To resolve this, I am now explicitly setting the missing levels:
>> CarpetRegrid2::radius_1[1]             = 1024
>> CarpetRegrid2::radius_1[2]             = 512
>> CarpetRegrid2::radius_1[3]             = 256
>> CarpetRegrid2::radius_1[4]             = 160
>> CarpetRegrid2::radius_2[1]             = 1024
>> CarpetRegrid2::radius_2[2]             = 512
>> CarpetRegrid2::radius_2[3]             = 256
>> CarpetRegrid2::radius_2[4]             = 160
>> This runs, but complains every time step about mismatched "simulated
>>domain
>> volume" and "reduction weight sum":
>> 
>>-------------------------------------------------------------------------
>>---------
>> INFO (CarpetReduce): Simulation domain volume: 442368
>> INFO (CarpetReduce): Reduction weight sum:     442367.999999072
>> WARNING level 1 in thorn CarpetReduce processor 0 host
>> r186i2n12.p4.nas.nasa.gov
>>   (line 120 of
>> 
>>/home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetReduce/mask_test
>>.c):
>>   -> Simulation domain volume and reduction weight sum differ
>> 
>>-------------------------------------------------------------------------
>>---------
>> After a short evolution time (7.25 M, or more than 1700 time steps),
>>the run
>> crashes. First there are several messages of the form:
>> 
>>-------------------------------------------------------------------------
>>---------
>> INFO (CarpetReduce): Simulation domain volume: 442368
>> INFO (CarpetReduce): Reduction weight sum:     442367.999991311
>> INFO (CarpetRegrid2): Centre 1 is at position
>> [-3.29831,-0.331478,7.81462e-15] with 12 levels
>> INFO (CarpetRegrid2): Centre 2 is at position
>>[6.58787,0.867484,8.98107e-15]
>> with 13 levels
>> INFO (CarpetRegrid2): Centre 3 is not active
>> INFO (CarpetRegrid2): Regridding
>> INFO (CarpetRegrid2): Regridding levels 10 and up
>> WARNING level 1 in thorn CarpetLib processor 58 host
>> r186i3n7.p4.nas.nasa.gov
>>   (line 182 of
>> /home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetLib/dh.cc):
>> [...]
>>   ->
>> /home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetLib/dh.cc:948:
>>    [ml=0 rl=10 c=59] The following grid structure consistency check
>>failed:
>>    Synchronisation and boundary prolongation: All points must have been
>> received
>>    needrecv.empty()
>> [...]
>> WARNING level 0 in thorn CarpetLib processor 94 host
>> r186i3n12.p4.nas.nasa.gov
>> 12676   (line 1973 of
>> /home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetLib/dh.cc):
>> 12677   -> The grid structure is inconsistent.  It is impossible to
>> continue.
>> --------------------------------------
>> I'm attaching my current parameter file in full. I'm reluctant to post
>>the
>> entire STDOUT/STDERR, since it's around a MB in size.
>> Any ideas would be appreciated. I would turn to the copious
>>documentation on
>> CarpetRegrid2, but as noted before, there is none.
>> Thanks, Bernard
>> 
>>-------------------------------------------------------------------------
>>----
>> Bernard Kelly -- CRESST Research Associate, NASA/GSFC
>>
>> Phone: +1 (301) 286-7243
>> E-Mail: bernard.j.kelly at nasa.gov
>> Web: 
>>http://science.gsfc.nasa.gov/sed/index.cfm?fuseAction=people.jumpBio&ipho
>>nebookid=13052
>> 
>>-------------------------------------------------------------------------
>>----
>>
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>
>>
>
>
>
>-- 
>Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/