[Users] changing CarpetRegrid2 parameters for ETK Maxwell + mercurial Carpet?

Erik Schnetter schnetter at cct.lsu.edu
Thu Nov 17 20:08:22 CST 2011


Bernard

It seems that the 8 components reporting a problem are all located at
the upper y boundary of level 10 of the refined region. To say more, I
will need to know which parts of these components remains undefined.

Can you add approximately the following code below line 948 (the
ASSERT_c statement) to Carpet/CarpetLib/src/dh.cc:

if (not needrecv_empty()) {
  cout << "r'l=" << rl << " c=" << c << " needrecv=" << needrecv << endl;
}

If this line does not show up in the output, then you need to add
"-roe" to the Cactus run-time flags. This will make your output
another 100 times larger (you will see 96 files). However, you may be
able to get by without any verbose, veryverbose, or output_bbox
settings.

-erik

On Thu, Nov 17, 2011 at 12:33 PM, Kelly, Bernard J.
(GSFC-660.0)[UNIVERSITY OF MARYLAND BALTIMORE COUNTY]
<bernard.j.kelly at nasa.gov> wrote:
> Hi all.
>
> I followed Erik's suggestion with the extra diagnostic information. The
> resulting screen output is about 200 MB in size, but zips down to 20MB.
>
> I've placed it on Goddard's "webdrive" for anyone to download. Here's the
> link:
>
> https://webdrive.gsfc.nasa.gov/shortauth/bernard.j.kelly/jvXqUYC
>
> You'll need the temporary username (beanykelly) and password
> (helpmercurial) to access it.
>
> All suggestions (especially the blindingly obvious) are appreciated.
>
> Thanks, Bernard
>
>
> ---------------------------------------------------------------------------
> --
> Bernard Kelly -- CRESST Research Associate, NASA/GSFC
>
> Phone: +1 (301) 286-7243
> E-Mail: bernard.j.kelly at nasa.gov
> Web:
> http://science.gsfc.nasa.gov/sed/index.cfm?fuseAction=people.jumpBio&iphone
> bookid=13052
> ---------------------------------------------------------------------------
> --
>
>
>
>
>
>
>
> On 11/10/11 8:18 PM, "Erik Schnetter" <schnetter at cct.lsu.edu> wrote:
>
>>Could you run with Carpet::veryverbose=yes,
>>CarpetLib::output_bboxes=yes, and make the resulting stdout/stderr
>>available somehow? It will balloon in size compared to what you have
>>now.
>>
>>-erik
>>
>>On Thu, Nov 10, 2011 at 4:49 PM, Kelly, Bernard J.
>>(GSFC-660.0)[UNIVERSITY OF MARYLAND BALTIMORE COUNTY]
>><bernard.j.kelly at nasa.gov> wrote:
>>> Hi all.
>>> At Erik's suggestion (off-list), I resolved some checkpoint recovery
>>>issues
>>> by moving from "Curie" to the new ETK release. However, my old grid
>>> specifications seem no longer to work.
>>> The parameters are for a 2:1 mass-ratio BHB evolution. I've used three
>>> "centres" -- one each for the punctures, and one centred at the origin.
>>> Below is what I used to have in CarpetRegrid2, which worked with the
>>>last
>>> "Curie" release of ETK (the full grid is [-2048,2048] in size).
>>> -------------------------------------------------------------------
>>> # Parameters of thorn CarpetRegrid2 (implementing CarpetRegrid2)
>>> CarpetRegrid2::freeze_unaligned_levels = "yes"
>>> CarpetRegrid2::num_centres             = 3
>>> CarpetRegrid2::num_levels_1            = 12
>>> CarpetRegrid2::position_x_1            = -3.3333333333333333333
>>> CarpetRegrid2::movement_threshold_1    = 0.10
>>> CarpetRegrid2::radius_1[5]             = 40
>>> CarpetRegrid2::radius_1[6]             = 20
>>> CarpetRegrid2::radius_1[7]             = 10
>>> CarpetRegrid2::radius_1[8]             = 5
>>> CarpetRegrid2::radius_1[9]             = 2.5
>>> CarpetRegrid2::radius_1[10]            = 1.25
>>> CarpetRegrid2::radius_1[11]            = 0.625
>>> CarpetRegrid2::num_levels_2            = 13
>>> CarpetRegrid2::position_x_2            =  6.6666666666666666667
>>> CarpetRegrid2::movement_threshold_2    = 0.05
>>> CarpetRegrid2::radius_2[5]             = 40
>>> CarpetRegrid2::radius_2[6]             = 20
>>> CarpetRegrid2::radius_2[7]             = 10
>>> CarpetRegrid2::radius_2[8]             = 5
>>> CarpetRegrid2::radius_2[9]             = 2.5
>>> CarpetRegrid2::radius_2[10]            = 1.25
>>> CarpetRegrid2::radius_2[11]            = 0.625
>>> CarpetRegrid2::radius_2[12]            = 0.3125
>>> CarpetRegrid2::num_levels_3            = 6
>>> CarpetRegrid2::radius_3[1]             = 1024
>>> CarpetRegrid2::radius_3[2]             = 512
>>> CarpetRegrid2::radius_3[3]             = 256
>>> CarpetRegrid2::radius_3[4]             = 160
>>> CarpetRegrid2::radius_3[5]             = 96
>>> CarpetRegrid2::regrid_every            = 60
>>> CarpetRegrid2::verbose                 = "yes"
>>> -------------------------------------------------------------------
>>> This doesn't work now, however. Current ETK complains that some entries
>>>in
>>> radius_1 and radius_2 are unspecified, and it sets them to zero with
>>> catastrophic results.
>>> To resolve this, I am now explicitly setting the missing levels:
>>> CarpetRegrid2::radius_1[1]             = 1024
>>> CarpetRegrid2::radius_1[2]             = 512
>>> CarpetRegrid2::radius_1[3]             = 256
>>> CarpetRegrid2::radius_1[4]             = 160
>>> CarpetRegrid2::radius_2[1]             = 1024
>>> CarpetRegrid2::radius_2[2]             = 512
>>> CarpetRegrid2::radius_2[3]             = 256
>>> CarpetRegrid2::radius_2[4]             = 160
>>> This runs, but complains every time step about mismatched "simulated
>>>domain
>>> volume" and "reduction weight sum":
>>>
>>>-------------------------------------------------------------------------
>>>---------
>>> INFO (CarpetReduce): Simulation domain volume: 442368
>>> INFO (CarpetReduce): Reduction weight sum:     442367.999999072
>>> WARNING level 1 in thorn CarpetReduce processor 0 host
>>> r186i2n12.p4.nas.nasa.gov
>>>   (line 120 of
>>>
>>>/home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetReduce/mask_test
>>>.c):
>>>   -> Simulation domain volume and reduction weight sum differ
>>>
>>>-------------------------------------------------------------------------
>>>---------
>>> After a short evolution time (7.25 M, or more than 1700 time steps),
>>>the run
>>> crashes. First there are several messages of the form:
>>>
>>>-------------------------------------------------------------------------
>>>---------
>>> INFO (CarpetReduce): Simulation domain volume: 442368
>>> INFO (CarpetReduce): Reduction weight sum:     442367.999991311
>>> INFO (CarpetRegrid2): Centre 1 is at position
>>> [-3.29831,-0.331478,7.81462e-15] with 12 levels
>>> INFO (CarpetRegrid2): Centre 2 is at position
>>>[6.58787,0.867484,8.98107e-15]
>>> with 13 levels
>>> INFO (CarpetRegrid2): Centre 3 is not active
>>> INFO (CarpetRegrid2): Regridding
>>> INFO (CarpetRegrid2): Regridding levels 10 and up
>>> WARNING level 1 in thorn CarpetLib processor 58 host
>>> r186i3n7.p4.nas.nasa.gov
>>>   (line 182 of
>>> /home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetLib/dh.cc):
>>> [...]
>>>   ->
>>> /home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetLib/dh.cc:948:
>>>    [ml=0 rl=10 c=59] The following grid structure consistency check
>>>failed:
>>>    Synchronisation and boundary prolongation: All points must have been
>>> received
>>>    needrecv.empty()
>>> [...]
>>> WARNING level 0 in thorn CarpetLib processor 94 host
>>> r186i3n12.p4.nas.nasa.gov
>>> 12676   (line 1973 of
>>> /home1/bjkelly1/CODES/Cactus/configs/hahndol/build/CarpetLib/dh.cc):
>>> 12677   -> The grid structure is inconsistent.  It is impossible to
>>> continue.
>>> --------------------------------------
>>> I'm attaching my current parameter file in full. I'm reluctant to post
>>>the
>>> entire STDOUT/STDERR, since it's around a MB in size.
>>> Any ideas would be appreciated. I would turn to the copious
>>>documentation on
>>> CarpetRegrid2, but as noted before, there is none.
>>> Thanks, Bernard
>>>
>>>-------------------------------------------------------------------------
>>>----
>>> Bernard Kelly -- CRESST Research Associate, NASA/GSFC
>>>
>>> Phone: +1 (301) 286-7243
>>> E-Mail: bernard.j.kelly at nasa.gov
>>> Web:
>>>http://science.gsfc.nasa.gov/sed/index.cfm?fuseAction=people.jumpBio&ipho
>>>nebookid=13052
>>>
>>>-------------------------------------------------------------------------
>>>----
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at einsteintoolkit.org
>>> http://lists.einsteintoolkit.org/mailman/listinfo/users
>>>
>>>
>>
>>
>>
>>--
>>Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
>
>



-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/


More information about the Users mailing list