[ET Trac] [Einstein Toolkit] #800: inconsistent boxes during refluxing very late in the run
Einstein Toolkit
trac-noreply at einsteintoolkit.org
Wed Apr 11 13:40:37 CDT 2012
#800: inconsistent boxes during refluxing very late in the run
-----------------------+----------------------------------------------------
Reporter: rhaas | Owner:
Type: defect | Status: new
Priority: major | Milestone:
Component: Other | Version:
Keywords: Refluxing |
-----------------------+----------------------------------------------------
this is the second half of #797. Same parameter files but using the
Refluxing thorn. Eventually I get:
{{{
srcbbox:
([25248,27424,26272]:[28704,30432,29088]:[64,64,64]/[394,428,410]:[448,475,454]/[55,48,45]/118800)
dstbbox:
([28736,27840,26688]:[28736,28096,28992]:[128,128,128]/[224,217,208]:[224,219,226]/[1,3,19]/57)
regbbox:
([28736,27840,26688]:[28736,28096,28992]:[128,128,128]/[224,217,208]:[224,219,226]/[1,3,19]/57)
srcbbox: ([25248,27424,31328]:[28704,30432,34144]:[64,WARNING level 0 in
thorn CarpetLib processor 80 host shc054
(line 174 of
/home/rhaas/cactus/Zelmani/arrangements/Carpet/CarpetLib/src/restrict_3d_vc_rf2.cc):
-> Internal error: region extent is not contained in array extent
srcbbox:
([25248,27424,23712]:[28704,30432,26592]:[64,64,64]/[394,428,370]:[448,475,415]/[55,48,46]/121440)
dstbbox:
([28736,27840,24512]:[28736,28096,25792]:[128,128,128]/[224,217,191]:[224,219,201]/[1,3,11]/33)
regbbox:
([28736,27840,24512]:[28736,28096,25792]:[128,128,128]/[224,217,191]:[224,219,201]/[1,3,11]/33)
64,64]/[394,428,489]:[448,475,533]/[55,48,45]/118800)
dstbbox:
([28736,27840,31680]:[28736,28096,33344]:[128,128,128]/[224,217,247]:[224,219,260]/[1,3,14]/42)
regbbox:
([28736,27840,31680]:[28736,28096,33344]:[128,128,128]/[224,217,247]:[224,219,260]/[1,3,14]/42)
WARNING level 0 in thorn CarpetLib processor 82 host shc053
(line 174 of
/home/rhaas/cactus/Zelmani/arrangements/Carpet/CarpetLib/src/restrict_3d_vc_rf2.cc):
-> Internal error: region extent is not contained in array extent
WARNING level 0 in thorn CarpetLib processor 79 host shc055
(line 174 of
/home/rhaas/cactus/Zelmani/arrangements/Carpet/CarpetLib/src/restrict_3d_vc_rf2.cc):
-> Internal error: region extent is not contained in array extent
srcbbox:
([25248,27424,28768]:[28704,30432,31648]:[64,64,64]/[394,428,449]:[448,475,494]/[55,48,46]/121440)
dstbbox:
([28736,27840,29120]:[28736,28096,29888]:[128,128,128]/[224,217,227]:[224,219,233]/[1,3,7]/21)
regbbox:
([28736,27840,29120]:[28736,28096,29888]:[128,128,128]/[224,217,227]:[224,219,233]/[1,3,7]/21)
WARNING level 0 in thorn CarpetLib processor 81 host shc054
(line 174 of
/home/rhaas/cactus/Zelmani/arrangements/Carpet/CarpetLib/src/restrict_3d_vc_rf2.cc):
-> Internal error: region extent is not contained in array extent
}}}
which happens during refluxing
{{{
INFO (Carpet): [ml=0][rl=2][tl=0] Evolution/PostRestrict at iteration
83712 time 196.2
INFO (Carpet): [ml=0][rl=2][tl=0] Scheduling CCTK_POSTRESTRICT
INFO (Carpet): [ml=0][rl=2][tl=0] Level mode call at CCTK_POSTRESTRICT to
Refluxing::Refluxing_CorrectState
INFO (Refluxing): Refluxing at iteration 83712 on level 2 of 4
INFO (Refluxing): Refluxing on level 2:
INFO (Carpet): [ml=0][rl=2][tl=0] Entering singlemap mode
INFO (Carpet): [ml=0][rl=2][m=0][tl=0] Entering local mode
INFO (Carpet): [ml=0][rl=2][m=0][c=0,lc=0][tl=0] Leaving local mode
INFO (Carpet): [ml=0][rl=2][m=0][tl=0] Leaving singlemap mode
INFO (Carpet): [ml=0][rl=2][tl=0] Leaving level mode
INFO (Carpet): [ml=0][tl=0] Entering level mode
INFO (Carpet): [ml=0][rl=3][tl=0] Entering singlemap mode
INFO (Carpet): [ml=0][rl=3][m=0][tl=0] Entering local mode
INFO (Carpet): [ml=0][rl=3][m=0][c=0,lc=0][tl=0] Leaving local mode
INFO (Carpet): [ml=0][rl=3][m=0][tl=0] Leaving singlemap mode
INFO (Carpet): [ml=0][rl=3][tl=0] Leaving level mode
INFO (Carpet): [ml=0][tl=0] Entering level mode
INFO (CarpetLib): About to MPI_Isend to processor 1 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 2 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 3 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 7 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 8 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 9 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Waitall
INFO (CarpetLib): Finished MPI_Waitall
INFO (CarpetLib): About to MPI_Waitall
INFO (CarpetLib): Finished MPI_Waitall
INFO (Carpet): [ml=0][rl=2][tl=0] Entering singlemap mode
INFO (Carpet): [ml=0][rl=2][m=0][tl=0] Entering local mode
INFO (Refluxing): Refluxing on level 2 map 0 component 0 direction 0 face
0: [25,25,25]:[25,31,35]
INFO (Refluxing): Refluxing on level 2 map 0 component 0 direction 1 face
0: [25,25,25]:[33,25,35]
INFO (Refluxing): Refluxing on level 2 map 0 component 0 direction 2 face
0: [25,25,25]:[33,31,25]
INFO (Carpet): [ml=0][rl=2][m=0][c=0,lc=0][tl=0] Leaving local mode
INFO (Carpet): [ml=0][rl=2][m=0][tl=0] Leaving singlemap mode
INFO (Carpet): [ml=0][rl=2][tl=0] Leaving level mode
INFO (Carpet): [ml=0][tl=0] Entering level mode
INFO (Carpet): [ml=0][rl=3][tl=0] Entering singlemap mode
INFO (Carpet): [ml=0][rl=3][m=0][tl=0] Entering local mode
INFO (Carpet): [ml=0][rl=3][m=0][c=0,lc=0][tl=0] Leaving local mode
INFO (Carpet): [ml=0][rl=3][m=0][tl=0] Leaving singlemap mode
INFO (Carpet): [ml=0][rl=3][tl=0] Leaving level mode
INFO (Carpet): [ml=0][tl=0] Entering level mode
INFO (Carpet): [ml=0][rl=2][tl=0] Entering singlemap mode
INFO (Carpet): [ml=0][rl=2][m=0][tl=0] Entering local mode
INFO (Carpet): [ml=0][rl=2][m=0][c=0,lc=0][tl=0] Leaving local mode
INFO (Carpet): [ml=0][rl=2][m=0][tl=0] Leaving singlemap mode
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "GRHYDRO::DENS"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "GRHYDRO::SCON"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "GRHYDRO::TAU" iteration=83712
time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] ProlongateGroups
INFO (CarpetLib): About to MPI_Irecv from processor 1 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 6 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 36 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 39 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 48 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Waitall
INFO (CarpetLib): Finished MPI_Waitall
INFO (CarpetLib): About to MPI_Waitall
INFO (CarpetLib): Finished MPI_Waitall
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroups
INFO (CarpetLib): About to MPI_Irecv from processor 1 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 2 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 3 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 7 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 8 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 9 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 10 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Isend to processor 1 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 2 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 3 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 7 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 8 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 9 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Isend to processor 10 for type double
INFO (CarpetLib): Finished MPI_Isend
INFO (CarpetLib): About to MPI_Waitall
INFO (CarpetLib): Finished MPI_Waitall
INFO (CarpetLib): About to MPI_Waitall
INFO (CarpetLib): Finished MPI_Waitall
INFO (Carpet): [ml=0][rl=2][tl=0] Level mode call at MoL_PostStep to
ADMBase::ADMBase_Boundaries
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "ADMBASE::LAPSE"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "ADMBASE::DTLAPSE"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "ADMBASE::SHIFT"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "ADMBASE::DTSHIFT"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "ADMBASE::METRIC"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] SyncGroup "ADMBASE::CURV"
iteration=83712 time=196.2
INFO (Carpet): [ml=0][rl=2][tl=0] ProlongateGroups
INFO (CarpetLib): About to MPI_Irecv from processor 1 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 6 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 36 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 39 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Irecv from processor 48 for type double
INFO (CarpetLib): Finished MPI_Irecv
INFO (CarpetLib): About to MPI_Waitall
}}}
The full stdout and stderr files are about 250MB for this (because of all
the verbosity).
I agree that this is not a very helpful error report. Unfortunately I
don't know how to distill more useful information out of it :-(
--
Ticket URL: <https://trac.einsteintoolkit.org/ticket/800>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit
More information about the Trac
mailing list