[ET Trac] [Einstein Toolkit] #800: inconsistent boxes during refluxing very late in the run

[Einstein Toolkit]

#800: inconsistent boxes during refluxing very late in the run
---------------------+------------------------------------------------------
  Reporter:  rhaas   |       Owner:           
      Type:  defect  |      Status:  new      
  Priority:  major   |   Milestone:           
 Component:  Other   |     Version:           
Resolution:          |    Keywords:  Refluxing
---------------------+------------------------------------------------------

Comment (by rhaas):

 I did some debugging on this and have come up a set of patches to actually
 evolve past the point of failure. Since the error happens in the mixed
 vertex-cell-centered restriciton routine used during refluxing I changed
 the containedness test that triggers the assert to shift the boxes by half
 a stride in the directions in which the boxes are vertex centered. Ie. it
 undoes the shift that happened when the sendrecv boxes are set up in
 dh.cc's regrid function (in the refluxing section). What sounds fishy
 about this to me is that this seems indicate that the dstbox was at the
 very edge of the source box so I am not sure if relxuing is actually
 happening at the correct location. Looking at the boxes that are
 distributed between the processors I have that the error was triggered by
 {{{
 srcbbox:
 ([23904,27552,28768]:[29088,30304,34144]:[64,64,64]/[373,430,449]:[454,473,533]/[82,44,85]/306680)
 dstbbox:
 ([29120,29632,29120]:[29120,29888,32576]:[128,128,128]/[227,231,227]:[227,233,254]/[1,3,28]/84)
 regbbox:
 ([29120,29632,29120]:[29120,29888,32576]:[128,128,128]/[227,231,227]:[227,233,254]/[1,3,28]/84)
 }}}
 in component 27 (processor 27) which is the source in the following
 pseudoregion
 {{{
  fast_ref_refl_sendrecv_0_0:
 [(send:(ext:([29088,29600,29088]:[29088,29920,32608]:[64,64,64]/[454,462,454]:[454,467,509]/[1,6,56]/336),c:27),recv:(ext:([29120,29632,29120]:[29120,29888,32576]:[128,128,128]/[227,231,227]:[227,233,254]/[1,3,28]/84),c:24)),...
 }}}
 However the sendrecv actually is at the edge of two components namely
 component 27 and 29:
 {{{
 ml=0 rl=3 c=27
 dh::light_dboxes:{
    exterior:
 ([23904,27552,28768]:[29088,30304,34144]:[64,64,64]/[373,430,449]:[454,473,533]/[82,44,85]/306680)
    owned:
 ([24096,27744,28960]:[28896,30112,33952]:[64,64,64]/[376,433,452]:[451,470,530]/[76,38,79]/228152)
    interior:
 ([24096,27744,28960]:[28896,30112,33952]:[64,64,64]/[376,433,452]:[451,470,530]/[76,38,79]/228152)
    active_size: 153020
 }

 ml=0 rl=3 c=29
 dh::light_dboxes:{
    exterior:
 ([28768,27552,28768]:[34016,30304,34144]:[64,64,64]/[449,430,449]:[531,473,533]/[83,44,85]/310420)
    owned:
 ([28960,27744,28960]:[33824,30112,33952]:[64,64,64]/[452,433,452]:[528,470,530]/[77,38,79]/231154)
    interior:
 ([28960,27744,28960]:[33824,30112,33952]:[64,64,64]/[452,433,452]:[528,470,530]/[77,38,79]/231154)
    active_size: 155050
 }
 }}}

 So it would seems as if this is not actually a refinement boundary but
 just an inter-processor boundary and that refluxing should not happen here
 (the interface coordinate is 29056) at all. On the other hand, the logic
 in dh.cc that computes the interfaces seems sound (module issues that I
 would have expected more instances where face==1 interfaces are shifted
 right and face==0 interfaces are shifted left but that does not actually
 change any of the sendrecv created; I assume that the ghost zones help).

 I attach my patches that remove the warning but that might well not fix
 that underlying problem if my reading of the box layout is correct.

-- 
Ticket URL: <https://trac.einsteintoolkit.org/ticket/800#comment:2>
Einstein Toolkit <http://einsteintoolkit.org>
The Einstein Toolkit