#800: inconsistent boxes during refluxing very late in the run ---------------------+------------------------------------------------------ Reporter: rhaas | Owner: Type: defect | Status: new Priority: major | Milestone: Component: Other | Version: Resolution: | Keywords: Refluxing ---------------------+------------------------------------------------------
Comment (by rhaas):
I did some debugging on this and have come up a set of patches to actually evolve past the point of failure. Since the error happens in the mixed vertex-cell-centered restriciton routine used during refluxing I changed the containedness test that triggers the assert to shift the boxes by half a stride in the directions in which the boxes are vertex centered. Ie. it undoes the shift that happened when the sendrecv boxes are set up in dh.cc's regrid function (in the refluxing section). What sounds fishy about this to me is that this seems indicate that the dstbox was at the very edge of the source box so I am not sure if relxuing is actually happening at the correct location. Looking at the boxes that are distributed between the processors I have that the error was triggered by {{{ srcbbox: ([23904,27552,28768]:[29088,30304,34144]:[64,64,64]/[373,430,449]:[454,473,533]/[82,44,85]/306680) dstbbox: ([29120,29632,29120]:[29120,29888,32576]:[128,128,128]/[227,231,227]:[227,233,254]/[1,3,28]/84) regbbox: ([29120,29632,29120]:[29120,29888,32576]:[128,128,128]/[227,231,227]:[227,233,254]/[1,3,28]/84) }}} in component 27 (processor 27) which is the source in the following pseudoregion {{{ fast_ref_refl_sendrecv_0_0: [(send:(ext:([29088,29600,29088]:[29088,29920,32608]:[64,64,64]/[454,462,454]:[454,467,509]/[1,6,56]/336),c:27),recv:(ext:([29120,29632,29120]:[29120,29888,32576]:[128,128,128]/[227,231,227]:[227,233,254]/[1,3,28]/84),c:24)),... }}} However the sendrecv actually is at the edge of two components namely component 27 and 29: {{{ ml=0 rl=3 c=27 dh::light_dboxes:{ exterior: ([23904,27552,28768]:[29088,30304,34144]:[64,64,64]/[373,430,449]:[454,473,533]/[82,44,85]/306680) owned: ([24096,27744,28960]:[28896,30112,33952]:[64,64,64]/[376,433,452]:[451,470,530]/[76,38,79]/228152) interior: ([24096,27744,28960]:[28896,30112,33952]:[64,64,64]/[376,433,452]:[451,470,530]/[76,38,79]/228152) active_size: 153020 }
ml=0 rl=3 c=29 dh::light_dboxes:{ exterior: ([28768,27552,28768]:[34016,30304,34144]:[64,64,64]/[449,430,449]:[531,473,533]/[83,44,85]/310420) owned: ([28960,27744,28960]:[33824,30112,33952]:[64,64,64]/[452,433,452]:[528,470,530]/[77,38,79]/231154) interior: ([28960,27744,28960]:[33824,30112,33952]:[64,64,64]/[452,433,452]:[528,470,530]/[77,38,79]/231154) active_size: 155050 } }}}
So it would seems as if this is not actually a refinement boundary but just an inter-processor boundary and that refluxing should not happen here (the interface coordinate is 29056) at all. On the other hand, the logic in dh.cc that computes the interfaces seems sound (module issues that I would have expected more instances where face==1 interfaces are shifted right and face==0 interfaces are shifted left but that does not actually change any of the sendrecv created; I assume that the ghost zones help).
I attach my patches that remove the warning but that might well not fix that underlying problem if my reading of the box layout is correct.