#2117: CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" break 'IO::out_mode = "onefile"' --------------------+------------------------------------------------------- Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: major | Milestone: Component: Carpet | Version: development version Keywords: | --------------------+------------------------------------------------------- Setting {{{ IO::out_mode = "onefile" }}} in a parfile and running with more than 1 MPI ranks causes and out-of- range array access in Output.cc in line 913: {{{ ostringstream buf; buf << (vdd.at(Carpet::map) ->local_boxes.at(mglevel) .at(refinementlevel) ----> .at(local_component) .active); }}} due to {{{local_component}}} being -1. This is the backtrace: {{{ #6 0x00007ffff3a03355 in std::__throw_out_of_range_fmt(char const*, ...) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00005555578f17ee in std::vector<dh::local_dboxes, std::allocatordh::local_dboxes >::_M_range_check (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:825 #8 std::vector<dh::local_dboxes, std::allocatordh::local_dboxes >::at (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:846 #9 CarpetIOHDF5::AddAttributes (cctkGH=cctkGH@entry=0x55555f829ab0, fullname=fullname@entry=0x5555634efb10 "TMUNUBASE::eTxx", vdim=3, refinementlevel=<optimized out>, refinementlevel@entry=0, request=request@entry=0x5555635801b0, bbox=..., dataset=360287970189640065, is_index=false) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:913 #10 0x00005555578f2a45 in CarpetIOHDF5::WriteVarChunkedSequential (cctkGH=cctkGH@entry=0x55555f829ab0, outfile=outfile@entry=72057594037928056, io_bytes=@0x7fffffffb960: 62212, request=0x5555635801b0, called_from_checkpoint=called_from_checkpoint@entry=true, indexfile=indexfile@entry=-1) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:513 #11 0x00005555578d9bdf in CarpetIOHDF5::Checkpoint (cctkGH=0x55555f829ab0, called_from=0) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/CarpetIOHDF5.cc:1218 }}}
To reproduce:
{{{ wget https://einsteintoolkit.org/gallery/ns/tov.par OMP_NUM_THREADS=1 mpirun -n 2 exe/cactus_sim tov.par }}} }}}
#2117: CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" break 'IO::out_mode = "onefile"' ---------------------+------------------------------------------------------ Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: major | Milestone: Component: Carpet | Version: development version Resolution: | Keywords: ---------------------+------------------------------------------------------ Description changed by rhaas:
Old description:
Setting {{{ IO::out_mode = "onefile" }}} in a parfile and running with more than 1 MPI ranks causes and out-of- range array access in Output.cc in line 913: {{{ ostringstream buf; buf << (vdd.at(Carpet::map) ->local_boxes.at(mglevel) .at(refinementlevel) ----> .at(local_component) .active); }}} due to {{{local_component}}} being -1. This is the backtrace: {{{ #6 0x00007ffff3a03355 in std::__throw_out_of_range_fmt(char const*, ...) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00005555578f17ee in std::vector<dh::local_dboxes, std::allocatordh::local_dboxes >::_M_range_check (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:825 #8 std::vector<dh::local_dboxes, std::allocatordh::local_dboxes >::at (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:846 #9 CarpetIOHDF5::AddAttributes (cctkGH=cctkGH@entry=0x55555f829ab0, fullname=fullname@entry=0x5555634efb10 "TMUNUBASE::eTxx", vdim=3, refinementlevel=<optimized out>, refinementlevel@entry=0, request=request@entry=0x5555635801b0, bbox=..., dataset=360287970189640065, is_index=false) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:913 #10 0x00005555578f2a45 in CarpetIOHDF5::WriteVarChunkedSequential (cctkGH=cctkGH@entry=0x55555f829ab0, outfile=outfile@entry=72057594037928056, io_bytes=@0x7fffffffb960: 62212, request=0x5555635801b0, called_from_checkpoint=called_from_checkpoint@entry=true, indexfile=indexfile@entry=-1) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:513 #11 0x00005555578d9bdf in CarpetIOHDF5::Checkpoint (cctkGH=0x55555f829ab0, called_from=0) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/CarpetIOHDF5.cc:1218 }}}
To reproduce:
{{{ wget https://einsteintoolkit.org/gallery/ns/tov.par OMP_NUM_THREADS=1 mpirun -n 2 exe/cactus_sim tov.par }}} }}}
New description:
Setting {{{ IO::out_mode = "onefile" }}} in a parfile and running with more than 1 MPI ranks causes and out-of- range array access in Output.cc in line 913: {{{ ostringstream buf; buf << (vdd.at(Carpet::map) ->local_boxes.at(mglevel) .at(refinementlevel) ----> .at(local_component) .active); }}} due to {{{local_component}}} being -1. This is the backtrace: {{{ #6 0x00007ffff3a03355 in std::__throw_out_of_range_fmt(char const*, ...) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00005555578f17ee in std::vector<dh::local_dboxes, std::allocatordh::local_dboxes >::_M_range_check (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:825 #8 std::vector<dh::local_dboxes, std::allocatordh::local_dboxes >::at (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:846 #9 CarpetIOHDF5::AddAttributes (cctkGH=cctkGH@entry=0x55555f829ab0, fullname=fullname@entry=0x5555634efb10 "TMUNUBASE::eTxx", vdim=3, refinementlevel=<optimized out>, refinementlevel@entry=0, request=request@entry=0x5555635801b0, bbox=..., dataset=360287970189640065, is_index=false) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:913 #10 0x00005555578f2a45 in CarpetIOHDF5::WriteVarChunkedSequential (cctkGH=cctkGH@entry=0x55555f829ab0, outfile=outfile@entry=72057594037928056, io_bytes=@0x7fffffffb960: 62212, request=0x5555635801b0, called_from_checkpoint=called_from_checkpoint@entry=true, indexfile=indexfile@entry=-1) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:513 #11 0x00005555578d9bdf in CarpetIOHDF5::Checkpoint (cctkGH=0x55555f829ab0, called_from=0) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/CarpetIOHDF5.cc:1218 }}}
To reproduce:
{{{ wget https://einsteintoolkit.org/gallery/ns/tov.par OMP_NUM_THREADS=1 mpirun -n 2 exe/cactus_sim tov.par }}}
--
#2117: CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" break 'IO::out_mode = "onefile"' ---------------------+------------------------------------------------------ Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: major | Milestone: ET_2018_02 Component: Carpet | Version: development version Resolution: | Keywords: ---------------------+------------------------------------------------------ Changes (by rhaas):
* milestone: => ET_2018_02
#2117: CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" breaks 'IO::out_mode = "onefile"' ---------------------+------------------------------------------------------ Reporter: rhaas | Owner: eschnett Type: defect | Status: new Priority: major | Milestone: ET_2018_02 Component: Carpet | Version: development version Resolution: | Keywords: ---------------------+------------------------------------------------------
trac@lists.einsteintoolkit.org