<div dir="ltr"><div>Dear Sir,</div><div><br></div><div>Thankyou very much for the swift response. I have created a ticket (<span class="gmail-issue-id">Issue #2451) and referenced this thread therein. <br></span></div><div><span class="gmail-issue-id"><br></span></div><div><span class="gmail-issue-id">I will tryout the quick workaround suggested.</span></div><div><span class="gmail-issue-id"><br></span></div><div><span class="gmail-issue-id">Thanking you,</span></div><div><span class="gmail-issue-id"><br></span></div><div><span class="gmail-issue-id">Yours sincerely<br></span></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Aug 26, 2020 at 7:11 PM Roland Haas <<a href="mailto:rhaas@illinois.edu">rhaas@illinois.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello all,<br>
<br>
a simple reproducer is attached (modified qlm-ks.par test of QLM).<br>
<br>
To make it fail I need more than 1 refinement level active. It fails on<br>
both 1 and 4 MPI ranks (that I have tested on).<br>
<br>
Adding a bit of debug output:<br>
<br>
std::cerr << "active1: " << active0 << " outputslab: " << outputslab << std::endl;<br>
<br>
I get:<br>
<br>
active1: bboxset<CCTK_INT4,3>(set<bbox>:{([12,12,12]:[136,136,136]:[4,4,4]/[3,3,3]:[34,34,34]/[32,32,32]/32768)},stride:[4,4,4],offset:[0,0,0]) outputslab: ([0,0,0]:[38,75,0]:[1,1,1]/[0,0,0]:[38,75,0]/[39,76,1]/2964)<br>
cactus_sim: /data/rhaas/postdoc/gr/cactus/ET_trunk/arrangements/Carpet/CarpetLib/src/bboxset2.hh:262: CarpetLib::bboxset2::bboxset<T, D> CarpetLib::bboxset2::bboxset<T, D>::binary_operator(const F&, const CarpetLib::bboxset2::bboxset<T, D>&) const [with F = CarpetLib::bboxset2::bboxset<T, D>::operator&<int, 3>::<lambda(const bboxset1&, const bboxset1&)>; T = int; int D = 3]: Assertion `all(stride == other.stride)' failed.<br>
<br>
which seems to indicate the issue is really that one should not try to<br>
use "active" for anything other than grid functions by adding a:<br>
<br>
if (groupdata.grouptype == CCTK_GF)<br>
<br>
before the block of code that I suggested be commented out ("active"<br>
only makes sense for grid functions, since it refers to [basically] the<br>
non-buffer points due to mesh refinement).<br>
<br>
@Vaishak: please create a ticket with all files and text included, so<br>
that a fix can be properly proposed and reviewed?<br>
<br>
Yours,<br>
Roland<br>
<br>
> Hello Vaishak,<br>
> <br>
> > I am currently facing an issue when attempting to output QuasiLocalMeasures<br>
> > variables in HDF5 (in a BBH simulation similar to the gallery one) files<br>
> > instead of ASCII. I only face this issue when I request any<br>
> > QuasiLocalMeasures variables to be output in IOHDF5.<br>
> > <br>
> > I feel I might have done something silly here. Requesting your help.<br>
> > <br>
> > I am getting the same error message as in 005361<br>
> > <<a href="http://lists.einsteintoolkit.org/pipermail/users/2017-March/005361.html" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/pipermail/users/2017-March/005361.html</a>> (<br>
> > <a href="http://lists.einsteintoolkit.org/pipermail/users/2017-March/005361.html" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/pipermail/users/2017-March/005361.html</a>). . <br>
> Hmm, ok. So in your case your output from QLM is:<br>
> <br>
> IOHDF5::out2D_vars = "<br>
> [...]<br>
> QuasiLocalMeasures::qlm_newman_penrose<br>
> [...]<br>
> <br>
> which is declared in interface.ccl as a:<br>
> <br>
> COMPLEX qlm_newman_penrose[num_surfaces] TYPE=array DIM=2 SIZE=SphericalSurface::maxntheta, SphericalSurface::maxnphi DISTRIB=constant TAGS='Checkpoint="no",convergence_power=1'<br>
> <br>
> ie a DISTRIB=constant vector of 2-dimensional grid arrays.<br>
> <br>
> As Erik suggested in the year 2017 to Gwyneth, it may be best to take<br>
> the content of your email and create a ticket at:<br>
> <br>
> <a href="https://bitbucket.org/einsteintoolkit/tickets/issues/new" rel="noreferrer" target="_blank">https://bitbucket.org/einsteintoolkit/tickets/issues/new</a><br>
> <br>
> which lets you upload attachments after the ticket has been created<br>
> through the "More v" menu (next to the "Edit" button).<br>
> <br>
> > I am attaching the parameter file, the backtrace and the out and err files<br>
> > from the run. <br>
> Thank you! These are very helpful.<br>
> <br>
> My own, unverified guess, is that is that is something connected to<br>
> being a DISTRIB=constant 2D array which, if I remember correctly, are<br>
> implemented by setting up a 3D array where the last dimension is the<br>
> MPI rank number.<br>
> <br>
> Then in CarpetIOHDF5::WriteHDF5<2> in line 1395 bboxsset::operator& is<br>
> called:<br>
> <br>
> const ibset active1 = active0 & outputslab;<br>
> <br>
> where outputslab is an actual 2D set while active0 has the (pseudo) 3D<br>
> shape due to DISTRIB=constant.<br>
> <br>
> At least that would be my initial assumption. The good thing is that<br>
> this should be trivial to test since one only has to request any output.<br>
> <br>
> If you want a quick (possible) workaround, then you can edit <br>
> <br>
> CarpetIOHDF5/src/OutputSlice.cc and in the WriteHDF5 function comment<br>
> out the block of code after "string active" in line 1378ff:<br>
> <br>
> --8<--<br>
> string active;<br>
> #if 0 // comment out block, leaving active an empty string<br>
> {<br>
> // Determine extent of hyperslab that is output<br>
> ivect lo = gfext.lower();<br>
> ivect up = gfext.upper();<br>
> ivect str = gfext.stride();<br>
> for (int d = 0; d < dim; ++d) {<br>
> bool isoutdir = false;<br>
> for (int e = 0; e < outdim; ++e)<br>
> isoutdir |= d == dirs[e];<br>
> if (!isoutdir) {<br>
> lo[d] = org[d];<br>
> up[d] = org[d];<br>
> }<br>
> }<br>
> const ibbox outputslab(lo, up, str);<br>
> // Intersect active region with this hyperslab<br>
> const ibset &active0 = <a href="http://vdd.at" rel="noreferrer" target="_blank">vdd.at</a>(m)-><a href="http://level_boxes.at" rel="noreferrer" target="_blank">level_boxes.at</a>(ml).at(rl).active;<br>
> const ibset active1 = active0 & outputslab;<br>
> // Reduce dimensionality of active region<br>
> bboxset<int, outdim> active2;<br>
> for (const ibbox &box0 : active1.iterator()) {<br>
> const vect<int, outdim> lo = box0.lower()[dirs];<br>
> const vect<int, outdim> up = box0.upper()[dirs];<br>
> const vect<int, outdim> str = box0.stride()[dirs];<br>
> const CarpetLib::bbox<int, outdim> box(lo, up, str);<br>
> active2 += box;<br>
> }<br>
> ostringstream buf;<br>
> buf << active2;<br>
> active = buf.str();<br>
> }<br>
> #endif<br>
> --8<--<br>
> <br>
> ASCII output is not affected by this since computing of "active" is not<br>
> done there as the "active" region is stored only as an attribute in<br>
> HDF5 files (and ASCII files have no way of storing attributes).<br>
> <br>
> Yours,<br>
> Roland<br>
> <br>
<br>
<br>
-- <br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://pgp.mit.edu" rel="noreferrer" target="_blank">http://pgp.mit.edu</a> .<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">Vaishak P<br><br>PhD Scholar,<br>Shyama Prasad Mukherjee Fellow<br>
Inter-University Center for Astronomy and Astrophysics (IUCAA)<br>
Pune, India<div><br></div></div></div></div></div>