<div dir="ltr">Hi Ian, Frank, Erik,<div><br></div><div>Thanks for the comments!<br><div><div><br></div><div>Ian, I'm using my full cluster core allocation for simulations at the moment, but should be able to get back to you about adjusting CPPFLAGS in a few days' time. The same goes for changing the number of MPI processes (I was indeed using a much higher number of processes when running on the cluster).</div></div><div><br></div><div>Frank, I haven't been able to find a (direct and obvious) indication of which bboxset implementation is used in the output. But I've attached it to the ticket, so you're welcome to have a look. (Erik did say that it was bboxset2.)</div><div><br></div><div>Gwyneth</div><div><br></div></div><div class="gmail_extra"><div class="gmail_quote">On Sun, Mar 12, 2017 at 4:27 PM, Ian Hinder <span dir="ltr"><<a href="mailto:ian.hinder@aei.mpg.de" target="_blank">ian.hinder@aei.mpg.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="word-wrap:break-word">
<br>
<div><span class="gmail-">
<div>On 12 Mar 2017, at 09:08, Gwyneth Allwright <<a href="mailto:allgwy001@myuct.ac.za" target="_blank">allgwy001@myuct.ac.za</a>> wrote:</div>
<br class="gmail-m_-5484415464625277551Apple-interchange-newline">
<blockquote type="cite">
<div dir="ltr">Hi All,<br>
<br>
One of my simulations recently aborted with the following error:<br>
<br>
cactus_ET: /opt/exp_soft/EinsteinToolkit/<wbr>Cactus/arrangements/Carpet/<wbr>CarpetLib/src/bboxset2.hh:261: bboxset2::bboxset<T, D> bboxset2::bboxset<T, D>::binary_operator(const F&, const bboxset2::bboxset<T, D>&) const [with F = bboxset2::bboxset<T, D>::operator&(const
bboxset2::bboxset<T, D>&) const [with T = int; int D = 3]::<lambda(const bboxset1&, const bboxset1&)>; T = int; int D = 3]: Assertion `all(stride == other.stride)' failed.<br>
<br>
This was when running on an HPC cluster. On my laptop, it works fine and I don't get this error.
<div><br>
</div>
<div>However, commenting out the following section in my parameter file resulted in a successful HPC run:
<div><br>
Activethorns = "CarpetIOHDF5"<br>
IOHDF5::one_file_per_group = "yes"<br>
IOHDF5::out0D_every = 128<br>
IOHDF5::out0D_vars = "<br>
WEYLSCAL4::Psi4r<br>
WEYLSCAL4::Psi4i<br>
PunctureTracker::pt_loc<br>
"<br>
<br>
This is the only HDF5 output I'm requesting. Am I doing something silly here?<br>
<br>
If it would help, I'd be happy to open a ticket and attach the whole parameter file, backtrace etc.<br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span><div>Hi Gwyneth,</div>
<div><br>
</div>
<div>There are two different "bboxset" implementations in Carpet. bboxset1 and bboxset2. Which one is used depends on the optionlist, and I think the availability of certain C++11 features. I suspect that your laptop is using bboxset1, and the cluster is
using bboxset2, and the error is only triggering in bboxset2, perhaps because it has a bug, or maybe because of a compiler bug in the cluster's compiler. Can you add</div>
<div><br>
</div>
<div> -DCARPET_DISABLE_BBOXSET2</div>
<div><br>
</div>
<div>to CPPFLAGS in the cluster's optionlist, and see if the problem goes away?</div>
<div><br>
</div>
<div>Another possibility is that you are running on a different number of MPI processes on your laptop and the cluster, and the (possible) bug is only being triggered for one particular domain decomposition. If you are not running on the same number of MPI
processes, you could try that, and see if this is responsible. In any case, I think this looks like a bug, so yes, please do open a ticket for this.</div><span class="gmail-HOEnZb"><font color="#888888">
<div>
<div><br>
</div>
</div>
</font></span></div><span class="gmail-HOEnZb"><font color="#888888">
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div>-- </div>
<div>Ian Hinder</div>
<div><a href="http://members.aei.mpg.de/ianhin" target="_blank">http://members.aei.mpg.de/<wbr>ianhin</a></div>
</div>
</div>
</div>
</div>
</div>
<br>
Disclaimer - University of Cape Town This e-mail is subject to UCT policies and e-mail disclaimer published on our website at <a href="http://www.uct.ac.za/about/policies/emaildisclaimer/" target="_blank">http://www.uct.ac.za/about/<wbr>policies/emaildisclaimer/</a> or obtainable from <a href="tel:+27%2021%20650%209111" value="+27216509111" target="_blank">+27 21 650 9111</a>. If this e-mail is not related to the business
of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via <a href="mailto:csirt@uct.ac.za" target="_blank">csirt@uct.ac.za</a>
</font></span></div>
</blockquote></div><br></div></div>