Hi Ian, Frank, Erik,

Thanks for the comments!

Ian, I'm using my full cluster core allocation for simulations at the moment, but should be able to get back to you about adjusting CPPFLAGS in a few days' time. The same goes for changing the number of MPI processes (I was indeed using a much higher number of processes when running on the cluster).

Frank, I haven't been able to find a (direct and obvious) indication of which bboxset implementation is used in the output. But I've attached it to the ticket, so you're welcome to have a look. (Erik did say that it was bboxset2.)

Gwyneth

On Sun, Mar 12, 2017 at 4:27 PM, Ian Hinder <ian.hinder@aei.mpg.de> wrote:

On 12 Mar 2017, at 09:08, Gwyneth Allwright <allgwy001@myuct.ac.za> wrote:

Hi All,

One of my simulations recently aborted with the following error:

cactus_ET: /opt/exp_soft/EinsteinToolkit/Cactus/arrangements/Carpet/CarpetLib/src/bboxset2.hh:261: bboxset2::bboxset<T, D> bboxset2::bboxset<T, D>::binary_operator(const F&, const bboxset2::bboxset<T, D>&) const [with F = bboxset2::bboxset<T, D>::operator&(const bboxset2::bboxset<T, D>&) const [with T = int; int D = 3]::<lambda(const bboxset1&, const bboxset1&)>; T = int; int D = 3]: Assertion `all(stride == other.stride)' failed.

This was when running on an HPC cluster. On my laptop, it works fine and I don't get this error. 

However, commenting out the following section in my parameter file resulted in a successful HPC run:

Activethorns = "CarpetIOHDF5"
IOHDF5::one_file_per_group =  "yes"
IOHDF5::out0D_every =  128
IOHDF5::out0D_vars =  "
  WEYLSCAL4::Psi4r
  WEYLSCAL4::Psi4i
  PunctureTracker::pt_loc
"

This is the only HDF5 output I'm requesting. Am I doing something silly here?

If it would help, I'd be happy to open a ticket and attach the whole parameter file, backtrace etc.

Hi Gwyneth,

There are two different "bboxset" implementations in Carpet.  bboxset1 and bboxset2.  Which one is used depends on the optionlist, and I think the availability of certain C++11 features.  I suspect that your laptop is using bboxset1, and the cluster is using bboxset2, and the error is only triggering in bboxset2, perhaps because it has a bug, or maybe because of a compiler bug in the cluster's compiler.  Can you add

 -DCARPET_DISABLE_BBOXSET2

to CPPFLAGS in the cluster's optionlist, and see if the problem goes away?

Another possibility is that you are running on a different number of MPI processes on your laptop and the cluster, and the (possible) bug is only being triggered for one particular domain decomposition.  If you are not running on the same number of MPI processes, you could try that, and see if this is responsible.  In any case, I think this looks like a bug, so yes, please do open a ticket for this.


Disclaimer - University of Cape Town This e-mail is subject to UCT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 9111. If this e-mail is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via csirt@uct.ac.za