[Users] Help with hyperslabbing error message

[Shoup, Anthony]

Hi All,


I am conducting initial testing of using the Einstein Toolkit on my cluster.  I have been able to complete successful runs of the example simulation: BBHMedRes.par with multiple mpi processes per node with 1 thread per process. Now I am exploring using multiple threads per process. My first attempt was 1 process per node and 2 threads per process.  However, simulation aborted after about the third regrid with the error:


WARNING level 0 from host node1 process 0
  while executing schedule bin BoundaryConditions, routine RotatingSymmetry180::Rot180_ApplyBC
  in thorn RotatingSymmetry180, file /home/Cactus/Cactus/configs/sim/build/RotatingSymmetry180/rotatingsymmetry180.c:460:
  -> TAT/Slab can only be used if there is a single local component per MPI process
cactus_sim: /home/Cactus/Cactus/configs/sim/build/Carpet/helpers.cc:275: int Carpet::Abort(const cGH*, int): Assertion `0' failed.
Rank WARNING level 0 from host node1 process 0
  while executing schedule bin BoundaryConditions, routine RotatingSymmetry180::Rot180_ApplyBC
  in thorn RotatingSymmetry180, file /home/Cactus/Cactus/configs/sim/build/RotatingSymmetry180/rotatingsymmetry180.c:460:
  -> TAT/Slab can only be used if there is a single local component per MPI process
0 with PID 5869 received signal 6
Writing backtrace to BBHMedRes/backtrace.0.txt
Aborted
Sat Jan 26 13:36:02 UTC 2019
Simfactory Done at date: 0


I think what is happening is that a "slab" is trying to be moved/copied during the regrid attempt and it crosses a boundry to another patch?


Is there a parameter in my .par file I need to change?  I have attached my par file to this email.


Any help anyone can give would be greatly appreciated.  Tony...


[https://email.osu.edu/owa/attachment.ashx?id=RgAAAAAb%2fHy0wVvTSoHQx8OJXAaLBwCiA5IZrwRKTqiLVNbt4xWyAAAAAAFUAADltPc25wRDT4tJbW9en2wXAHKArVn%2fAAAJ&attcnt=1&attid0=EAD8Cse5Lj5uQ6ZJWk98Q%2blj]
Anthony Shoup PhD, Senior Lecturer
College of Arts & Sciences, College of Engineering Departments of Physics, Astronomy, EEIC
315 Science Bldg. | 4250 Campus Dr. Lima, OH 45807
419-995-8018 Office | 419-516-2257 Mobile
shoup.31 at osu.edu<https://email.osu.edu/owa/redir.aspx?C=j5WpnJiBk0W5oVlCbtvB-xiCkA_lbdEIi9hlk7ByHiG7ARrxjwDFmAW8S_XespJbMLJRblY5JKc.&URL=mailto%3ashoup.31%40osu.edu> osu.edu<https://email.osu.edu/owa/redir.aspx?C=j5WpnJiBk0W5oVlCbtvB-xiCkA_lbdEIi9hlk7ByHiG7ARrxjwDFmAW8S_XespJbMLJRblY5JKc.&URL=http%3a%2f%2fosu.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20190126/0f284081/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: BBHMedRes.par
Type: application/octet-stream
Size: 21801 bytes
Desc: BBHMedRes.par
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20190126/0f284081/attachment-0001.obj