<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Hi All,</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">I am conducting initial testing of using the Einstein Toolkit on my cluster. I have been able to complete successful runs of the example simulation: BBHMedRes.par with multiple mpi processes per node with 1 thread per
process. Now I am exploring using multiple threads per process. My first attempt was 1 process per node and 2 threads per process. However, simulation aborted after about the third regrid with the error:</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"></p>
<div>WARNING level 0 from host node1 process 0</div>
<div> while executing schedule bin BoundaryConditions, routine RotatingSymmetry180::Rot180_ApplyBC</div>
<div> in thorn RotatingSymmetry180, file /home/Cactus/Cactus/configs/sim/build/RotatingSymmetry180/rotatingsymmetry180.c:460:</div>
<div> -> TAT/Slab can only be used if there is a single local component per MPI process</div>
<div>cactus_sim: /home/Cactus/Cactus/configs/sim/build/Carpet/helpers.cc:275: int Carpet::Abort(const cGH*, int): Assertion `0' failed.</div>
<div>Rank WARNING level 0 from host node1 process 0</div>
<div> while executing schedule bin BoundaryConditions, routine RotatingSymmetry180::Rot180_ApplyBC</div>
<div> in thorn RotatingSymmetry180, file /home/Cactus/Cactus/configs/sim/build/RotatingSymmetry180/rotatingsymmetry180.c:460:</div>
<div> -> TAT/Slab can only be used if there is a single local component per MPI process</div>
<div>0 with PID 5869 received signal 6</div>
<div>Writing backtrace to BBHMedRes/backtrace.0.txt</div>
<div>Aborted</div>
<div>Sat Jan 26 13:36:02 UTC 2019</div>
<div>Simfactory Done at date: 0</div>
<br>
<p></p>
<p style="margin-top:0;margin-bottom:0">I think what is happening is that a "slab" is trying to be moved/copied during the regrid attempt and it crosses a boundry to another patch?</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Is there a parameter in my .par file I need to change? I have attached my par file to this email.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Any help anyone can give would be greatly appreciated. Tony...</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<div id="Signature">
<div style="font-family:Tahoma; font-size:13px"><img style="max-width: 100%; height: auto; user-select: none;" src="https://email.osu.edu/owa/attachment.ashx?id=RgAAAAAb%2fHy0wVvTSoHQx8OJXAaLBwCiA5IZrwRKTqiLVNbt4xWyAAAAAAFUAADltPc25wRDT4tJbW9en2wXAHKArVn%2fAAAJ&attcnt=1&attid0=EAD8Cse5Lj5uQ6ZJWk98Q%2blj"><br style="font-family:Calibri; font-size:15px">
<font face="Helvetica" size="2" color="#BB0000"><span style="font-size:9pt"><b>Anthony Shoup</b></span></font><font face="Helvetica" size="2" color="#333333"><span style="font-size:9pt"> PhD, Senior Lecturer<br>
</span></font><font face="Helvetica" size="2" color="#BB0000"><span style="font-size:9pt">College of Arts & Sciences, College of Engineering</span></font><font face="Helvetica" size="2" color="#333333"><span style="font-size:9pt"> Departments of Physics, Astronomy,
EEIC<br>
</span></font><font face="Helvetica" size="2" color="#333333"><span style="font-size:9pt">315 Science Bldg. | 4250 Campus Dr. Lima, OH 45807<br>
419-995-8018 Office | 419-516-2257 Mobile<br>
</span></font><a href="https://email.osu.edu/owa/redir.aspx?C=j5WpnJiBk0W5oVlCbtvB-xiCkA_lbdEIi9hlk7ByHiG7ARrxjwDFmAW8S_XespJbMLJRblY5JKc.&URL=mailto%3ashoup.31%40osu.edu" target="_blank" style="font-family:Calibri; font-size:15px" id="LPNoLP"><font face="Helvetica" size="2" color="blue"><span style="font-size:9pt"><b><u>shoup.31@osu.edu</u></b></span></font></a><font face="Helvetica" size="2" color="#333333"><span style="font-size:9pt"> </span></font><a href="https://email.osu.edu/owa/redir.aspx?C=j5WpnJiBk0W5oVlCbtvB-xiCkA_lbdEIi9hlk7ByHiG7ARrxjwDFmAW8S_XespJbMLJRblY5JKc.&URL=http%3a%2f%2fosu.edu" target="_blank" style="font-family:Calibri; font-size:15px" id="LPNoLP"><font face="Helvetica" size="2" color="blue"><span style="font-size:9pt"><b><u>osu.edu</u></b></span></font></a></div>
</div>
</div>
</body>
</html>