<div dir="ltr"><div>Dear Dr. Ronald and Dr. Peter,</div><div>I have a 4 core laptop, and I am using 'Simfactory' command to run the par file.</div><div>I have tested both of your suggestions. The addition of "--cores=2 --num-threads=1" gives me the attached error while the "--cores=4 --num-threads=2" </div>option gives, "<span style="background-color:transparent;color:rgb(0,0,0);font-size:14px;white-space:pre-wrap">Error: Too many nodes specified: nodes=2 (maxnodes is 1)</span><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;background-color:transparent;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">Aborting Simfactory".</pre><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;background-color:transparent;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline"><br></pre><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;background-color:transparent;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">Thanks</pre><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;background-color:transparent;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">Best,</pre><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;background-color:transparent;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">KS</pre><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 19, 2021 at 11:29 PM <<a href="mailto:users-request@einsteintoolkit.org">users-request@einsteintoolkit.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send Users mailing list submissions to<br>
<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:users-request@einsteintoolkit.org" target="_blank">users-request@einsteintoolkit.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:users-owner@einsteintoolkit.org" target="_blank">users-owner@einsteintoolkit.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. TAT/ Slab Error (KARIMA SHAHZAD)<br>
2. Re: TAT/ Slab Error (Peter Diener)<br>
3. Agenda for Thursday's Meeting (<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>)<br>
4. Re: TAT/ Slab Error (Roland Haas)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Mon, 18 Jan 2021 17:41:04 +0000<br>
From: KARIMA SHAHZAD <<a href="mailto:02141911015@student.qau.edu.pk" target="_blank">02141911015@student.qau.edu.pk</a>><br>
Subject: [Users] TAT/ Slab Error<br>
To: "<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>" <<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>><br>
Message-ID:<br>
<<a href="mailto:DB7PR07MB5435B609DAF0209B60980C8E8CA40@DB7PR07MB5435.eurprd07.prod.outlook.com" target="_blank">DB7PR07MB5435B609DAF0209B60980C8E8CA40@DB7PR07MB5435.eurprd07.prod.outlook.com</a>><br>
<br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Hello there,<br>
<br>
Here is the error attached I encountered while evolving black holes binary. The work-around I guess is to increase the number of processes (if I am not wrong), but I am not quite sure how to do that. I need some help to get around this issue.<br>
<br>
Thanks in advance<br>
<br>
Best regards,<br>
KS<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.html</a> <br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: bbhHr.err<br>
Type: application/octet-stream<br>
Size: 1075 bytes<br>
Desc: bbhHr.err<br>
Url : <a href="http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.obj" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.obj</a> <br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Mon, 18 Jan 2021 14:19:02 -0600 (CST)<br>
From: Peter Diener <<a href="mailto:diener@cct.lsu.edu" target="_blank">diener@cct.lsu.edu</a>><br>
Subject: Re: [Users] TAT/ Slab Error<br>
To: KARIMA SHAHZAD <<a href="mailto:02141911015@student.qau.edu.pk" target="_blank">02141911015@student.qau.edu.pk</a>><br>
Cc: "<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>" <<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>><br>
Message-ID: <alpine.LSU.2.20.2101181408440.2879@yvahk-8u91.fhfr><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Hi Karima,<br>
<br>
Yes, that parameter file requires at least 2 mpi processes to run.<br>
<br>
You seem to have submitted using simfactory and ended up running on 1<br>
mpi process awith 2 openmp threads. If you instead do something like<br>
<br>
simfactory/bin/sim run bbhHr --cores=2 --num-threads=1<br>
<br>
if you're on a workstation that doesn't use a queueing system or<br>
<br>
simfactory/bin/sim submit bbhHr --cores=2 --num-threads=1<br>
<br>
if you're on a cluster. This should give you 2 mpi processes running on<br>
1 openmp thread each. You might have to add other options to simfactory<br>
if you did so in your original run.<br>
<br>
Cheers,<br>
<br>
Peter<br>
<br>
On Monday 2021-01-18 11:41, KARIMA SHAHZAD wrote:<br>
<br>
>Date: Mon, 18 Jan 2021 11:41:04<br>
>From: KARIMA SHAHZAD <<a href="mailto:02141911015@student.qau.edu.pk" target="_blank">02141911015@student.qau.edu.pk</a>><br>
>To: "<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>" <<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>><br>
>Subject: [Users] TAT/ Slab Error<br>
><br>
>Hello there,?<br>
><br>
>Here is the error attached I encountered while evolving black holes binary.<br>
>The work-around I guess is to increase the number of processes (if I am not<br>
>wrong), but I am not quite sure how to do that. I need some help to get<br>
>around this issue.<br>
><br>
>Thanks in advance<br>
><br>
>Best regards,<br>
>KS<br>
><br>
><br>
><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Mon, 18 Jan 2021 15:18:01 -0600<br>
From: <a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a><br>
Subject: [Users] Agenda for Thursday's Meeting<br>
To: <a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a><br>
Message-ID: <E1l1buT-009sDh-Ev@ekohaes8><br>
Content-Type: text/plain; charset="ANSI_X3.4-1968"<br>
<br>
Please update the Wiki with agenda items for Thursday's meeting. Thanks!<br>
<br>
<a href="https://docs.einsteintoolkit.org/et-docs/meeting_agenda" rel="noreferrer" target="_blank">https://docs.einsteintoolkit.org/et-docs/meeting_agenda</a><br>
<br>
--The Maintainers<br>
<br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Mon, 18 Jan 2021 16:17:32 -0600<br>
From: Roland Haas <<a href="mailto:rhaas@illinois.edu" target="_blank">rhaas@illinois.edu</a>><br>
Subject: Re: [Users] TAT/ Slab Error<br>
To: KARIMA SHAHZAD <<a href="mailto:02141911015@student.qau.edu.pk" target="_blank">02141911015@student.qau.edu.pk</a>><br>
Cc: "<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>" <<a href="mailto:users@einsteintoolkit.org" target="_blank">users@einsteintoolkit.org</a>><br>
Message-ID: <<a href="mailto:20210118161732.71c09b22@8992d193.ncsa.illinois.edu" target="_blank">20210118161732.71c09b22@8992d193.ncsa.illinois.edu</a>><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
Hello Karima,<br>
<br>
> Here is the error attached I encountered while evolving black holes<br>
> binary. The work-around I guess is to increase the number of<br>
> processes (if I am not wrong), but I am not quite sure how to do<br>
> that. I need some help to get around this issue.<br>
Assuming you use simulation factory you need to pass options to make<br>
sure it creates multiple MPI processes. The auto-detection code tends<br>
to choose a number of threads equal to the the number of cores on your<br>
laptop and and a single MPI rank.<br>
<br>
Eg if you have a 4 core laptop then you have to use:<br>
<br>
./simfactory/bin/sim submit testrun01 --cores 4 --num-threads 2<br>
--parfile ...<br>
<br>
which uses a total of 4 cores and starts 2 threads per MPI rank so that<br>
you end up with 4 / 2 = 2 MPI ranks each of which uses 2 threads.<br>
<br>
Similarly on clusters, the basic idea is to set --num-threads so that<br>
there are more than 1 MPI rank started (ie --num-threads is larger than<br>
the value for --cores or --procs [which are synonyms for each other]).<br>
<br>
If not using simulation factory you have to manually use mpirun and<br>
OMP_NUM_THREADS. Eg:<br>
<br>
export OMP_NUM_THREADS=2<br>
<br>
mpirun -np 2 /home/karima/simulations/bbhHr/SIMFACTORY/exe/cactus_sim<br>
-L 3 /home/karima/simulations/bbhHr/output-0000/BBHHigherRes.par<br>
<br>
which starts 2 MPI ranks each will use 2 OpenMP threads.<br>
<br>
Yours,<br>
Roland<br>
<br>
-- <br>
My email is as private as my paper mail. I therefore support encrypting<br>
and signing email messages. Get my PGP key from <a href="http://pgp.mit.edu" rel="noreferrer" target="_blank">http://pgp.mit.edu</a> .<br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: not available<br>
Type: application/pgp-signature<br>
Size: 833 bytes<br>
Desc: OpenPGP digital signature<br>
Url : <a href="http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/78ab7576/attachment-0001.bin" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/78ab7576/attachment-0001.bin</a> <br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@einsteintoolkit.org" target="_blank">Users@einsteintoolkit.org</a><br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
<br>
<br>
End of Users Digest, Vol 130, Issue 18<br>
**************************************<br>
</blockquote></div></div>