<div dir="auto"><div>Hello Konrad,</div><div dir="auto"><br></div><div dir="auto">To my understanding, it seems like the .cfg file has something wrong (<br><a href="http://lists.einsteintoolkit.org/pipermail/users/2018-February/006103.html">http://lists.einsteintoolkit.org/pipermail/users/2018-February/006103.html</a> - i.e. the num-smt of your cluster should be 2, that should mean hyper threading - NOTE: I could be wrong with that). Moreover, you should use --procs=24 when submitting (from my experience this flag should be interpreted as the number of mpi task, that should correspond to the physical cores).</div><div dir="auto"><br></div><div dir="auto">I am only providing my experience, but for sure, some more experienced users/developers could drive you to the most correct answer.</div><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Federico Cipolletta.</div><div dir="auto"><br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">Il sab 3 lug 2021, 19:00 <<a href="mailto:users-request@einsteintoolkit.org">users-request@einsteintoolkit.org</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send Users mailing list submissions to<br>
<a href="mailto:users@einsteintoolkit.org" target="_blank" rel="noreferrer">users@einsteintoolkit.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:users-request@einsteintoolkit.org" target="_blank" rel="noreferrer">users-request@einsteintoolkit.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:users-owner@einsteintoolkit.org" target="_blank" rel="noreferrer">users-owner@einsteintoolkit.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Submitting a simulation via simfactory on a cluster<br>
(Konrad Topolski)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Sat, 3 Jul 2021 17:59:24 +0200<br>
From: Konrad Topolski <<a href="mailto:k.topolski2@student.uw.edu.pl" target="_blank" rel="noreferrer">k.topolski2@student.uw.edu.pl</a>><br>
Subject: [Users] Submitting a simulation via simfactory on a cluster<br>
To: <a href="mailto:users@einsteintoolkit.org" target="_blank" rel="noreferrer">users@einsteintoolkit.org</a><br>
Message-ID:<br>
<<a href="mailto:CAEx5SMHM6GszcaqE3_L0Rvx5%2BU29D6STMYH2d0PW777acyqrRA@mail.gmail.com" target="_blank" rel="noreferrer">CAEx5SMHM6GszcaqE3_L0Rvx5+U29D6STMYH2d0PW777acyqrRA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi<br>
<br>
I am trying to get things right setting up simfactory on my cluster.<br>
I have created the okeanos.ini and okeanos.cfg files, as well as built the<br>
Cactus executable that's called cactus_okeanos. First I want to debug the<br>
machine setup, then I hope to take care of the submission script.<br>
<br>
To submit a job, I use:<br>
simfactory/bin/sim create-submit GW150914_4 --parfile<br>
par/GW150914/GW150914.rpar --walltime 24:00:00 --configuration=okeanos<br>
--machine=okeanos<br>
<br>
The error I am getting is the following:<br>
<br>
Warning: Too many threads per process specified: specified num-threads=48<br>
(ppn-used is 24)<br>
Warning: Number of used cores per node, number of SMT threads, and number<br>
of threads per process are inconsistent: ppn-used=24, num-smt=1,<br>
num-threads=48_threads (ppn-used*num-smt must be an integer multiple of<br>
num-threads)<br>
Warning: Total number of threads and number of threads per process are<br>
inconsistent: procs=1, num-threads=48 (procs*num-smt must be an integer<br>
multiple of num-threads)<br>
Warning: Total number of threads and number of cores per node are<br>
inconsistent: procs=1, ppn-used=24 (procs must be an integer multiple of<br>
ppn-used)<br>
<br>
The relevant part of my machine .ini file looks like this:<br>
cpu = Intel Xeon E5-2690 v3<br>
cpufreq = 2.6<br>
ppn = 24 # or more<br>
max-num-threads = 48 # or more<br>
num-threads = 48 # or more<br>
nodes = 1084<br>
memory = 131072 #any node has 128GB ram<br>
min-ppn = 24<br>
maxwalltime = 48:00:00<br>
<br>
I know that each node in my cluster consists of 12 sockets for cpus, each<br>
cpu has two cores and each core supports multithreading (=2), so it's<br>
12x2x2 = 48.<br>
What am I getting wrong?<br>
<br>
<br>
Best regards<br>
Konrad Topolski<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.einsteintoolkit.org/pipermail/users/attachments/20210703/25fd9879/attachment-0001.html" rel="noreferrer noreferrer" target="_blank">http://lists.einsteintoolkit.org/pipermail/users/attachments/20210703/25fd9879/attachment-0001.html</a> <br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@einsteintoolkit.org" target="_blank" rel="noreferrer">Users@einsteintoolkit.org</a><br>
<a href="http://lists.einsteintoolkit.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">http://lists.einsteintoolkit.org/mailman/listinfo/users</a><br>
<br>
<br>
End of Users Digest, Vol 136, Issue 2<br>
*************************************<br>
</blockquote></div></div></div>