<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 10/14/2025 3:55 AM, IOSIF PANAGIOTIS
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:GV1P250MB085752663EB611B705D6819EF9EBA@GV1P250MB0857.EURP250.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"
class="elementToProof">
Hi all,<br>
<br>
I am sending a reminder regarding two unanswered questions on
the mailing list, in case someone has a suggestion.<br>
<br>
</div>
<ol
data-editing-info="{"applyListStyleFromLevel":false,"orderedStyleType":3}"
style="margin-top: 0px; margin-bottom: 0px;" start="1">
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "1) ";">
<div role="presentation" class="elementToProof">Clarification
about how SimFactory handles the "--memory" option and how
this affects how one should navigate the cluster's billing
policy:</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "1) ";">
<div role="presentation" class="elementToProof"><a
id="LPlnk557853"
href="https://lists.einsteintoolkit.org/pipermail/users/2025-September/009761.html"
moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.einsteintoolkit.org/pipermail/users/2025-September/009761.html</a></div>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "1) ";">
<div role="presentation" class="elementToProof"><br>
</div>
</div>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "2) ";">
<div role="presentation" class="elementToProof">Using
'leonardo-dcgp.ini' and understanding how to properly
request one full node:</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "2) ";">
<div role="presentation" class="elementToProof"><a
id="LPlnk416851"
href="https://lists.einsteintoolkit.org/pipermail/users/2025-September/009762.html"
moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.einsteintoolkit.org/pipermail/users/2025-September/009762.html</a></div>
</div>
</li>
</ol>
</blockquote>
<p>Normally, one requests --procs equal to the number of cores on
the node.</p>
<p><br>
</p>
<p>So, imagine one has a machine with nodes that have 32 cores each.</p>
<p><br>
</p>
<p>One could say --procs 32, and that should be an entire node.
However, maybe you want to run with 8 threads per MPI task. In
that case, you would say --procs 32 --num-threads 8.</p>
<p><br>
</p>
<p>If you want to run on N nodes, then the number of procs would be
32*N, and Simfactory will figure it out.</p>
<p><br>
</p>
<p>--Steve</p>
<blockquote type="cite"
cite="mid:GV1P250MB085752663EB611B705D6819EF9EBA@GV1P250MB0857.EURP250.PROD.OUTLOOK.COM">
<ol
data-editing-info="{"applyListStyleFromLevel":false,"orderedStyleType":3}"
style="margin-top: 0px; margin-bottom: 0px;" start="1">
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "2) ";">
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); list-style-type: "2) ";">
</div>
<br>
</li>
</ol>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"
class="elementToProof">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"
class="elementToProof">
Thanks,</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"
class="elementToProof">
Panayotis</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif"
style="font-size:11pt" color="#000000"><b>From:</b> Users
<a class="moz-txt-link-rfc2396E" href="mailto:users-bounces@einsteintoolkit.org"><users-bounces@einsteintoolkit.org></a> on behalf of IOSIF
PANAGIOTIS <a class="moz-txt-link-rfc2396E" href="mailto:PANAGIOTIS.IOSIF@units.it"><PANAGIOTIS.IOSIF@units.it></a><br>
<b>Sent:</b> Monday, September 29, 2025 12:24 PM<br>
<b>To:</b> Roland Haas <a class="moz-txt-link-rfc2396E" href="mailto:rhaas@mail.ubc.ca"><rhaas@mail.ubc.ca></a>; Bruno
Giacomazzo <a class="moz-txt-link-rfc2396E" href="mailto:bruno.giacomazzo@unimib.it"><bruno.giacomazzo@unimib.it></a><br>
<b>Cc:</b> Einstein Toolkit Users
<a class="moz-txt-link-rfc2396E" href="mailto:users@einsteintoolkit.org"><users@einsteintoolkit.org></a><br>
<b>Subject:</b> Re: [Users] Inconsistency warnings:
cores/threads mismatch [Leonardo cluster]</font>
<div> </div>
</div>
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<div dir="ltr">
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Roland,<br>
<br>
Thanks for your reply.<br>
You touch on an important point, i.e the <b>cluster's</b> <b>billing
policy</b>, that hadn't crossed my mind.<br>
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
From the billing policy of Leonardo, it seems that <b>it is</b> <b>possible
to use only a fraction of a node's total CPUs.</b></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<a class="x_OWAAutoLink moz-txt-link-freetext"
id="LPlnk971574"
href="https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy"
moz-do-not-send="true">https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy</a></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>However</b>, the documentation also stresses that:</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<i>...if a job reserves all of a node’s RAM — even without
utilizing all its CPUs — the node becomes unusable for other
jobs and is therefore billed accordingly.<br>
<br>
</i></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
So, apart from the cores requested, <b>should I also try to
calculate the RAM requirements?</b></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
For example, I see that Bruno's "leonardo-dcgp.ini" file
specifies:</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<code>memory = 494000</code></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
And the respective submitscript also has this line:</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<code>#SBATCH --mem 494000MB</code></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I note that each node in Leonardo has 512GB of RAM, so that
means that <b>the script requests ~94.2% of the RAM.</b></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am not sure I follow the reasoning behind this.<br>
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
What is the default behavior of SimFactory if I were to remove
the above specifications from the config files?</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Because, if by default Simfactory requests/uses all the RAM
available in a node, then as far as I understand, it does not
make sense to request fewer cores than a full node.<br>
Let me know what you think.<br>
<br>
Best,<br>
Panayotis <br>
<br>
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font
face="Calibri, sans-serif" color="#000000"
style="font-size:11pt"><b>From:</b> Roland Haas
<a class="moz-txt-link-rfc2396E" href="mailto:rhaas@mail.ubc.ca"><rhaas@mail.ubc.ca></a><br>
<b>Sent:</b> Friday, September 26, 2025 4:31 PM<br>
<b>To:</b> Bruno Giacomazzo
<a class="moz-txt-link-rfc2396E" href="mailto:bruno.giacomazzo@unimib.it"><bruno.giacomazzo@unimib.it></a><br>
<b>Cc:</b> IOSIF PANAGIOTIS
<a class="moz-txt-link-rfc2396E" href="mailto:PANAGIOTIS.IOSIF@units.it"><PANAGIOTIS.IOSIF@units.it></a>; Einstein Toolkit Users
<a class="moz-txt-link-rfc2396E" href="mailto:users@einsteintoolkit.org"><users@einsteintoolkit.org></a><br>
<b>Subject:</b> Re: [Users] Inconsistency warnings:
cores/threads mismatch [Leonardo cluster]</font>
<div> </div>
</div>
<div class="x_BodyFragment"><font size="2"><span
style="font-size:11pt">
<div class="x_PlainText">Hello all,<br>
<br>
> I never used --cores and I don't know the
difference with procs.<br>
<br>
--cores is a synonym for --procs in simfactory. The hope
was to avoid<br>
the confusion of "procs" being "Processes" or
"Processors". Though it<br>
has been pointed out that the best name would actually
be "--threads"<br>
since that is what simfactory actually starts, which
then collides with<br>
"--num-threads" (threads per process).<br>
<br>
Does Leonardo actually charge you for partial nodes if
you do no use a<br>
full one? Simfactory is mostly written under the
assumption (true at<br>
the time) that HPC systems would give you full nodes all
the time, so<br>
if you use 1 core or 112 cores of a node, the charge
would be the same<br>
(though shared node systems are becoming more common for
HPC now [or<br>
again]).<br>
<br>
Yours,<br>
Roland<br>
<br>
-- <br>
My email is as private as my paper mail. I therefore
support encrypting<br>
and signing email messages. Get my PGP key from <a
href="http://pgp.mit.edu" moz-do-not-send="true"
class="moz-txt-link-freetext">http://pgp.mit.edu</a> .<br>
</div>
</span></font></div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre wrap="" class="moz-quote-pre">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@einsteintoolkit.org">Users@einsteintoolkit.org</a>
<a class="moz-txt-link-freetext" href="http://lists.einsteintoolkit.org/mailman/listinfo/users">http://lists.einsteintoolkit.org/mailman/listinfo/users</a>
</pre>
</blockquote>
</body>
</html>