On 10/14/2025 3:55 AM, IOSIF PANAGIOTIS wrote:
Hi all,

I am sending a reminder regarding two unanswered questions on the mailing list, in case someone has a suggestion.

Normally, one requests --procs equal to the number of cores on the node.


So, imagine one has a machine with nodes that have 32 cores each.


One could say --procs 32, and that should be an entire node. However, maybe you want to run with 8 threads per MPI task. In that case, you would say --procs 32 --num-threads 8.


If you want to run on N nodes, then the number of procs would be 32*N, and Simfactory will figure it out.


--Steve



Thanks,
Panayotis

From: Users <users-bounces@einsteintoolkit.org> on behalf of IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF@units.it>
Sent: Monday, September 29, 2025 12:24 PM
To: Roland Haas <rhaas@mail.ubc.ca>; Bruno Giacomazzo <bruno.giacomazzo@unimib.it>
Cc: Einstein Toolkit Users <users@einsteintoolkit.org>
Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster]
 
Hi Roland,

Thanks for your reply.
You touch on an important point, i.e the cluster's billing policy, that hadn't crossed my mind.

From the billing policy of Leonardo, it seems that it is possible to use only a fraction of a node's total CPUs.

However, the documentation also stresses that:
...if a job reserves all of a node’s RAM — even without utilizing all its CPUs — the node becomes unusable for other jobs and is therefore billed accordingly.

So, apart from the cores requested, should I also try to calculate the RAM requirements?
For example, I see that Bruno's "leonardo-dcgp.ini" file specifies:
memory = 494000
And the respective submitscript also has this line:
#SBATCH --mem 494000MB

I note that each node in Leonardo has 512GB of RAM, so that means that the script requests ~94.2% of the RAM.
I am not sure I follow the reasoning behind this.

What is the default behavior of SimFactory if I were to remove the above specifications from the config files?
Because, if by default Simfactory requests/uses all the RAM available in a node, then as far as I understand, it does not make sense to request fewer cores than a full node.
Let me know what you think.

Best,
Panayotis 



From: Roland Haas <rhaas@mail.ubc.ca>
Sent: Friday, September 26, 2025 4:31 PM
To: Bruno Giacomazzo <bruno.giacomazzo@unimib.it>
Cc: IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF@units.it>; Einstein Toolkit Users <users@einsteintoolkit.org>
Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster]
 
Hello all,

> I never used --cores and I don't know the difference with procs.

--cores is a synonym for --procs in simfactory. The hope was to avoid
the confusion of "procs" being "Processes" or "Processors". Though it
has been pointed out that the best name would actually be "--threads"
since that is what simfactory actually starts, which then collides with
"--num-threads" (threads per process).

Does Leonardo actually charge you for partial nodes if you do no use a
full one? Simfactory is mostly written under the assumption (true at
the time) that HPC systems would give you full nodes all the time, so
if you use 1 core or 112 cores of a node, the charge would be the same
(though shared node systems are becoming more common for HPC now [or
again]).

Yours,
Roland

--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .

_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users