Hi all,

I am sending a reminder regarding two unanswered questions on the mailing list, in case someone has a suggestion.

1. Clarification about how SimFactory handles the "--memory" option and how this affects how one should navigate the cluster's billing policy:
   https://lists.einsteintoolkit.org/pipermail/users/2025-September/009761.html
   
   
2. Using 'leonardo-dcgp.ini' and understanding how to properly request one full node:
   https://lists.einsteintoolkit.org/pipermail/users/2025-September/009762.html

Normally, one requests --procs equal to the number of cores on the node.

So, imagine one has a machine with nodes that have 32 cores each.

One could say --procs 32, and that should be an entire node. However, maybe you want to run with 8 threads per MPI task. In that case, you would say --procs 32 --num-threads 8.

If you want to run on N nodes, then the number of procs would be 32*N, and Simfactory will figure it out.

--Steve

1. 
   

Thanks,

Panayotis

---

From: Users <users-bounces@einsteintoolkit.org> on behalf of IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF@units.it>
Sent: Monday, September 29, 2025 12:24 PM
To: Roland Haas <rhaas@mail.ubc.ca>; Bruno Giacomazzo <bruno.giacomazzo@unimib.it>
Cc: Einstein Toolkit Users <users@einsteintoolkit.org>
Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster]

 

Hi Roland,

Thanks for your reply.
You touch on an important point, i.e the cluster's billing policy, that hadn't crossed my mind.

>From the billing policy of Leonardo, it seems that it is possible to use only a fraction of a node's total CPUs.

https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy

However, the documentation also stresses that:

...if a job reserves all of a node’s RAM — even without utilizing all its CPUs — the node becomes unusable for other jobs and is therefore billed accordingly.

So, apart from the cores requested, should I also try to calculate the RAM requirements?

For example, I see that Bruno's "leonardo-dcgp.ini" file specifies:

memory = 494000

And the respective submitscript also has this line:

#SBATCH --mem 494000MB

I note that each node in Leonardo has 512GB of RAM, so that means that the script requests ~94.2% of the RAM.

I am not sure I follow the reasoning behind this.

What is the default behavior of SimFactory if I were to remove the above specifications from the config files?

Because, if by default Simfactory requests/uses all the RAM available in a node, then as far as I understand, it does not make sense to request fewer cores than a full node.
Let me know what you think.

Best,
Panayotis 

---

From: Roland Haas <rhaas@mail.ubc.ca>
Sent: Friday, September 26, 2025 4:31 PM
To: Bruno Giacomazzo <bruno.giacomazzo@unimib.it>
Cc: IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF@units.it>; Einstein Toolkit Users <users@einsteintoolkit.org>
Subject: Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster]

 

Hello all,

> I never used --cores and I don't know the difference with procs.

--cores is a synonym for --procs in simfactory. The hope was to avoid
the confusion of "procs" being "Processes" or "Processors". Though it
has been pointed out that the best name would actually be "--threads"
since that is what simfactory actually starts, which then collides with
"--num-threads" (threads per process).

Does Leonardo actually charge you for partial nodes if you do no use a
full one? Simfactory is mostly written under the assumption (true at
the time) that HPC systems would give you full nodes all the time, so
if you use 1 core or 112 cores of a node, the charge would be the same
(though shared node systems are becoming more common for HPC now [or
again]).

Yours,
Roland

--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .

_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users