On 10/14/2025 3:55 AM, IOSIF PANAGIOTIS wrote:
Hi all,
I am sending a reminder regarding two unanswered questions on the mailing list, in case someone has a suggestion.
Clarification about how SimFactory handles the "--memory" option and how this affects how one should navigate the cluster's billing policy: https://lists.einsteintoolkit.org/pipermail/users/2025-September/009761.html
Using 'leonardo-dcgp.ini' and understanding how to properly request one full node: https://lists.einsteintoolkit.org/pipermail/users/2025-September/009762.html
Normally, one requests --procs equal to the number of cores on the node.
So, imagine one has a machine with nodes that have 32 cores each.
One could say --procs 32, and that should be an entire node. However, maybe you want to run with 8 threads per MPI task. In that case, you would say --procs 32 --num-threads 8.
If you want to run on N nodes, then the number of procs would be 32*N, and Simfactory will figure it out.
--Steve
Thanks, Panayotis
*From:* Users users-bounces@einsteintoolkit.org on behalf of IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it *Sent:* Monday, September 29, 2025 12:24 PM *To:* Roland Haas rhaas@mail.ubc.ca; Bruno Giacomazzo bruno.giacomazzo@unimib.it *Cc:* Einstein Toolkit Users users@einsteintoolkit.org *Subject:* Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hi Roland,
Thanks for your reply. You touch on an important point, i.e the *cluster's* *billing policy*, that hadn't crossed my mind.
From the billing policy of Leonardo, it seems that *it is* *possible to use only a fraction of a node's total CPUs.* https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy
*However*, the documentation also stresses that: /...if a job reserves all of a node’s RAM — even without utilizing all its CPUs — the node becomes unusable for other jobs and is therefore billed accordingly.
/ So, apart from the cores requested, *should I also try to calculate the RAM requirements?* For example, I see that Bruno's "leonardo-dcgp.ini" file specifies: |memory = 494000| And the respective submitscript also has this line: |#SBATCH --mem 494000MB|
I note that each node in Leonardo has 512GB of RAM, so that means that *the script requests ~94.2% of the RAM.* I am not sure I follow the reasoning behind this.
What is the default behavior of SimFactory if I were to remove the above specifications from the config files? Because, if by default Simfactory requests/uses all the RAM available in a node, then as far as I understand, it does not make sense to request fewer cores than a full node. Let me know what you think.
Best, Panayotis
*From:* Roland Haas rhaas@mail.ubc.ca *Sent:* Friday, September 26, 2025 4:31 PM *To:* Bruno Giacomazzo bruno.giacomazzo@unimib.it *Cc:* IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it; Einstein Toolkit Users users@einsteintoolkit.org *Subject:* Re: [Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster] Hello all,
I never used --cores and I don't know the difference with procs.
--cores is a synonym for --procs in simfactory. The hope was to avoid the confusion of "procs" being "Processes" or "Processors". Though it has been pointed out that the best name would actually be "--threads" since that is what simfactory actually starts, which then collides with "--num-threads" (threads per process).
Does Leonardo actually charge you for partial nodes if you do no use a full one? Simfactory is mostly written under the assumption (true at the time) that HPC systems would give you full nodes all the time, so if you use 1 core or 112 cores of a node, the charge would be the same (though shared node systems are becoming more common for HPC now [or again]).
Yours, Roland
-- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users