[Users] Inconsistency warnings: cores/threads mismatch [Leonardo cluster]

[Steven Brandt]

On 10/14/2025 3:55 AM, IOSIF PANAGIOTIS wrote:
> Hi all,
>
> I am sending a reminder regarding two unanswered questions on the 
> mailing list, in case someone has a suggestion.
>
> 1.
>     Clarification about how SimFactory handles the "--memory" option
>     and how this affects how one should navigate the cluster's billing
>     policy:
>     https://lists.einsteintoolkit.org/pipermail/users/2025-September/009761.html
>
> 2.
>     Using 'leonardo-dcgp.ini' and understanding how to properly
>     request one full node:
>     https://lists.einsteintoolkit.org/pipermail/users/2025-September/009762.html
>
Normally, one requests --procs equal to the number of cores on the node.


So, imagine one has a machine with nodes that have 32 cores each.


One could say --procs 32, and that should be an entire node. However, 
maybe you want to run with 8 threads per MPI task. In that case, you 
would say --procs 32 --num-threads 8.


If you want to run on N nodes, then the number of procs would be 32*N, 
and Simfactory will figure it out.


--Steve

> 1.
>
>
> Thanks,
> Panayotis
> ------------------------------------------------------------------------
> *From:* Users <users-bounces at einsteintoolkit.org> on behalf of IOSIF 
> PANAGIOTIS <PANAGIOTIS.IOSIF at units.it>
> *Sent:* Monday, September 29, 2025 12:24 PM
> *To:* Roland Haas <rhaas at mail.ubc.ca>; Bruno Giacomazzo 
> <bruno.giacomazzo at unimib.it>
> *Cc:* Einstein Toolkit Users <users at einsteintoolkit.org>
> *Subject:* Re: [Users] Inconsistency warnings: cores/threads mismatch 
> [Leonardo cluster]
> Hi Roland,
>
> Thanks for your reply.
> You touch on an important point, i.e the *cluster's* *billing policy*, 
> that hadn't crossed my mind.
>
> From the billing policy of Leonardo, it seems that *it is* *possible 
> to use only a fraction of a node's total CPUs.*
> https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy 
> <https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy>
>
> *However*, the documentation also stresses that:
> /...if a job reserves all of a node’s RAM — even without utilizing all 
> its CPUs — the node becomes unusable for other jobs and is therefore 
> billed accordingly.
>
> /
> So, apart from the cores requested, *should I also try to calculate 
> the RAM requirements?*
> For example, I see that Bruno's "leonardo-dcgp.ini" file specifies:
> |memory = 494000|
> And the respective submitscript also has this line:
> |#SBATCH --mem 494000MB|
>
> I note that each node in Leonardo has 512GB of RAM, so that means that 
> *the script requests ~94.2% of the RAM.*
> I am not sure I follow the reasoning behind this.
>
> What is the default behavior of SimFactory if I were to remove the 
> above specifications from the config files?
> Because, if by default Simfactory requests/uses all the RAM available 
> in a node, then as far as I understand, it does not make sense to 
> request fewer cores than a full node.
> Let me know what you think.
>
> Best,
> Panayotis
>
>
> ------------------------------------------------------------------------
> *From:* Roland Haas <rhaas at mail.ubc.ca>
> *Sent:* Friday, September 26, 2025 4:31 PM
> *To:* Bruno Giacomazzo <bruno.giacomazzo at unimib.it>
> *Cc:* IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF at units.it>; Einstein Toolkit 
> Users <users at einsteintoolkit.org>
> *Subject:* Re: [Users] Inconsistency warnings: cores/threads mismatch 
> [Leonardo cluster]
> Hello all,
>
> > I never used --cores and I don't know the difference with procs.
>
> --cores is a synonym for --procs in simfactory. The hope was to avoid
> the confusion of "procs" being "Processes" or "Processors". Though it
> has been pointed out that the best name would actually be "--threads"
> since that is what simfactory actually starts, which then collides with
> "--num-threads" (threads per process).
>
> Does Leonardo actually charge you for partial nodes if you do no use a
> full one? Simfactory is mostly written under the assumption (true at
> the time) that HPC systems would give you full nodes all the time, so
> if you use 1 core or 112 cores of a node, the charge would be the same
> (though shared node systems are becoming more common for HPC now [or
> again]).
>
> Yours,
> Roland
>
> -- 
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20251023/c577c95a/attachment-0001.htm>