Hi Wei and all,

Thank you for your suggestion.

The properties.ini file and the two relations you mentioned are very useful to check that what I specify as options when submitting the job is actually what I want.

Indeed, specifying at the same time e.g.
--procs 32

I can force the use of 32 cores, overriding the settings in the machine.ini file, without getting inconsistency warnings.

Therefore, it seems that if I am to leave the <machine>.ini file unchanged, all 3 options above are needed.

Now, taking also into account the cluster's billing policy (that Roland Haas flagged in his answer), I am trying to understand how best to formulate my resource requests.

So, I created a new simulation, not making any specific resource request via the command line, to see what happens with the default values in <machine.ini>, i.e:

sim create-submit tov_test3 --parfile par/tov_ET.par --walltime 00:20:00

I have attached the respective properties.ini, .err and .out files.

Inspecting the files, do I understand correctly that only 56 threads/cores were used in this scenario?

Given that in the Leonardo cluster there are 112 cores per node (namely 2 x Intel® Xeon® Platinum 8480+ Processor , each of which has 56 cores) does this mean that this way I am using only half the node?

Should I specify:

--procs 112 --num-threads 112

to use one whole node in my case?

For reference, I have also attached the ini file that I am using (note that there we have the value num-threads=56).
Does this make sense in the light of the above discussion?

Best regards,
Panayotis

From: Wei Sun <wsun3@nd.edu>
Sent: Friday, September 26, 2025 4:58 PM
To: IOSIF PANAGIOTIS <PANAGIOTIS.IOSIF@units.it>
Cc: users@einsteintoolkit.org <users@einsteintoolkit.org>
Subject: Re: Inconsistency warnings: cores/threads mismatch [Leonardo cluster]

Can you try specifying --procs, --num-threads, and


--ppn-used

at the same time? It looks like your machine defaulted to


--ppn-used=112

, which is not what you want, so you may need to overwrite it.

You can check your allocation details in

/output-000x/SIMFACTORY/properties.ini.

A useful strategy is to verify whether the allocation matches these relations:

Below is my understanding (I personally use numsmt=1, so threads = cores in my case). Please correct me if I’m wrong on them:

nodes = number of nodes
procsrequested = total number of cores requested (may not all be used, but you still occupy them on the HPC)
ppn = number of cores per node requested (may not all be used)
numprocs = total number of MPI ranks actually used
nodeprocs = number of MPI ranks per node actually used
procs = total number of cores actually used
numthreads = number of OpenMP threads per MPI rank actually used
ppnused = number of cores per node actually used
numsmt = number of threads per physical core