Hi Wei and all,
Thank you for your suggestion.
The properties.ini file and the two relations you mentioned are very useful to check that what I specify as options when submitting the job is actually what I want.
Indeed, specifying at the same time e.g. --procs 32 --num-threads 32 --ppn-used 32
I can force the use of 32 cores, overriding the settings in the machine.ini file, without getting inconsistency warnings.
Therefore, it seems that if I am to leave the <machine>.ini file unchanged, all 3 options above are needed.
Now, taking also into account the cluster's billing policy (that Roland Haas flagged in his answer), I am trying to understand how best to formulate my resource requests.
So, I created a new simulation, not making any specific resource request via the command line, to see what happens with the default values in <machine.ini>, i.e: sim create-submit tov_test3 --parfile par/tov_ET.par --walltime 00:20:00
I have attached the respective properties.ini, .err and .out files.
Inspecting the files, do I understand correctly that only 56 threads/cores were used in this scenario?
Given that in the Leonardo cluster there are 112 cores per node (namely 2 x Intel® Xeon® Platinum 8480+ Processorhttps://www.intel.com/content/www/us/en/products/sku/231746/intel-xeon-platinum-8480-processor-105m-cache-2-00-ghz/specifications.html , each of which has 56 cores) does this mean that this way I am using only half the node?
Should I specify: --procs 112 --num-threads 112 to use one whole node in my case?
For reference, I have also attached the ini file that I am using (note that there we have the value num-threads=56). Does this make sense in the light of the above discussion?
Best regards, Panayotis
________________________________ From: Wei Sun wsun3@nd.edu Sent: Friday, September 26, 2025 4:58 PM To: IOSIF PANAGIOTIS PANAGIOTIS.IOSIF@units.it Cc: users@einsteintoolkit.org users@einsteintoolkit.org Subject: Re: Inconsistency warnings: cores/threads mismatch [Leonardo cluster]
Hi Panayotis,
Can you try specifying --procs, --num-threads, and --ppn-used at the same time? It looks like your machine defaulted to --ppn-used=112, which is not what you want, so you may need to overwrite it.
You can check your allocation details in /output-000x/SIMFACTORY/properties.ini. A useful strategy is to verify whether the allocation matches these relations:
ppn-used = numthreads * nodeprocs procs = numthreads * numprocs = nodes * ppn-used
Below is my understanding (I personally use numsmt=1, so threads = cores in my case). Please correct me if I’m wrong on them:
nodes = number of nodes procsrequested = total number of cores requested (may not all be used, but you still occupy them on the HPC) ppn = number of cores per node requested (may not all be used) numprocs = total number of MPI ranks actually used nodeprocs = number of MPI ranks per node actually used procs = total number of cores actually used numthreads = number of OpenMP threads per MPI rank actually used ppnused = number of cores per node actually used numsmt = number of threads per physical core
Best, Wei