Hello everyone!
I’m new to the ET. I have the ET installed on my desktop. It has one Intel(R) Core(TM) i5-10500 CPU with 6 physical cores and two threads per single physical core (but according to the Ubuntu’s conventions, the computer has 12 processors in total, as shown in the attachment of this email) and it also has Ubuntu 20.4. So I wonder whether you could please tell me how to set the correct parallel parameters for ET simulations, such as the options of the submit command ./simfactory/bin/sim (--cores, --num-threads and –procs) and the initial values of the file <username>.ini (pn, max-num-threads, num-threads, nodes).
Up till now, I have tried many possible combinations without success. It usually returns messages like “Warning: Too many threads per process specified: specified num-threads=16 (ppn-used is 1)”, “Warning: Number of used cores per node, number of SMT threads, and number of threads per process are inconsistent: ppn-used=1, num-smt=1, num-threads=16_threads (ppn-used*num-smt must be an integer multiple of num-threads)” and “Warning: Total number of threads and number of threads per process are inconsistent: procs=1, num-threads=16 (procs*num-smt must be an integer multiple of num-threads)” and doesn't start running.
I know parallelization plays a very important role in massive numerical computations like the ET. However, I’m very confused by many of the concepts it uses. I don’t find a documentation for these parameters after browsing websites (like https://bitbucket.org/simfactory/simfactory2/src/master/doc/userguide/, https://simfactory.org/info/documentation/, http://einsteintoolkit.org/usersguide/UsersGuide.html or even in the source code in the directory “./simfactory/lib/”). There are merely some hints in the jupyter tutorial “CactusTutorial.ipynb”.
Kuarks
Hello Kuark,
Typically we see no benefit using hyperthreads due the nature of the workload of our scientific codes.
So I will only describe things without hyperthreads (which mostly just add a complications).
With that in mind simfactory's './simfactory/bin/sim setup-silent' should do a good job at setting up a default machine description.
When you submit you should set ppn-used to the number of compute cores you will use, one per OS thread. So with 16 threads you would use ppn-used to 16 as well.
If you are looking to get most speed on a typical workstation (single node) then most likely you fastest way is actually to use 1 thread per MPI rank and as many MPI ranks as you have physical cores.
So something like:
--cores 6 --num-threads 1 --ppn-used 6
in your case.
Yours, Roland
Hello everyone!
I’m new to the ET. I have the ET installed on my desktop. It has one Intel(R) Core(TM) i5-10500 CPU with 6 physical cores and two threads per single physical core (but according to the Ubuntu’s conventions, the computer has 12 processors in total, as shown in the attachment of this email) and it also has Ubuntu 20.4. So I wonder whether you could please tell me how to set the correct parallel parameters for ET simulations, such as the options of the submit command ./simfactory/bin/sim (--cores, --num-threads and –procs) and the initial values of the file <username>.ini (pn, max-num-threads, num-threads, nodes).
Up till now, I have tried many possible combinations without success. It usually returns messages like “Warning: Too many threads per process specified: specified num-threads=16 (ppn-used is 1)”, “Warning: Number of used cores per node, number of SMT threads, and number of threads per process are inconsistent: ppn-used=1, num-smt=1, num-threads=16_threads (ppn-used*num-smt must be an integer multiple of num-threads)” and “Warning: Total number of threads and number of threads per process are inconsistent: procs=1, num-threads=16 (procs*num-smt must be an integer multiple of num-threads)” and doesn't start running.
I know parallelization plays a very important role in massive numerical computations like the ET. However, I’m very confused by many of the concepts it uses. I don’t find a documentation for these parameters after browsing websites (like https://urldefense.com/v3/__https://bitbucket.org/simfactory/simfactory2/src... , https://urldefense.com/v3/__https://simfactory.org/info/documentation/__;!!D... , https://urldefense.com/v3/__http://einsteintoolkit.org/usersguide/UsersGuide... or even in the source code in the directory “./simfactory/lib/”). There are merely some hints in the jupyter tutorial “CactusTutorial.ipynb”.
Kuarks
users@lists.einsteintoolkit.org