[Users] Submitting a simulation via simfactory on a cluster

[Roland Haas]

Hello Konrad,

the quickest solution is likely to change your ini file to have:

ppn = 48

which tells simfactory that there are 48 cores per node and it will be
happy. This denies simfactory any knowledge of the distinction between
hyperthreads and "real" cores but should be just fine.

Alternatively (much less tested), you can add:

max-num-smt = 2
smt = 2

to the ini file which tells simfactory that each hardware core supports
2 hyperthreads and to use 2 hyperthreads by default.

Note that I have not tested any of this. If you are looking for some
(somewhat) tested examples, you can search the existing ini files for
"max-num-smt" (summit.ini should match for example).

Yours,
Roland

> Hi
> 
> I am trying to get things right setting up simfactory on my cluster.
> I have created the okeanos.ini and okeanos.cfg files, as well as built the
> Cactus executable that's called cactus_okeanos. First I want to debug the
> machine setup, then I hope to take care of the submission script.
> 
> To submit a job, I use:
> simfactory/bin/sim create-submit GW150914_4 --parfile
> par/GW150914/GW150914.rpar --walltime 24:00:00 --configuration=okeanos
> --machine=okeanos
> 
> The error I am getting is the following:
> 
> Warning: Too many threads per process specified: specified num-threads=48
> (ppn-used is 24)
> Warning: Number of used cores per node, number of SMT threads, and number
> of threads per process are inconsistent: ppn-used=24, num-smt=1,
> num-threads=48_threads (ppn-used*num-smt must be an integer multiple of
> num-threads)
> Warning: Total number of threads and number of threads per process are
> inconsistent: procs=1, num-threads=48 (procs*num-smt must be an integer
> multiple of num-threads)
> Warning: Total number of threads and number of cores per node are
> inconsistent: procs=1, ppn-used=24 (procs must be an integer multiple of
> ppn-used)
> 
> The relevant part of my machine .ini file looks like this:
> cpu             = Intel Xeon E5-2690 v3
> cpufreq         = 2.6
> ppn             = 24   # or more
> max-num-threads = 48   # or more
> num-threads     = 48   # or more
> nodes           = 1084
> memory          = 131072           #any node has 128GB ram
> min-ppn         = 24
> maxwalltime     = 48:00:00
> 
> I know that each node in my cluster consists of 12 sockets for cpus, each
> cpu has two cores and each core supports multithreading (=2), so it's
> 12x2x2 = 48.
> What am I getting wrong?
> 
> 
> Best regards
> Konrad Topolski


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20210703/8b94928d/attachment.bin