Hi
I am trying to get things right setting up simfactory on my cluster. I have created the okeanos.ini and okeanos.cfg files, as well as built the Cactus executable that's called cactus_okeanos. First I want to debug the machine setup, then I hope to take care of the submission script.
To submit a job, I use: simfactory/bin/sim create-submit GW150914_4 --parfile par/GW150914/GW150914.rpar --walltime 24:00:00 --configuration=okeanos --machine=okeanos
The error I am getting is the following:
Warning: Too many threads per process specified: specified num-threads=48 (ppn-used is 24) Warning: Number of used cores per node, number of SMT threads, and number of threads per process are inconsistent: ppn-used=24, num-smt=1, num-threads=48_threads (ppn-used*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of threads per process are inconsistent: procs=1, num-threads=48 (procs*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of cores per node are inconsistent: procs=1, ppn-used=24 (procs must be an integer multiple of ppn-used)
The relevant part of my machine .ini file looks like this: cpu = Intel Xeon E5-2690 v3 cpufreq = 2.6 ppn = 24 # or more max-num-threads = 48 # or more num-threads = 48 # or more nodes = 1084 memory = 131072 #any node has 128GB ram min-ppn = 24 maxwalltime = 48:00:00
I know that each node in my cluster consists of 12 sockets for cpus, each cpu has two cores and each core supports multithreading (=2), so it's 12x2x2 = 48. What am I getting wrong?
Best regards Konrad Topolski
Hello Konrad,
the quickest solution is likely to change your ini file to have:
ppn = 48
which tells simfactory that there are 48 cores per node and it will be happy. This denies simfactory any knowledge of the distinction between hyperthreads and "real" cores but should be just fine.
Alternatively (much less tested), you can add:
max-num-smt = 2 smt = 2
to the ini file which tells simfactory that each hardware core supports 2 hyperthreads and to use 2 hyperthreads by default.
Note that I have not tested any of this. If you are looking for some (somewhat) tested examples, you can search the existing ini files for "max-num-smt" (summit.ini should match for example).
Yours, Roland
Hi
I am trying to get things right setting up simfactory on my cluster. I have created the okeanos.ini and okeanos.cfg files, as well as built the Cactus executable that's called cactus_okeanos. First I want to debug the machine setup, then I hope to take care of the submission script.
To submit a job, I use: simfactory/bin/sim create-submit GW150914_4 --parfile par/GW150914/GW150914.rpar --walltime 24:00:00 --configuration=okeanos --machine=okeanos
The error I am getting is the following:
Warning: Too many threads per process specified: specified num-threads=48 (ppn-used is 24) Warning: Number of used cores per node, number of SMT threads, and number of threads per process are inconsistent: ppn-used=24, num-smt=1, num-threads=48_threads (ppn-used*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of threads per process are inconsistent: procs=1, num-threads=48 (procs*num-smt must be an integer multiple of num-threads) Warning: Total number of threads and number of cores per node are inconsistent: procs=1, ppn-used=24 (procs must be an integer multiple of ppn-used)
The relevant part of my machine .ini file looks like this: cpu = Intel Xeon E5-2690 v3 cpufreq = 2.6 ppn = 24 # or more max-num-threads = 48 # or more num-threads = 48 # or more nodes = 1084 memory = 131072 #any node has 128GB ram min-ppn = 24 maxwalltime = 48:00:00
I know that each node in my cluster consists of 12 sockets for cpus, each cpu has two cores and each core supports multithreading (=2), so it's 12x2x2 = 48. What am I getting wrong?
Best regards Konrad Topolski
users@lists.einsteintoolkit.org
-
Konrad Topolski -
Roland Haas