Hello Konrad,

To my understanding, it seems like the .cfg file has something wrong (
http://lists.einsteintoolkit.org/pipermail/users/2018-February/006103.html - i.e. the num-smt of your cluster should be 2, that should mean hyper threading - NOTE: I could be wrong with that). Moreover, you should use --procs=24 when submitting (from my experience this flag should be interpreted as the number of mpi task, that should correspond to the physical cores).

I am only providing my experience, but for sure, some more experienced users/developers could drive you to the most correct answer.

Regards,
Federico Cipolletta.

Il sab 3 lug 2021, 19:00 <users-request@einsteintoolkit.org> ha scritto:
Send Users mailing list submissions to
        users@einsteintoolkit.org

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.einsteintoolkit.org/mailman/listinfo/users
or, via email, send a message with subject or body 'help' to
        users-request@einsteintoolkit.org

You can reach the person managing the list at
        users-owner@einsteintoolkit.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Users digest..."


Today's Topics:

   1. Submitting a simulation via simfactory on a cluster
      (Konrad Topolski)


----------------------------------------------------------------------

Message: 1
Date: Sat, 3 Jul 2021 17:59:24 +0200
From: Konrad Topolski <k.topolski2@student.uw.edu.pl>
Subject: [Users] Submitting a simulation via simfactory on a cluster
To: users@einsteintoolkit.org
Message-ID:
        <CAEx5SMHM6GszcaqE3_L0Rvx5+U29D6STMYH2d0PW777acyqrRA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi

I am trying to get things right setting up simfactory on my cluster.
I have created the okeanos.ini and okeanos.cfg files, as well as built the
Cactus executable that's called cactus_okeanos. First I want to debug the
machine setup, then I hope to take care of the submission script.

To submit a job, I use:
simfactory/bin/sim create-submit GW150914_4 --parfile
par/GW150914/GW150914.rpar --walltime 24:00:00 --configuration=okeanos
--machine=okeanos

The error I am getting is the following:

Warning: Too many threads per process specified: specified num-threads=48
(ppn-used is 24)
Warning: Number of used cores per node, number of SMT threads, and number
of threads per process are inconsistent: ppn-used=24, num-smt=1,
num-threads=48_threads (ppn-used*num-smt must be an integer multiple of
num-threads)
Warning: Total number of threads and number of threads per process are
inconsistent: procs=1, num-threads=48 (procs*num-smt must be an integer
multiple of num-threads)
Warning: Total number of threads and number of cores per node are
inconsistent: procs=1, ppn-used=24 (procs must be an integer multiple of
ppn-used)

The relevant part of my machine .ini file looks like this:
cpu             = Intel Xeon E5-2690 v3
cpufreq         = 2.6
ppn             = 24   # or more
max-num-threads = 48   # or more
num-threads     = 48   # or more
nodes           = 1084
memory          = 131072           #any node has 128GB ram
min-ppn         = 24
maxwalltime     = 48:00:00

I know that each node in my cluster consists of 12 sockets for cpus, each
cpu has two cores and each core supports multithreading (=2), so it's
12x2x2 = 48.
What am I getting wrong?


Best regards
Konrad Topolski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.einsteintoolkit.org/pipermail/users/attachments/20210703/25fd9879/attachment-0001.html

------------------------------

_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


End of Users Digest, Vol 136, Issue 2
*************************************