Hello,
I am trying to run the static_tov simulation on my university's cluster (CRC Notre Dame https://docs.crc.nd.edu/new_user/quick_start.html). However, I got this error when acquiring 48 cores:
Error: Too many nodes specified: nodes=2 (maxnodes is 1)
I am assuming this is due to the *#Source tree management* module in the Cactus/simfactory/mdb/machines/<host>.ini file. In my case, the automatic *.ini file I got has these configuration:
ppn = 24
max-num-threads = 24
num-threads = 12
nodes = 1
However, the machine I use has 24 cores for each node, and the number of threads for each core is 1. That means if I run the simulation using 48 cores, I need 2 nodes.
Although I didn't get 2 hosts successfully(I got only 1), I am wondering if I have to edit the <host>.ini file after the command ./simfactory/bin/sim setup-silent by hand, for example: num-threads=1, nodes=2? Or is there any other correct way for getting more than 1 node?
I would be appreciate it if you could give me some ideas on this problem.
Best regards, Wei
If your ini file is not part of the simfactory checkout (i.e. we don't know about it), you may have to edit by hand. I think simfactory attempts to run lscpu to determine some of these parameters, but if it is not present 1 may be the default.
If you are part of a group, it may be beneficial to have your ini file checked in to simafactory (after it is debugged and tested) so that other members of your group have access to it. We can help make that happen.
--Steve
On 6/20/2023 3:01 PM, Wei Sun wrote:
Hello,
I am trying to run the static_tov simulation on my university's cluster (CRC Notre Dame https://docs.crc.nd.edu/new_user/quick_start.html). However, I got this error when acquiring 48 cores:
Error: Too many nodes specified: nodes=2 (maxnodes is 1)
I am assuming this is due to the /#Source tree management/ module in the Cactus/simfactory/mdb/machines/<host>.ini file. In my case, the automatic *.ini file I got has these configuration:
ppn = 24
max-num-threads = 24
num-threads = 12
nodes = 1
However, the machine I use has 24 cores for each node, and the number of threads for each core is 1. That means if I run the simulation using 48 cores, I need 2 nodes.
Although I didn't get 2 hosts successfully(I got only 1), I am wondering if I have to edit the <host>.ini file after the command ./simfactory/bin/sim setup-silent by hand, for example: num-threads=1, nodes=2? Or is there any other correct way for getting more than 1 node?
I would be appreciate it if you could give me some ideas on this problem.
Best regards, Wei
Users mailing list Users@einsteintoolkit.org http://lists.einsteintoolkit.org/mailman/listinfo/users
Hello Wei,
sorry for the long delay in responding.
The automated setup command `setup` or `setup-silent` is only designed to set up simfactory on a workstation, not a cluster.
Clusters are complex and individual enough so that a simple script does not work and you will have to manually adjust or set up things (eg the queuing system).
See eg:
https://www.einsteintoolkit.org/seminars/2022_02_24/index.html
and the wiki documentation
https://docs.einsteintoolkit.org/et-docs/Configuring_a_new_machine
Note that under the hood this all just uses your cluster's queuing system, and you can also "run your own" sbatch script using eg an example provided in the cluster's documentation:
--8<--
#!/bin/bash # various SBATCH or similar options
# eg 4 threads per MPI rank export OMP_NUM_THREADS=4
# 16 MPI ranks srun -n 16 --cpus-per-task $OMP_NUM_THREADS cactus_sim myparfile.par
--8<--
Yours, Roland
Hello,
I am trying to run the static_tov simulation on my university's cluster (CRC Notre Dame <https://urldefense.com/v3/__https://docs.crc.nd.edu/new_user/quick_start.htm... >). However, I got this error when acquiring 48 cores:
Error: Too many nodes specified: nodes=2 (maxnodes is 1)
I am assuming this is due to the *#Source tree management* module in the Cactus/simfactory/mdb/machines/<host>.ini file. In my case, the automatic *.ini file I got has these configuration:
ppn = 24
max-num-threads = 24
num-threads = 12
nodes = 1
However, the machine I use has 24 cores for each node, and the number of threads for each core is 1. That means if I run the simulation using 48 cores, I need 2 nodes.
Although I didn't get 2 hosts successfully(I got only 1), I am wondering if I have to edit the <host>.ini file after the command ./simfactory/bin/sim setup-silent by hand, for example: num-threads=1, nodes=2? Or is there any other correct way for getting more than 1 node?
I would be appreciate it if you could give me some ideas on this problem.
Best regards, Wei
Yours, Roland
users@lists.einsteintoolkit.org