| Reporter: | Steven R. Brandt |
| Status: | new |
| Milestone: | |
| Version: | |
| Type: | bug |
| Priority: | major |
| Component: | SimFactory |
Comment (by Roland Haas):
SLURM’s task notion is site specific. Eg for Comet the SDSC docs https://www.sdsc.edu/support/user_guides/comet.html#running give for a hybrid MPI/OpenMP job:
#!/bin/bash #SBATCH --job-name="hellohybrid" #SBATCH --output="hellohybrid.%j.%N.out" #SBATCH --partition=compute #SBATCH --nodes=2 #SBATCH --ntasks-per-node=24 #SBATCH --export=ALL #SBATCH -t 01:30:00 #This job runs with 2 nodes, 24 cores per node for a total of 48 cores. # We use 8 MPI tasks and 6 OpenMP threads per MPI task export OMP_NUM_THREADS=6 ibrun --npernode 4 ./hello_hybrid
ie --tasks-per-node is the number of threads that will be started.
On Stampede2 on the other hand the docs at https://portal.tacc.utexas.edu/user-guides/stampede2#job-scripts have hybrid script that reads:
#SBATCH -J myjob # Job name #SBATCH -o myjob.o%j # Name of stdout output file #SBATCH -e myjob.e%j # Name of stderr error file #SBATCH -p skx-normal # Queue (partition) name #SBATCH -N 10 # Total # of nodes #SBATCH -n 40 # Total # of mpi tasks #SBATCH -t 01:30:00 # Run time (hh:mm:ss) #SBATCH --mail-user=username@tacc.utexas.edu #SBATCH --mail-type=all # Send email at begin and end of job #SBATCH -A myproject # Allocation name (req'd if you have mor--
Ticket URL: https://bitbucket.org/einsteintoolkit/tickets/issues/2482/inconsistency-in-simfactory-submission