[Users] Too many nodes specified: nodes=2 (maxnodes is 1)
Roland Haas
rhaas at illinois.edu
Thu Jan 21 09:28:55 CST 2021
Hello Karima,
summarizing the discussion in the call today:
* for --cores 4 --num-threads 2 the issue is that simulation factory
counts only "physical" cores while the OS reports "logical" cores
including the hyperthreading which we originally chose to ignore
* for the other error --cores 2 --num-threads 1 "The domain size for
map 0 scaled for convergence level 0 with convergence factor 2 is not
integer" would seem to be that the coarse grid spacing "dx" does not
evenly divide the grid size xmax - xmin. If the grid spacing is
indeed correct, as you indicated in the call, then best would be to
provide the exact parameter file that you ran.
Yours,
Roland
> Dear Dr. Ronald and Dr. Peter,
> I have a 4 core laptop, and I am using 'Simfactory' command to run the par
> file.
> I have tested both of your suggestions. The addition of "--cores=2
> --num-threads=1" gives me the attached error while the "--cores=4
> --num-threads=2"
> option gives, "Error: Too many nodes specified: nodes=2 (maxnodes is 1)
>
> Aborting Simfactory".
>
>
> Thanks
>
> Best,
>
> KS
>
> On Tue, Jan 19, 2021 at 11:29 PM <users-request at einsteintoolkit.org> wrote:
>
> > Send Users mailing list submissions to
> > users at einsteintoolkit.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$
> > or, via email, send a message with subject or body 'help' to
> > users-request at einsteintoolkit.org
> >
> > You can reach the person managing the list at
> > users-owner at einsteintoolkit.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Users digest..."
> >
> >
> > Today's Topics:
> >
> > 1. TAT/ Slab Error (KARIMA SHAHZAD)
> > 2. Re: TAT/ Slab Error (Peter Diener)
> > 3. Agenda for Thursday's Meeting (users at einsteintoolkit.org)
> > 4. Re: TAT/ Slab Error (Roland Haas)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Mon, 18 Jan 2021 17:41:04 +0000
> > From: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > Subject: [Users] TAT/ Slab Error
> > To: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > Message-ID:
> > <
> > DB7PR07MB5435B609DAF0209B60980C8E8CA40 at DB7PR07MB5435.eurprd07.prod.outlook.com
> > >
> >
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hello there,
> >
> > Here is the error attached I encountered while evolving black holes
> > binary. The work-around I guess is to increase the number of processes (if
> > I am not wrong), but I am not quite sure how to do that. I need some help
> > to get around this issue.
> >
> > Thanks in advance
> >
> > Best regards,
> > KS
> >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.html__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqqXuJnIl$
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: bbhHr.err
> > Type: application/octet-stream
> > Size: 1075 bytes
> > Desc: bbhHr.err
> > Url :
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.obj__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqre9Jkw5$
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Mon, 18 Jan 2021 14:19:02 -0600 (CST)
> > From: Peter Diener <diener at cct.lsu.edu>
> > Subject: Re: [Users] TAT/ Slab Error
> > To: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > Cc: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > Message-ID: <alpine.LSU.2.20.2101181408440.2879 at yvahk-8u91.fhfr>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hi Karima,
> >
> > Yes, that parameter file requires at least 2 mpi processes to run.
> >
> > You seem to have submitted using simfactory and ended up running on 1
> > mpi process awith 2 openmp threads. If you instead do something like
> >
> > simfactory/bin/sim run bbhHr --cores=2 --num-threads=1
> >
> > if you're on a workstation that doesn't use a queueing system or
> >
> > simfactory/bin/sim submit bbhHr --cores=2 --num-threads=1
> >
> > if you're on a cluster. This should give you 2 mpi processes running on
> > 1 openmp thread each. You might have to add other options to simfactory
> > if you did so in your original run.
> >
> > Cheers,
> >
> > Peter
> >
> > On Monday 2021-01-18 11:41, KARIMA SHAHZAD wrote:
> >
> > >Date: Mon, 18 Jan 2021 11:41:04
> > >From: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > >To: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > >Subject: [Users] TAT/ Slab Error
> > >
> > >Hello there,?
> > >
> > >Here is the error attached I encountered while evolving black holes
> > binary.
> > >The work-around I guess is to increase the number of processes (if I am
> > not
> > >wrong), but I am not quite sure how to do that. I need some help to get
> > >around this issue.
> > >
> > >Thanks in advance
> > >
> > >Best regards,
> > >KS
> > >
> > >
> > >
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Mon, 18 Jan 2021 15:18:01 -0600
> > From: users at einsteintoolkit.org
> > Subject: [Users] Agenda for Thursday's Meeting
> > To: users at einsteintoolkit.org
> > Message-ID: <E1l1buT-009sDh-Ev at ekohaes8>
> > Content-Type: text/plain; charset="ANSI_X3.4-1968"
> >
> > Please update the Wiki with agenda items for Thursday's meeting. Thanks!
> >
> > https://urldefense.com/v3/__https://docs.einsteintoolkit.org/et-docs/meeting_agenda__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqhEBI4Ov$
> >
> > --The Maintainers
> >
> >
> > ------------------------------
> >
> > Message: 4
> > Date: Mon, 18 Jan 2021 16:17:32 -0600
> > From: Roland Haas <rhaas at illinois.edu>
> > Subject: Re: [Users] TAT/ Slab Error
> > To: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > Cc: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > Message-ID: <20210118161732.71c09b22 at 8992d193.ncsa.illinois.edu>
> > Content-Type: text/plain; charset="us-ascii"
> >
> > Hello Karima,
> >
> > > Here is the error attached I encountered while evolving black holes
> > > binary. The work-around I guess is to increase the number of
> > > processes (if I am not wrong), but I am not quite sure how to do
> > > that. I need some help to get around this issue.
> > Assuming you use simulation factory you need to pass options to make
> > sure it creates multiple MPI processes. The auto-detection code tends
> > to choose a number of threads equal to the the number of cores on your
> > laptop and and a single MPI rank.
> >
> > Eg if you have a 4 core laptop then you have to use:
> >
> > ./simfactory/bin/sim submit testrun01 --cores 4 --num-threads 2
> > --parfile ...
> >
> > which uses a total of 4 cores and starts 2 threads per MPI rank so that
> > you end up with 4 / 2 = 2 MPI ranks each of which uses 2 threads.
> >
> > Similarly on clusters, the basic idea is to set --num-threads so that
> > there are more than 1 MPI rank started (ie --num-threads is larger than
> > the value for --cores or --procs [which are synonyms for each other]).
> >
> > If not using simulation factory you have to manually use mpirun and
> > OMP_NUM_THREADS. Eg:
> >
> > export OMP_NUM_THREADS=2
> >
> > mpirun -np 2 /home/karima/simulations/bbhHr/SIMFACTORY/exe/cactus_sim
> > -L 3 /home/karima/simulations/bbhHr/output-0000/BBHHigherRes.par
> >
> > which starts 2 MPI ranks each will use 2 OpenMP threads.
> >
> > Yours,
> > Roland
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqglVIuVi$ .
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: not available
> > Type: application/pgp-signature
> > Size: 833 bytes
> > Desc: OpenPGP digital signature
> > Url :
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/78ab7576/attachment-0001.bin__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqifXAHiE$
> >
> > ------------------------------
> >
> > _______________________________________________
> > Users mailing list
> > Users at einsteintoolkit.org
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$
> >
> >
> > End of Users Digest, Vol 130, Issue 18
> > **************************************
> >
--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20210121/0e328703/attachment.bin
More information about the Users
mailing list