The error was due to an incorrect par file setting. "---core= 2, thread= 1--" worked, thank you very much for the support.
Best,
Karima S
Send Users mailing list submissions to
users@einsteintoolkit.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.einsteintoolkit.org/mailman/listinfo/users
or, via email, send a message with subject or body 'help' to
users-request@einsteintoolkit.org
You can reach the person managing the list at
users-owner@einsteintoolkit.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Users digest..."
Today's Topics:
1. Einstein Toolkit Meeting Reminder (users@einsteintoolkit.org)
2. Re: Too many nodes specified: nodes=2 (maxnodes is 1)
(Roland Haas)
3. Meeting Minutes 2021-01-21 (Steven R. Brandt)
4. Re: Meeting Minutes 2021-01-21 (Roland Haas)
----------------------------------------------------------------------
Message: 1
Date: Wed, 20 Jan 2021 17:15:01 -0600
From: users@einsteintoolkit.org
Subject: [Users] Einstein Toolkit Meeting Reminder
To: users@einsteintoolkit.org
Message-ID: <E1l2Mgn-00Bybi-Hb@ekohaes8>
Content-Type: text/plain; charset="ANSI_X3.4-1968"
Hello,
Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.
https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call
--The Maintainers
------------------------------
Message: 2
Date: Thu, 21 Jan 2021 09:28:55 -0600
From: Roland Haas <rhaas@illinois.edu>
Subject: Re: [Users] Too many nodes specified: nodes=2 (maxnodes is 1)
To: Karima Shahzad <karimashahzad@gmail.com>
Cc: users@einsteintoolkit.org
Message-ID: <20210121092855.647dde27@8992d193.ncsa.illinois.edu>
Content-Type: text/plain; charset="us-ascii"
Hello Karima,
summarizing the discussion in the call today:
* for --cores 4 --num-threads 2 the issue is that simulation factory
counts only "physical" cores while the OS reports "logical" cores
including the hyperthreading which we originally chose to ignore
* for the other error --cores 2 --num-threads 1 "The domain size for
map 0 scaled for convergence level 0 with convergence factor 2 is not
integer" would seem to be that the coarse grid spacing "dx" does not
evenly divide the grid size xmax - xmin. If the grid spacing is
indeed correct, as you indicated in the call, then best would be to
provide the exact parameter file that you ran.
Yours,
Roland
> Dear Dr. Ronald and Dr. Peter,
> I have a 4 core laptop, and I am using 'Simfactory' command to run the par
> file.
> I have tested both of your suggestions. The addition of "--cores=2
> --num-threads=1" gives me the attached error while the "--cores=4
> --num-threads=2"
> option gives, "Error: Too many nodes specified: nodes=2 (maxnodes is 1)
>
> Aborting Simfactory".
>
>
> Thanks
>
> Best,
>
> KS
>
> On Tue, Jan 19, 2021 at 11:29 PM <users-request@einsteintoolkit.org> wrote:
>
> > Send Users mailing list submissions to
> > users@einsteintoolkit.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$
> > or, via email, send a message with subject or body 'help' to
> > users-request@einsteintoolkit.org
> >
> > You can reach the person managing the list at
> > users-owner@einsteintoolkit.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Users digest..."
> >
> >
> > Today's Topics:
> >
> > 1. TAT/ Slab Error (KARIMA SHAHZAD)
> > 2. Re: TAT/ Slab Error (Peter Diener)
> > 3. Agenda for Thursday's Meeting (users@einsteintoolkit.org)
> > 4. Re: TAT/ Slab Error (Roland Haas)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Mon, 18 Jan 2021 17:41:04 +0000
> > From: KARIMA SHAHZAD <02141911015@student.qau.edu.pk>
> > Subject: [Users] TAT/ Slab Error
> > To: "users@einsteintoolkit.org" <users@einsteintoolkit.org>
> > Message-ID:
> > <
> > DB7PR07MB5435B609DAF0209B60980C8E8CA40@DB7PR07MB5435.eurprd07.prod.outlook.com
> > >
> >
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hello there,
> >
> > Here is the error attached I encountered while evolving black holes
> > binary. The work-around I guess is to increase the number of processes (if
> > I am not wrong), but I am not quite sure how to do that. I need some help
> > to get around this issue.
> >
> > Thanks in advance
> >
> > Best regards,
> > KS
> >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.html__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqqXuJnIl$
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: bbhHr.err
> > Type: application/octet-stream
> > Size: 1075 bytes
> > Desc: bbhHr.err
> > Url :
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.obj__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqre9Jkw5$
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Mon, 18 Jan 2021 14:19:02 -0600 (CST)
> > From: Peter Diener <diener@cct.lsu.edu>
> > Subject: Re: [Users] TAT/ Slab Error
> > To: KARIMA SHAHZAD <02141911015@student.qau.edu.pk>
> > Cc: "users@einsteintoolkit.org" <users@einsteintoolkit.org>
> > Message-ID: <alpine.LSU.2.20.2101181408440.2879@yvahk-8u91.fhfr>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hi Karima,
> >
> > Yes, that parameter file requires at least 2 mpi processes to run.
> >
> > You seem to have submitted using simfactory and ended up running on 1
> > mpi process awith 2 openmp threads. If you instead do something like
> >
> > simfactory/bin/sim run bbhHr --cores=2 --num-threads=1
> >
> > if you're on a workstation that doesn't use a queueing system or
> >
> > simfactory/bin/sim submit bbhHr --cores=2 --num-threads=1
> >
> > if you're on a cluster. This should give you 2 mpi processes running on
> > 1 openmp thread each. You might have to add other options to simfactory
> > if you did so in your original run.
> >
> > Cheers,
> >
> > Peter
> >
> > On Monday 2021-01-18 11:41, KARIMA SHAHZAD wrote:
> >
> > >Date: Mon, 18 Jan 2021 11:41:04
> > >From: KARIMA SHAHZAD <02141911015@student.qau.edu.pk>
> > >To: "users@einsteintoolkit.org" <users@einsteintoolkit.org>
> > >Subject: [Users] TAT/ Slab Error
> > >
> > >Hello there,?
> > >
> > >Here is the error attached I encountered while evolving black holes
> > binary.
> > >The work-around I guess is to increase the number of processes (if I am
> > not
> > >wrong), but I am not quite sure how to do that. I need some help to get
> > >around this issue.
> > >
> > >Thanks in advance
> > >
> > >Best regards,
> > >KS
> > >
> > >
> > >
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Mon, 18 Jan 2021 15:18:01 -0600
> > From: users@einsteintoolkit.org
> > Subject: [Users] Agenda for Thursday's Meeting
> > To: users@einsteintoolkit.org
> > Message-ID: <E1l1buT-009sDh-Ev@ekohaes8>
> > Content-Type: text/plain; charset="ANSI_X3.4-1968"
> >
> > Please update the Wiki with agenda items for Thursday's meeting. Thanks!
> >
> > https://urldefense.com/v3/__https://docs.einsteintoolkit.org/et-docs/meeting_agenda__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqhEBI4Ov$
> >
> > --The Maintainers
> >
> >
> > ------------------------------
> >
> > Message: 4
> > Date: Mon, 18 Jan 2021 16:17:32 -0600
> > From: Roland Haas <rhaas@illinois.edu>
> > Subject: Re: [Users] TAT/ Slab Error
> > To: KARIMA SHAHZAD <02141911015@student.qau.edu.pk>
> > Cc: "users@einsteintoolkit.org" <users@einsteintoolkit.org>
> > Message-ID: <20210118161732.71c09b22@8992d193.ncsa.illinois.edu>
> > Content-Type: text/plain; charset="us-ascii"
> >
> > Hello Karima,
> >
> > > Here is the error attached I encountered while evolving black holes
> > > binary. The work-around I guess is to increase the number of
> > > processes (if I am not wrong), but I am not quite sure how to do
> > > that. I need some help to get around this issue.
> > Assuming you use simulation factory you need to pass options to make
> > sure it creates multiple MPI processes. The auto-detection code tends
> > to choose a number of threads equal to the the number of cores on your
> > laptop and and a single MPI rank.
> >
> > Eg if you have a 4 core laptop then you have to use:
> >
> > ./simfactory/bin/sim submit testrun01 --cores 4 --num-threads 2
> > --parfile ...
> >
> > which uses a total of 4 cores and starts 2 threads per MPI rank so that
> > you end up with 4 / 2 = 2 MPI ranks each of which uses 2 threads.
> >
> > Similarly on clusters, the basic idea is to set --num-threads so that
> > there are more than 1 MPI rank started (ie --num-threads is larger than
> > the value for --cores or --procs [which are synonyms for each other]).
> >
> > If not using simulation factory you have to manually use mpirun and
> > OMP_NUM_THREADS. Eg:
> >
> > export OMP_NUM_THREADS=2
> >
> > mpirun -np 2 /home/karima/simulations/bbhHr/SIMFACTORY/exe/cactus_sim
> > -L 3 /home/karima/simulations/bbhHr/output-0000/BBHHigherRes.par
> >
> > which starts 2 MPI ranks each will use 2 OpenMP threads.
> >
> > Yours,
> > Roland
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqglVIuVi$ .
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: not available
> > Type: application/pgp-signature
> > Size: 833 bytes
> > Desc: OpenPGP digital signature
> > Url :
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/78ab7576/attachment-0001.bin__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqifXAHiE$
> >
> > ------------------------------
> >
> > _______________________________________________
> > Users mailing list
> > Users@einsteintoolkit.org
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$
> >
> >
> > End of Users Digest, Vol 130, Issue 18
> > **************************************
> >
--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20210121/0e328703/attachment-0001.bin
------------------------------
Message: 3
Date: Thu, 21 Jan 2021 09:36:57 -0600
From: "Steven R. Brandt" <sbrandt@cct.lsu.edu>
Subject: [Users] Meeting Minutes 2021-01-21
To: Einstein Toolkit Users <users@einsteintoolkit.org>
Message-ID: <e277d327-2ad8-5161-f1aa-32ecf0ad17ff@cct.lsu.edu>
Content-Type: text/plain; charset=utf-8; format=flowed
Present: Yosef, Steve, Roland, Peter D, Bill, Kenneth, Atul, Karim,
Miguel, Zach
Chair: Yosef
Notes: Steve
SPEC Benchmark:
====================
All paperwork is done.
Roland hasn't set the poll out, but ready to send out. All three
versions, though, compile and run. He had to
modify Kranc.
Overview of CarpetX:
====================
Can do some science but could use some real world workload
No existing thorns work, but new thorns could be adapted.
Can use GPUs (can run Hydro toy code)
Indicate interest in seeing the overview on Gitter chat
https://gitter.im/EinsteinToolkit/EinsteinToolkit?at=600712b2004fab4741504966
Yosef would like to attend but cannot login to Gitter
Maybe have an overview presented on the call?
Maybe start regular presentations up again?
Have Gabriele Rozzola give a talk?
Seminars should be recorded. They can be edited.
??? Gabriele Rozzola accepted
Old seminars: http://einsteintoolkit.org/seminars/
Discussed Karima's issue with simfactory and the parfile. She should
send the parfile to the mailing list.
Open tickets:
Python3 version of simfactory. Roland is still working on the survey for
which simfactory features are used. He will send the survey out.
Ticket 963, open since 2012. Peter Diener will take a look.
Steve will try to look at 2462 to potentially approve it.
Yosef looked at ticket 2486 on the call and approved it.
Feb 4:
??? Minutes: Roland
??? Chair: Yosef
------------------------------
Message: 4
Date: Thu, 21 Jan 2021 11:34:29 -0600
From: Roland Haas <rhaas@illinois.edu>
Subject: Re: [Users] Meeting Minutes 2021-01-21
To: "Steven R. Brandt" <sbrandt@cct.lsu.edu>
Cc: Einstein Toolkit Users <users@einsteintoolkit.org>
Message-ID: <20210121113429.4b5fce3d@8992d193.ncsa.illinois.edu>
Content-Type: text/plain; charset="utf-8"
Hello all,
for the SPECv8 contribution we had at one point mentioned 3 options
that we could propose:
* McLachlan's ccZ4 formulation (in master)
* McLachlan's Z4c formulation (in rhaas/Z4c)
* a version of McLachlan that pre-computes all derivatives as grid
functions the way that the CUDA code does (in
rhaas/separatedderivatives)
Please use this form:
https://docs.google.com/forms/d/e/1FAIpQLSdtnvxu-PA5gxY7o-fFWARPFOby8joz4-nC51ru_t3oKlJT7w/viewform?usp=sf_link
to voice preference for one or the other.
At this point none have been extracted and incorporated into the
min-driver used for SPEC.
Yours,
Roland
> Present: Yosef, Steve, Roland, Peter D, Bill, Kenneth, Atul, Karim,
> Miguel, Zach
> Chair: Yosef
> Notes: Steve
>
> SPEC Benchmark:
> ====================
> All paperwork is done.
>
> Roland hasn't set the poll out, but ready to send out. All three
> versions, though, compile and run. He had to
> modify Kranc.
>
> Overview of CarpetX:
> ====================
> Can do some science but could use some real world workload
> No existing thorns work, but new thorns could be adapted.
> Can use GPUs (can run Hydro toy code)
>
> Indicate interest in seeing the overview on Gitter chat
> https://urldefense.com/v3/__https://gitter.im/EinsteinToolkit/EinsteinToolkit?at=600712b2004fab4741504966__;!!DZ3fjg!pUhgFTuMQ9PienZhCRrABC7c2uSZXGjuO91MDrvPNvppA3STHoxrYe8Fq1Xxfq1B$
>
> Yosef would like to attend but cannot login to Gitter
>
> Maybe have an overview presented on the call?
> Maybe start regular presentations up again?
> Have Gabriele Rozzola give a talk?
> Seminars should be recorded. They can be edited.
> ??? Gabriele Rozzola accepted
> Old seminars: https://urldefense.com/v3/__http://einsteintoolkit.org/seminars/__;!!DZ3fjg!pUhgFTuMQ9PienZhCRrABC7c2uSZXGjuO91MDrvPNvppA3STHoxrYe8Fq14ZQ2zL$
>
> Discussed Karima's issue with simfactory and the parfile. She should
> send the parfile to the mailing list.
>
> Open tickets:
> Python3 version of simfactory. Roland is still working on the survey for
> which simfactory features are used. He will send the survey out.
>
> Ticket 963, open since 2012. Peter Diener will take a look.
> Steve will try to look at 2462 to potentially approve it.
> Yosef looked at ticket 2486 on the call and approved it.
>
> Feb 4:
> ??? Minutes: Roland
> ??? Chair: Yosef
>
> _______________________________________________
> Users mailing list
> Users@einsteintoolkit.org
> https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!pUhgFTuMQ9PienZhCRrABC7c2uSZXGjuO91MDrvPNvppA3STHoxrYe8Fq1eMGUNk$
--
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20210121/670abba9/attachment.bin
------------------------------
_______________________________________________
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users
End of Users Digest, Vol 130, Issue 21
**************************************