[Users] Users Digest, Vol 130, Issue 21

Roland Haas rhaas at illinois.edu
Fri Jan 22 11:09:54 CST 2021 

Hello Karima,

great. Glad to have been able to help fixing this.

Yours,
Roland

> The error was due to an incorrect par file setting.   "---core= 2, thread=
> 1--" worked, thank you very much for the support.
> 
> Best,
> Karima S
> 
> On Thu, Jan 21, 2021 at 10:34 PM <users-request at einsteintoolkit.org> wrote:
> 
> > Send Users mailing list submissions to
> >         users at einsteintoolkit.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >         https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYRZ9kWXM$ 
> > or, via email, send a message with subject or body 'help' to
> >         users-request at einsteintoolkit.org
> >
> > You can reach the person managing the list at
> >         users-owner at einsteintoolkit.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Users digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Einstein Toolkit Meeting Reminder (users at einsteintoolkit.org)
> >    2. Re: Too many nodes specified: nodes=2 (maxnodes is 1)
> >       (Roland Haas)
> >    3. Meeting Minutes 2021-01-21 (Steven R. Brandt)
> >    4. Re: Meeting Minutes 2021-01-21 (Roland Haas)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 20 Jan 2021 17:15:01 -0600
> > From: users at einsteintoolkit.org
> > Subject: [Users] Einstein Toolkit Meeting Reminder
> > To: users at einsteintoolkit.org
> > Message-ID: <E1l2Mgn-00Bybi-Hb at ekohaes8>
> > Content-Type: text/plain; charset="ANSI_X3.4-1968"
> >
> > Hello,
> >
> > Please consider joining the weekly Einstein Toolkit phone call at
> > 9:00 am US central time on Thursdays. For details on how to connect
> > and what agenda items are to be discussed, use the link below.
> >
> > https://urldefense.com/v3/__https://docs.einsteintoolkit.org/et-docs/Main_Page*Weekly_Users_Call__;Iw!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYUjfxPNN$ 
> >
> > --The Maintainers
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Thu, 21 Jan 2021 09:28:55 -0600
> > From: Roland Haas <rhaas at illinois.edu>
> > Subject: Re: [Users] Too many nodes specified: nodes=2 (maxnodes is 1)
> > To: Karima Shahzad <karimashahzad at gmail.com>
> > Cc: users at einsteintoolkit.org
> > Message-ID: <20210121092855.647dde27 at 8992d193.ncsa.illinois.edu>
> > Content-Type: text/plain; charset="us-ascii"
> >
> > Hello Karima,
> >
> > summarizing the discussion in the call today:
> >
> > * for --cores 4 --num-threads 2 the issue is that simulation factory
> >   counts only "physical" cores while the OS reports "logical" cores
> >   including the hyperthreading which we originally chose to ignore
> >
> > * for the other error --cores 2 --num-threads 1 "The domain size for
> >   map 0 scaled for convergence level 0 with convergence factor 2 is not
> >   integer" would seem to be that the coarse grid spacing "dx" does not
> >   evenly divide the grid size xmax - xmin. If the grid spacing is
> >   indeed correct, as you indicated in the call, then best would be to
> >   provide the exact parameter file that you ran.
> >
> > Yours,
> > Roland
> >  
> > > Dear Dr. Ronald and Dr. Peter,
> > > I have a 4 core laptop, and I am using 'Simfactory' command to run the  
> > par  
> > > file.
> > > I have tested both of your suggestions. The addition of  "--cores=2
> > > --num-threads=1" gives me the attached error while the "--cores=4
> > > --num-threads=2"
> > > option gives, "Error: Too many nodes specified: nodes=2 (maxnodes is 1)
> > >
> > > Aborting Simfactory".
> > >
> > >
> > > Thanks
> > >
> > > Best,
> > >
> > > KS
> > >
> > > On Tue, Jan 19, 2021 at 11:29 PM <users-request at einsteintoolkit.org>  
> > wrote:  
> > >  
> > > > Send Users mailing list submissions to
> > > >         users at einsteintoolkit.org
> > > >
> > > > To subscribe or unsubscribe via the World Wide Web, visit
> > > >  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$  
> > > > or, via email, send a message with subject or body 'help' to
> > > >         users-request at einsteintoolkit.org
> > > >
> > > > You can reach the person managing the list at
> > > >         users-owner at einsteintoolkit.org
> > > >
> > > > When replying, please edit your Subject line so it is more specific
> > > > than "Re: Contents of Users digest..."
> > > >
> > > >
> > > > Today's Topics:
> > > >
> > > >    1. TAT/ Slab Error (KARIMA SHAHZAD)
> > > >    2. Re: TAT/ Slab Error (Peter Diener)
> > > >    3. Agenda for Thursday's Meeting (users at einsteintoolkit.org)
> > > >    4. Re: TAT/ Slab Error (Roland Haas)
> > > >
> > > >
> > > > ----------------------------------------------------------------------
> > > >
> > > > Message: 1
> > > > Date: Mon, 18 Jan 2021 17:41:04 +0000
> > > > From: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > > > Subject: [Users] TAT/ Slab Error
> > > > To: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > > > Message-ID:
> > > >         <
> > > >  
> > DB7PR07MB5435B609DAF0209B60980C8E8CA40 at DB7PR07MB5435.eurprd07.prod.outlook.com  
> > > > >  
> > > >
> > > > Content-Type: text/plain; charset="iso-8859-1"
> > > >
> > > > Hello there,
> > > >
> > > > Here is the error attached I encountered while evolving black holes
> > > > binary. The work-around I guess is to increase the number of processes  
> > (if  
> > > > I am not wrong), but I am not quite sure how to do that. I need some  
> > help  
> > > > to get around this issue.
> > > >
> > > > Thanks in advance
> > > >
> > > > Best regards,
> > > > KS
> > > >
> > > > -------------- next part --------------
> > > > An HTML attachment was scrubbed...
> > > > URL:
> > > >  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.html__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqqXuJnIl$  
> > > > -------------- next part --------------
> > > > A non-text attachment was scrubbed...
> > > > Name: bbhHr.err
> > > > Type: application/octet-stream
> > > > Size: 1075 bytes
> > > > Desc: bbhHr.err
> > > > Url :
> > > >  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.obj__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqre9Jkw5$  
> > > >
> > > > ------------------------------
> > > >
> > > > Message: 2
> > > > Date: Mon, 18 Jan 2021 14:19:02 -0600 (CST)
> > > > From: Peter Diener <diener at cct.lsu.edu>
> > > > Subject: Re: [Users] TAT/ Slab Error
> > > > To: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > > > Cc: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > > > Message-ID: <alpine.LSU.2.20.2101181408440.2879 at yvahk-8u91.fhfr>
> > > > Content-Type: text/plain; charset="iso-8859-1"
> > > >
> > > > Hi Karima,
> > > >
> > > > Yes, that parameter file requires at least 2 mpi processes to run.
> > > >
> > > > You seem to have submitted using simfactory and ended up running on 1
> > > > mpi process awith 2 openmp threads. If you instead do something like
> > > >
> > > > simfactory/bin/sim run bbhHr --cores=2 --num-threads=1
> > > >
> > > > if you're on a workstation that doesn't use a queueing system or
> > > >
> > > > simfactory/bin/sim submit bbhHr --cores=2 --num-threads=1
> > > >
> > > > if you're on a cluster. This should give you 2 mpi processes running on
> > > > 1 openmp thread each. You might have to add other options to simfactory
> > > > if you did so in your original run.
> > > >
> > > > Cheers,
> > > >
> > > >    Peter
> > > >
> > > > On Monday 2021-01-18 11:41, KARIMA SHAHZAD wrote:
> > > >  
> > > > >Date: Mon, 18 Jan 2021 11:41:04
> > > > >From: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > > > >To: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > > > >Subject: [Users] TAT/ Slab Error
> > > > >
> > > > >Hello there,?
> > > > >
> > > > >Here is the error attached I encountered while evolving black holes  
> > > > binary.  
> > > > >The work-around I guess is to increase the number of processes (if I  
> > am  
> > > > not  
> > > > >wrong), but I am not quite sure how to do that. I need some help to  
> > get  
> > > > >around this issue.
> > > > >
> > > > >Thanks in advance
> > > > >
> > > > >Best regards,
> > > > >KS
> > > > >
> > > > >
> > > > >  
> > > >
> > > > ------------------------------
> > > >
> > > > Message: 3
> > > > Date: Mon, 18 Jan 2021 15:18:01 -0600
> > > > From: users at einsteintoolkit.org
> > > > Subject: [Users] Agenda for Thursday's Meeting
> > > > To: users at einsteintoolkit.org
> > > > Message-ID: <E1l1buT-009sDh-Ev at ekohaes8>
> > > > Content-Type: text/plain; charset="ANSI_X3.4-1968"
> > > >
> > > > Please update the Wiki with agenda items for Thursday's meeting.  
> > Thanks!  
> > > >
> > > >  
> > https://urldefense.com/v3/__https://docs.einsteintoolkit.org/et-docs/meeting_agenda__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqhEBI4Ov$  
> > > >
> > > > --The Maintainers
> > > >
> > > >
> > > > ------------------------------
> > > >
> > > > Message: 4
> > > > Date: Mon, 18 Jan 2021 16:17:32 -0600
> > > > From: Roland Haas <rhaas at illinois.edu>
> > > > Subject: Re: [Users] TAT/ Slab Error
> > > > To: KARIMA SHAHZAD <02141911015 at student.qau.edu.pk>
> > > > Cc: "users at einsteintoolkit.org" <users at einsteintoolkit.org>
> > > > Message-ID: <20210118161732.71c09b22 at 8992d193.ncsa.illinois.edu>
> > > > Content-Type: text/plain; charset="us-ascii"
> > > >
> > > > Hello Karima,
> > > >  
> > > > > Here is the error attached I encountered while evolving black holes
> > > > > binary. The work-around I guess is to increase the number of
> > > > > processes (if I am not wrong), but I am not quite sure how to do
> > > > > that. I need some help to get around this issue.  
> > > > Assuming you use simulation factory you need to pass options to make
> > > > sure it creates multiple MPI processes. The auto-detection code tends
> > > > to choose a number of threads equal to the the number of cores on your
> > > > laptop and and a single MPI rank.
> > > >
> > > > Eg if you have a 4 core laptop then you have to use:
> > > >
> > > > ./simfactory/bin/sim submit testrun01 --cores 4 --num-threads 2
> > > > --parfile ...
> > > >
> > > > which uses a total of 4 cores and starts 2 threads per MPI rank so that
> > > > you end up with 4 / 2 = 2 MPI ranks each of which uses 2 threads.
> > > >
> > > > Similarly on clusters, the basic idea is to set --num-threads so that
> > > > there are more than 1 MPI rank started (ie --num-threads is larger than
> > > > the value for --cores or --procs [which are synonyms for each other]).
> > > >
> > > > If not using simulation factory you have to manually use mpirun and
> > > > OMP_NUM_THREADS. Eg:
> > > >
> > > > export OMP_NUM_THREADS=2
> > > >
> > > > mpirun -np 2 /home/karima/simulations/bbhHr/SIMFACTORY/exe/cactus_sim
> > > > -L 3 /home/karima/simulations/bbhHr/output-0000/BBHHigherRes.par
> > > >
> > > > which starts 2 MPI ranks each will use 2 OpenMP threads.
> > > >
> > > > Yours,
> > > > Roland
> > > >
> > > > --
> > > > My email is as private as my paper mail. I therefore support encrypting
> > > > and signing email messages. Get my PGP key from  
> > https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqglVIuVi$
> > .  
> > > > -------------- next part --------------
> > > > A non-text attachment was scrubbed...
> > > > Name: not available
> > > > Type: application/pgp-signature
> > > > Size: 833 bytes
> > > > Desc: OpenPGP digital signature
> > > > Url :
> > > >  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/78ab7576/attachment-0001.bin__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqifXAHiE$  
> > > >
> > > > ------------------------------
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users at einsteintoolkit.org
> > > >  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$  
> > > >
> > > >
> > > > End of Users Digest, Vol 130, Issue 18
> > > > **************************************
> > > >  
> >
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYTddbu1x$  .
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: not available
> > Type: application/pgp-signature
> > Size: 833 bytes
> > Desc: OpenPGP digital signature
> > Url :
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210121/0e328703/attachment-0001.bin__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYTEXKzuH$ 
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Thu, 21 Jan 2021 09:36:57 -0600
> > From: "Steven R. Brandt" <sbrandt at cct.lsu.edu>
> > Subject: [Users] Meeting Minutes 2021-01-21
> > To: Einstein Toolkit Users <users at einsteintoolkit.org>
> > Message-ID: <e277d327-2ad8-5161-f1aa-32ecf0ad17ff at cct.lsu.edu>
> > Content-Type: text/plain; charset=utf-8; format=flowed
> >
> > Present: Yosef, Steve, Roland, Peter D, Bill, Kenneth, Atul, Karim,
> > Miguel, Zach
> > Chair: Yosef
> > Notes: Steve
> >
> > SPEC Benchmark:
> > ====================
> > All paperwork is done.
> >
> > Roland hasn't set the poll out, but ready to send out. All three
> > versions, though, compile and run. He had to
> > modify Kranc.
> >
> > Overview of CarpetX:
> > ====================
> > Can do some science but could use some real world workload
> > No existing thorns work, but new thorns could be adapted.
> > Can use GPUs (can run Hydro toy code)
> >
> > Indicate interest in seeing the overview on Gitter chat
> >
> > https://urldefense.com/v3/__https://gitter.im/EinsteinToolkit/EinsteinToolkit?at=600712b2004fab4741504966__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYczDxiu3$ 
> >
> > Yosef would like to attend but cannot login to Gitter
> >
> > Maybe have an overview presented on the call?
> > Maybe start regular presentations up again?
> > Have Gabriele Rozzola give a talk?
> > Seminars should be recorded. They can be edited.
> >  ??? Gabriele Rozzola accepted
> > Old seminars: https://urldefense.com/v3/__http://einsteintoolkit.org/seminars/__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYTMP76kd$ 
> >
> > Discussed Karima's issue with simfactory and the parfile. She should
> > send the parfile to the mailing list.
> >
> > Open tickets:
> > Python3 version of simfactory. Roland is still working on the survey for
> > which simfactory features are used. He will send the survey out.
> >
> > Ticket 963, open since 2012. Peter Diener will take a look.
> > Steve will try to look at 2462 to potentially approve it.
> > Yosef looked at ticket 2486 on the call and approved it.
> >
> > Feb 4:
> >  ??? Minutes: Roland
> >  ??? Chair: Yosef
> >
> >
> >
> > ------------------------------
> >
> > Message: 4
> > Date: Thu, 21 Jan 2021 11:34:29 -0600
> > From: Roland Haas <rhaas at illinois.edu>
> > Subject: Re: [Users] Meeting Minutes 2021-01-21
> > To: "Steven R. Brandt" <sbrandt at cct.lsu.edu>
> > Cc: Einstein Toolkit Users <users at einsteintoolkit.org>
> > Message-ID: <20210121113429.4b5fce3d at 8992d193.ncsa.illinois.edu>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Hello all,
> >
> > for the SPECv8 contribution we had at one point mentioned 3 options
> > that we could propose:
> >
> > * McLachlan's ccZ4 formulation (in master)
> > * McLachlan's Z4c formulation (in rhaas/Z4c)
> > * a version of McLachlan that pre-computes all derivatives as grid
> >   functions the way that the CUDA code does (in
> >   rhaas/separatedderivatives)
> >
> > Please use this form:
> >
> >
> > https://urldefense.com/v3/__https://docs.google.com/forms/d/e/1FAIpQLSdtnvxu-PA5gxY7o-fFWARPFOby8joz4-nC51ru_t3oKlJT7w/viewform?usp=sf_link__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYR7_iSrB$ 
> >
> > to voice preference for one or the other.
> >
> > At this point none have been extracted and incorporated into the
> > min-driver used for SPEC.
> >
> > Yours,
> > Roland
> >  
> > > Present: Yosef, Steve, Roland, Peter D, Bill, Kenneth, Atul, Karim,
> > > Miguel, Zach
> > > Chair: Yosef
> > > Notes: Steve
> > >
> > > SPEC Benchmark:
> > > ====================
> > > All paperwork is done.
> > >
> > > Roland hasn't set the poll out, but ready to send out. All three
> > > versions, though, compile and run. He had to
> > > modify Kranc.
> > >
> > > Overview of CarpetX:
> > > ====================
> > > Can do some science but could use some real world workload
> > > No existing thorns work, but new thorns could be adapted.
> > > Can use GPUs (can run Hydro toy code)
> > >
> > > Indicate interest in seeing the overview on Gitter chat
> > >  
> > https://urldefense.com/v3/__https://gitter.im/EinsteinToolkit/EinsteinToolkit?at=600712b2004fab4741504966__;!!DZ3fjg!pUhgFTuMQ9PienZhCRrABC7c2uSZXGjuO91MDrvPNvppA3STHoxrYe8Fq1Xxfq1B$  
> > >
> > > Yosef would like to attend but cannot login to Gitter
> > >
> > > Maybe have an overview presented on the call?
> > > Maybe start regular presentations up again?
> > > Have Gabriele Rozzola give a talk?
> > > Seminars should be recorded. They can be edited.
> > >  ??? Gabriele Rozzola accepted
> > > Old seminars:  
> > https://urldefense.com/v3/__http://einsteintoolkit.org/seminars/__;!!DZ3fjg!pUhgFTuMQ9PienZhCRrABC7c2uSZXGjuO91MDrvPNvppA3STHoxrYe8Fq14ZQ2zL$  
> > >
> > > Discussed Karima's issue with simfactory and the parfile. She should
> > > send the parfile to the mailing list.
> > >
> > > Open tickets:
> > > Python3 version of simfactory. Roland is still working on the survey for
> > > which simfactory features are used. He will send the survey out.
> > >
> > > Ticket 963, open since 2012. Peter Diener will take a look.
> > > Steve will try to look at 2462 to potentially approve it.
> > > Yosef looked at ticket 2486 on the call and approved it.
> > >
> > > Feb 4:
> > >  ??? Minutes: Roland
> > >  ??? Chair: Yosef
> > >
> > > _______________________________________________
> > > Users mailing list
> > > Users at einsteintoolkit.org
> > >  
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!pUhgFTuMQ9PienZhCRrABC7c2uSZXGjuO91MDrvPNvppA3STHoxrYe8Fq1eMGUNk$
> >
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYTddbu1x$  .
> > -------------- next part --------------
> > A non-text attachment was scrubbed...
> > Name: not available
> > Type: application/pgp-signature
> > Size: 833 bytes
> > Desc: OpenPGP digital signature
> > Url :
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210121/670abba9/attachment.bin__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYZmjYOYt$ 
> >
> > ------------------------------
> >
> > _______________________________________________
> > Users mailing list
> > Users at einsteintoolkit.org
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!vmrhY_QNYGxhp9k9dTVkK9juX_W06J_reFMdnnJkvwtZoLnObKh7HAusYRZ9kWXM$ 
> >
> >
> > End of Users Digest, Vol 130, Issue 21
> > **************************************
> >  


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://lists.einsteintoolkit.org/pipermail/users/attachments/20210122/d4bb0812/attachment.bin

More information about the Users mailing list