From rhaas at illinois.edu  Tue Aug  1 09:12:58 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Tue, 1 Aug 2023 09:12:58 -0500
Subject: [Users] Error while running toolkit
In-Reply-To: <CA+a7ynO9vxDReP-Wwbp0zMsE_uP9Eh3h_cbeSjT=cX0_4youPg@mail.gmail.com>
References: <CA+a7ynMbnRpxdXrwxDPRhH3Xb+fJNUaQ1G_cEfBtMu9m4zMSSg@mail.gmail.com>
 <20230722211550.6beeef0d@fdea4908.spdns.org>
 <CA+a7ynOq3Tq34guWbJVRqjP1VVDm_9WxBdB69Pn88JZiHdCTbg@mail.gmail.com>
 <20230731161527.598f2073@ekohaes8>
 <CA+a7ynO9vxDReP-Wwbp0zMsE_uP9Eh3h_cbeSjT=cX0_4youPg@mail.gmail.com>
Message-ID: <20230801091258.7d62d661@ekohaes8>

Hello Thejas,

> ADM base is already there, and I have updated the thorn  list,
> https://urldefense.com/v3/__https://einsteintoolkit.org/gallery/poisson/index.html__;!!DZ3fjg!_uekN8ac4VQb49xW4C38Hu8o-MwHtFGjdSUXpQ7U1s7UhKTQFh61aoEV203zQW5fn7qHLtxdadBDv9jT3FT7DTgP$  I got the par file
> from here.

I may be that the file exists but as far as I can tell it is not listed
in the thornlist that you are compiling.

In the output to screen just after "Parsing configuration files" it
lists all the thorns present in the thornlist you are compiling and
that list (alphabetically sorted) does not contain ADMBase.

You may need (if you use simfactory) to pass the --thornlist option or
of not using simfactory set THORNLIST or edit configs/sim/Thornlist

The gallery example gives a thornlist (ET Schwarzschild release
(ET_2023_05)) that is known to work with the parfile. Have you tried
that one? I am asking because using it with GetComponents and
simfactory should really give you a working ADMBase.

If all else fails I would suggest starting from scratch, that is remove
your whole Cactus directory, run GetComponents and simfactory again as
described on 

http://einsteintoolkit.org/download.html

Yours,
Roland

PS: If possible, please keep the mailing list users at einsteintoolkit.org
in the list of addressees so that the email thread stays visible.

> 
> 
> On Tue, 1 Aug 2023 at 03:05, Roland Haas <rhaas at illinois.edu> wrote:
> 
> > Hello Thejas,
> >
> > Sorry for the delay, I was busy with the ET summer school at RIT and a
> > second meeting just afterwards 9best to cc users at einsteintoolkit.org
> > on these emails so that others also see them and can jump in).
> >
> > Looking at the error message:
> >
> > --8<--
> > CST error in /home/thejas/Cactus/repos/flesh/lib/sbin/interface_parser.pl
> > (at 273)  
> >   -> CT_MULTILEVEL is friends with ADMBase - non-existent implementation  
> > --8<--
> >
> > you are missing the thron ADMBase (in the EinsteinBase arrangement).
> >
> > You will need to update your thornlist file and add
> >
> > EinsteinBase/ADMBase
> >
> > to it. To be successful in this you may also need to add the correct
> > download instructions so that you can run GetComponents again to
> > download the missing thorn (component).
> >
> > You could eg take the respective section out of the main ET thornlist:
> >
> >
> > https://urldefense.com/v3/__https://bitbucket.org/einsteintoolkit/manifest/src/c590ffe5663d04788aaf4f562faf3c8d9befbac9/einsteintoolkit.th*lines-355__;Iw!!DZ3fjg!_uekN8ac4VQb49xW4C38Hu8o-MwHtFGjdSUXpQ7U1s7UhKTQFh61aoEV203zQW5fn7qHLtxdadBDv9jT3PIUX2Eu$ 
> >
> > Where did you get the thornlist you are currently using from? Would you
> > mind attaching it to the email?
> >
> >
> > Yours,
> > Roland
> >  
> > > Hi,
> > > Thank you for the consideration, I am attaching that here. Do you know  
> > any  
> > > codes for gravitational waves using Einstein's toolkit?
> > > The output.
> > >
> > > Using configuration: sim
> > > Warning: default thorn list contents have changed
> > > Updated thorn list for configuration sim
> > > Building sim
> > > Cactus - version: 4.14.0
> > > Building configuration sim
> > > Processing CCL files
> > > Reading ThornList...
> > > Parsing configuration files...
> > >    AEILocalInterp
> > >    Boundary
> > >            Provides:          Boundary
> > >    CT_Analytic
> > >            Requires:          GenericFD
> > >            Optional:          LoopControl
> > >    CT_MultiLevel
> > >            Requires:          Carpet MPI
> > >    Cactus
> > >            Optional:          MPI
> > >    Carpet
> > >            Provides:          Carpet
> > >            Requires:          CarpetLib IOUtil MPI Timers
> > >            Optional:          LoopControl
> > >    CarpetIOASCII
> > >            Requires:          Carpet CarpetLib IOUtil
> > >            Optional:          SDF
> > >    CarpetIOBasic
> > >            Requires:          Carpet CarpetLib IOUtil
> > >    CarpetIOHDF5
> > >            Provides:          CarpetIOHDF5
> > >            Requires:          Carpet CarpetLib IOUtil HDF5
> > >            Optional:          MPI rnpletal SDF
> > >    CarpetIOScalar
> > >            Requires:          Carpet CarpetLib IOUtil
> > >    CarpetInterp
> > >            Requires:          Carpet CarpetLib MPI
> > >    CarpetLib
> > >            Provides:          CarpetLib
> > >            Requires:          MPI Vectors
> > >            Optional:          Boost CycleClock LoopControl HRSCCore
> > >    CarpetReduce
> > >            Requires:          Carpet CarpetLib MPI
> > >            Optional:          LoopControl
> > >    CarpetRegrid2
> > >            Requires:          Carpet CarpetLib
> > >    CarpetSlab
> > >            Requires:          Carpet CarpetLib
> > >    CartGrid3D
> > >            Provides:          CartGrid3D
> > >            Requires:          CoordBase
> > >    CoordBase
> > >            Provides:          CoordBase
> > >    CycleClock
> > >            Provides:          CycleClock
> > >    Fortran
> > >            Provides:          Fortran
> > >            Requires:          Fortran
> > >    GenericFD
> > >            Provides:          GenericFD
> > >    HDF5
> > >            Provides:          HDF5
> > >            Requires:          HDF5 zlib
> > >            Optional:          MPI
> > >    IOUtil
> > >            Provides:          IOUTIL
> > >    InitBase
> > >    LocalInterp
> > >    LoopControl
> > >            Provides:          LoopControl
> > >            Optional:          CycleClock hwloc Vectors
> > >    MPI
> > >            Provides:          MPI
> > >            Requires:          MPI
> > >            Optional:          hwloc
> > >    MoL
> > >    Slab
> > >            Provides:          Slab
> > >            Optional:          LoopControl MPI
> > >    SymBase
> > >    SystemTopology
> > >            Requires:          hwloc MPI
> > >    Time
> > >    Timers
> > >            Provides:          Timers
> > >            Requires:          CarpetLib
> > >    Vectors
> > >            Provides:          Vectors
> > >            Requires:          Vectors
> > >    hwloc
> > >            Provides:          hwloc
> > >            Requires:          zlib hwloc
> > >            Optional:          pciutils
> > >    zlib
> > >            Provides:          zlib
> > > Parsing interface files...
> > >    AEILocalInterp
> > >            Implements: AEILocalInterp
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    Boundary
> > >            Implements: boundary
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    CT_Analytic
> > >            Implements: CT_Analytic
> > >            Inherits:  Boundary GenericFD Grid
> > >            41 Public, 0 Protected, 0 Private variable groups
> > >    CT_MultiLevel
> > >            Implements: CT_MultiLevel
> > >            Inherits:  Boundary Grid
> > >            Friend of: ADMBase
> > >            0 Public, 0 Protected, 10 Private variable groups
> > >    Cactus
> > >            Implements: Cactus
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    Carpet
> > >            Implements: Driver
> > >            0 Public, 0 Protected, 7 Private variable groups
> > >    CarpetIOASCII
> > >            Implements: IOASCII
> > >            0 Public, 0 Protected, 4 Private variable groups
> > >    CarpetIOBasic
> > >            Implements: IOBasic
> > >            Inherits:  IO
> > >            0 Public, 0 Protected, 4 Private variable groups
> > >    CarpetIOHDF5
> > >            Implements: IOHDF5
> > >            0 Public, 0 Protected, 7 Private variable groups
> > >    CarpetIOScalar
> > >            Implements: IOScalar
> > >            0 Public, 0 Protected, 4 Private variable groups
> > >    CarpetInterp
> > >            Implements: interp
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    CarpetLib
> > >            Implements: CarpetLib
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    CarpetReduce
> > >            Implements: reduce
> > >            0 Public, 0 Protected, 5 Private variable groups
> > >    CarpetRegrid2
> > >            Implements: CarpetRegrid2
> > >            7 Public, 0 Protected, 7 Private variable groups
> > >    CarpetSlab
> > >            Implements: Hyperslab
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    CartGrid3D
> > >            Implements: grid
> > >            Inherits:  coordbase
> > >            3 Public, 0 Protected, 0 Private variable groups
> > >    CoordBase
> > >            Implements: CoordBase
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    CycleClock
> > >            Implements: CycleClock
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    Fortran
> > >            Implements: Fortran
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    GenericFD
> > >            Implements: GenericFD
> > >            0 Public, 0 Protected, 3 Private variable groups
> > >    HDF5
> > >            Implements: HDF5
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    IOUtil
> > >            Implements: IO
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    InitBase
> > >            Implements: InitBase
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    LocalInterp
> > >            Implements: LocalInterp
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    LoopControl
> > >            Implements: LoopControl
> > >            Inherits:  CycleClock
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    MPI
> > >            Implements: MPI
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    MoL
> > >            Implements: MethodOfLines
> > >            0 Public, 0 Protected, 10 Private variable groups
> > >    Slab
> > >            Implements: Slab
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    SymBase
> > >            Implements: SymBase
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    SystemTopology
> > >            Implements: SystemTopology
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    Time
> > >            Implements: time
> > >            2 Public, 0 Protected, 2 Private variable groups
> > >    Timers
> > >            Implements: Timers
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    Vectors
> > >            Implements: Vectors
> > >            0 Public, 0 Protected, 2 Private variable groups
> > >    hwloc
> > >            Implements: hwloc
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >    zlib
> > >            Implements: zlib
> > >            0 Public, 0 Protected, 0 Private variable groups
> > >
> > > CST error in /home/thejas/Cactus/repos/flesh/lib/sbin/  
> > interface_parser.pl  
> > > (at 273)  
> > >   -> CT_MULTILEVEL is friends with ADMBase - non-existent  
> > implementation  
> > >
> > > Parsing parameter files...
> > >    AEILocalInterp
> > >           0 Global, 0 Restricted, 1 Private parameters
> > >    Boundary
> > >            Shares: CACTUS
> > >           0 Global, 8 Restricted, 0 Private parameters
> > >    CT_Analytic
> > >            Shares: GENERICFD METHODOFLINES
> > >           0 Global, 67 Restricted, 1 Private parameters
> > >    CT_MultiLevel
> > >            Shares: ADMBASE
> > >           0 Global, 0 Restricted, 66 Private parameters
> > >    Cactus
> > >           0 Global, 7 Restricted, 13 Private parameters
> > >    Carpet
> > >            Shares: CACTUS CARPETLIB INITBASE IO
> > >           0 Global, 4 Restricted, 100 Private parameters
> > >    CarpetIOASCII
> > >            Shares: CACTUS IO
> > >           0 Global, 0 Restricted, 63 Private parameters
> > >    CarpetIOBasic
> > >            Shares: CACTUS IO
> > >           0 Global, 0 Restricted, 15 Private parameters
> > >    CarpetIOHDF5
> > >            Shares: CACTUS IO
> > >           0 Global, 0 Restricted, 77 Private parameters
> > >    CarpetIOScalar
> > >            Shares: CACTUS IO
> > >           0 Global, 0 Restricted, 9 Private parameters
> > >    CarpetInterp
> > >           0 Global, 0 Restricted, 5 Private parameters
> > >    CarpetLib
> > >            Shares: IO
> > >           0 Global, 5 Restricted, 31 Private parameters
> > >    CarpetReduce
> > >           0 Global, 0 Restricted, 3 Private parameters
> > >    CarpetRegrid2
> > >           0 Global, 0 Restricted, 135 Private parameters
> > >    CarpetSlab
> > >           0 Global, 0 Restricted, 0 Private parameters
> > >    CartGrid3D
> > >            Shares: DRIVER
> > >           0 Global, 23 Restricted, 10 Private parameters
> > >    CoordBase
> > >           0 Global, 0 Restricted, 44 Private parameters
> > >    CycleClock
> > >           0 Global, 0 Restricted, 1 Private parameters
> > >    Fortran
> > >           0 Global, 0 Restricted, 1 Private parameters
> > >    GenericFD
> > >           0 Global, 13 Restricted, 0 Private parameters
> > >    HDF5
> > >           0 Global, 0 Restricted, 0 Private parameters
> > >    IOUtil
> > >           0 Global, 59 Restricted, 2 Private parameters
> > >    InitBase
> > >           0 Global, 1 Restricted, 0 Private parameters
> > >    LocalInterp
> > >           0 Global, 0 Restricted, 0 Private parameters
> > >    LoopControl
> > >            Shares: IO
> > >           0 Global, 0 Restricted, 20 Private parameters
> > >    MPI
> > >           0 Global, 0 Restricted, 0 Private parameters
> > >    MoL
> > >            Shares: CACTUS
> > >           0 Global, 9 Restricted, 26 Private parameters
> > >
> > >    Slab
> > >           0 Global, 0 Restricted, 2 Private parameters
> > >    SymBase
> > >           0 Global, 0 Restricted, 1 Private parameters
> > >    SystemTopology
> > >           0 Global, 0 Restricted, 2 Private parameters
> > >    Time
> > >            Shares: CACTUS
> > >           0 Global, 2 Restricted, 5 Private parameters
> > >    Timers
> > >            Shares: IO
> > >           0 Global, 0 Restricted, 6 Private parameters
> > >    Vectors
> > >           0 Global, 0 Restricted, 2 Private parameters
> > >    hwloc
> > >           0 Global, 0 Restricted, 0 Private parameters
> > >    zlib
> > >           0 Global, 0 Restricted, 0 Private parameters
> > > Parsing schedule files...
> > >    AEILocalInterp
> > >           1 schedule blocks.
> > >    Boundary
> > >           6 schedule blocks.
> > >    CT_Analytic
> > >           13 schedule blocks.
> > >    CT_MultiLevel
> > >           1 schedule blocks.
> > >    Cactus
> > >           24 schedule blocks.
> > >    Carpet
> > >           9 schedule blocks.
> > >    CarpetIOASCII
> > >           2 schedule blocks.
> > >    CarpetIOBasic
> > >           2 schedule blocks.
> > >    CarpetIOHDF5
> > >           10 schedule blocks.
> > >    CarpetIOScalar
> > >           2 schedule blocks.
> > >    CarpetInterp
> > >           1 schedule blocks.
> > >    CarpetLib
> > >           4 schedule blocks.
> > >    CarpetReduce
> > >           15 schedule blocks.
> > >    CarpetRegrid2
> > >           2 schedule blocks.
> > >    CarpetSlab
> > >           0 schedule blocks.
> > >    CartGrid3D
> > >           11 schedule blocks.
> > >    CoordBase
> > >           1 schedule blocks.
> > >    CycleClock
> > >           1 schedule blocks.
> > >    Fortran
> > >           1 schedule blocks.
> > >    GenericFD
> > >           0 schedule blocks.
> > >    HDF5
> > >           0 schedule blocks.
> > >    IOUtil
> > >           4 schedule blocks.
> > >    InitBase
> > >           0 schedule blocks.
> > >    LocalInterp
> > >           1 schedule blocks.
> > >    LoopControl
> > >           4 schedule blocks.
> > >    MPI
> > >           0 schedule blocks.
> > >    MoL
> > >           68 schedule blocks.
> > >    Slab
> > >           3 schedule blocks.
> > >    SymBase
> > >           5 schedule blocks.
> > >    SystemTopology
> > >           1 schedule blocks.
> > >    Time
> > >           7 schedule blocks.
> > >    Timers
> > >           2 schedule blocks.
> > >    Vectors
> > >           2 schedule blocks.
> > >    hwloc
> > >           2 schedule blocks.
> > >    zlib
> > >           0 schedule blocks.
> > >
> > >
> > > ------------------------------------------------------
> > > There was 1 error during execution of the CST
> > > This must be corrected before compilation can proceed
> > > ------------------------------------------------------
> > >
> > >
> > >
> > > ------------------------------------------------------
> > > Warnings were generated during execution of the CST
> > > ------------------------------------------------------
> > >
> > >
> > > CST error in /home/thejas/Cactus/repos/flesh/lib/sbin/  
> > interface_parser.pl  
> > > (at 273)  
> > >   -> CT_MULTILEVEL is friends with ADMBase - non-existent  
> > implementation  
> > >
> > > ------------------------------------------------------
> > >
> > > make[1]: *** [/home/thejas/Cactus/lib/make/make.configuration:213:
> > > /home/thejas/Cactus/configs/sim/config-data/make.thornlist] Error 1
> > > make: *** [Makefile:264: sim] Error 2
> > >
> > >
> > > On Sun, 23 Jul 2023 at 06:46, Roland Haas <rhaas at illinois.edu> wrote:
> > >  
> > > > Dear Thejas,
> > > >
> > > > Instead of the image (out of which I cannot copy and paste text), would
> > > > you mind instead attaching the text output that you have on screen (by
> > > > selecting it, CTRL-C to copy then CTRL-V to paste into the text email)?
> > > >
> > > > Yours,
> > > > Roland
> > > >  
> > > > > Dear Sir/Madam
> > > > > I am Thejas, final year Masters student at Central University of
> > > > > Kerala(India), my Masters project is on Gravitational waves. As a  
> > part of  
> > > > > it I have to work with Einstein's toolkit, unfortunately I am having  
> > some  
> > > > > error while running poisson par file, could you please guide me  
> > regarding  
> > > > > this?, for clarification I am attaching the image and I would also  
> > like  
> > > > to  
> > > > > know how to implement friend in einstein's toolkit.
> > > > > Warm Regards  
> > > >
> > > >
> > > >
> > > > --
> > > > My email is as private as my paper mail. I therefore support encrypting
> > > > and signing email messages. Get my PGP key from  
> > https://urldefense.com/v3/__http://keys.gnupg.net__;!!DZ3fjg!9IEB8rNzT2wvbaky6l_HMiHmoLnVIzx4yE4SFxV9L_3D9OHTJXCqtfom3kOaQHxgf2yjXu1ki-GfOWdQQRd0oHvp$
> > .  
> > > >  
> >
> >
> > --
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!_uekN8ac4VQb49xW4C38Hu8o-MwHtFGjdSUXpQ7U1s7UhKTQFh61aoEV203zQW5fn7qHLtxdadBDv9jT3D9BjwJA$  .
> >  


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230801/985c90d0/attachment-0001.sig>

From bruno.giacomazzo at unimib.it  Tue Aug  1 10:18:45 2023
From: bruno.giacomazzo at unimib.it (Bruno Giacomazzo)
Date: Tue, 1 Aug 2023 17:18:45 +0200
Subject: [Users] rel. fluids in the Toolkit
In-Reply-To: <6bfdcf70-19be-ad2c-9093-23f7d4649ce5@gmail.com>
References: <6bfdcf70-19be-ad2c-9093-23f7d4649ce5@gmail.com>
Message-ID: <CADjWfoiOJifyKTRKtr4tMNeV_SLJKJOeQH6uxQAHzTJsFec4vQ@mail.gmail.com>

Hi Bill,


> o Are there good reviews, references, or lectures that describe how we
> currently handle fluids?  For the different models?
>

There are introductory lectures typically given at the Einstein Toolkit
Schools and Workshops in the EU and USA. Some of these should have been
recorded. You may try to have a loot at the ETK YouTube channel:
https://www.youtube.com/@einsteintoolkit/videos


> o I gather we have several models/codes: GRHydro and IllinoisGRMHD, so
> resources on those would be much appreciated.
>

GRHydro paper:
https://ui.adsabs.harvard.edu/abs/2014CQGra..31a5005M/abstract
IllinoisGRMHD paper:
https://ui.adsabs.harvard.edu/abs/2015CQGra..32q5009E/abstract and
https://ui.adsabs.harvard.edu/abs/2023PhRvD.107d4037W/abstract


> o And initial data readers exist for the LORENE (1 and 2?) and for the
> FUKA data...resources there might be helpful too.
>

FUKA: https://bitbucket.org/fukaws/fuka/src/fuka/ (see references there)

There are also a couple of reviews on Living Reviews that describe the
methods that are also used in the Einstein Toolkit hydro codes:
special relativity: https://link.springer.com/article/10.1007/lrca-2015-3
general relativity: https://link.springer.com/article/10.12942/lrr-2008-7

Cheers,
Bruno


-- 

Prof. Bruno Giacomazzo
Department of Physics
University of Milano-Bicocca
Piazza della Scienza 3
20126 Milano
Italy

email: bruno.giacomazzo at unimib.it
phone: (+39) 02 6448 2321
web: http://www.brunogiacomazzo.org

---------------------------------------------------------------------
There are only 10 types of people in the world:
Those who understand binary, and those who don't
----------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230801/0d9512d9/attachment.htm>

From yrzsma at rit.edu  Thu Aug  3 10:00:15 2023
From: yrzsma at rit.edu (Yosef Zlochower)
Date: Thu, 3 Aug 2023 15:00:15 +0000
Subject: [Users] Meeting minutes. August 3, 2023
Message-ID: <747c0de5-9f32-f280-f1f4-dcceb3d53612@rit.edu>

Attendees:? Samuel Cupp, Roland Haas, Peter Diener, Steven Brandt, Zach 
Etienne, Keith Dow, Yosef Zlochower

Summer School recap:
 ?? We need more disk space
 ?? Encourage each tutorial speaker to test notebooks

 ?? Need to report number of both in person and online participants.

ET 2023_11 release:

 ?? Need volunteer reviewers (see list at 
https://docs.einsteintoolkit.org/et-docs/Meeting_agenda 2023-08-03 minutes)

 ? Need more CarpetX documentation
 ? Need more CarpetX testsuites

 ? May need multiple reviewers for spacetimeX (Yosef volunteers). Who 
are the champions ?

 ? Potential issues of duplicated thorns/code when Carpet and CarpetX 
are both in the toolkit.

 ? Reviewers needed for GRHayL thorns (Peter volunteers)

 ? Reviewers needed for AsterX


Chair Next week: Samuel Cupp
Minute taker:


From bill.gabella at gmail.com  Thu Aug  3 10:02:47 2023
From: bill.gabella at gmail.com (Bill Gabella)
Date: Thu, 3 Aug 2023 10:02:47 -0500
Subject: [Users] rel. fluids in the Toolkit
In-Reply-To: <CADjWfoiOJifyKTRKtr4tMNeV_SLJKJOeQH6uxQAHzTJsFec4vQ@mail.gmail.com>
References: <6bfdcf70-19be-ad2c-9093-23f7d4649ce5@gmail.com>
 <CADjWfoiOJifyKTRKtr4tMNeV_SLJKJOeQH6uxQAHzTJsFec4vQ@mail.gmail.com>
Message-ID: <4d5dcabb-1d0e-2c6a-e026-a510135ccfc4@gmail.com>

Bruno and Samuel,?? thanks for all the references...lots to absorb, as 
expected:)

bill e.g.

On 8/1/23 10:18, Bruno Giacomazzo wrote:
> Hi Bill,
>
>     o Are there good reviews, references, or lectures that describe
>     how we
>     currently handle fluids?? For the different models?
>
>
> There are introductory?lectures typically given at the Einstein 
> Toolkit Schools and Workshops in the EU and USA. Some of these should 
> have been recorded. You may try to have a loot at the ETK YouTube 
> channel: https://www.youtube.com/@einsteintoolkit/videos
>
>     o I gather we have several models/codes: GRHydro and
>     IllinoisGRMHD, so
>     resources on those would be much appreciated.
>
>
> GRHydro paper: 
> https://ui.adsabs.harvard.edu/abs/2014CQGra..31a5005M/abstract
> IllinoisGRMHD paper: 
> https://ui.adsabs.harvard.edu/abs/2015CQGra..32q5009E/abstract and 
> https://ui.adsabs.harvard.edu/abs/2023PhRvD.107d4037W/abstract
>
>     o And initial data readers exist for the LORENE (1 and 2?) and for
>     the
>     FUKA data...resources there might be helpful too.
>
>
> FUKA: https://bitbucket.org/fukaws/fuka/src/fuka/ (see references there)
>
> There are also a couple of reviews on Living Reviews that describe the 
> methods that are also used in the Einstein Toolkit hydro codes:
> special relativity: https://link.springer.com/article/10.1007/lrca-2015-3
> general relativity: https://link.springer.com/article/10.12942/lrr-2008-7
>
> Cheers,
> Bruno
>
>
> -- 
>
> Prof. Bruno Giacomazzo
> Department of Physics
> University of Milano-Bicocca
> Piazza della Scienza 3
> 20126 Milano
> Italy
>
> email: bruno.giacomazzo at unimib.it
> phone: (+39) 02 6448 2321
> web: http://www.brunogiacomazzo.org
>
> ---------------------------------------------------------------------
> There are only 10 types of people in the world:
> Those who understand binary, and those who don't
> ----------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230803/150d0b50/attachment-0001.htm>

From emost at caltech.edu  Wed Aug  2 18:45:52 2023
From: emost at caltech.edu (Most, Elias R.)
Date: Wed, 2 Aug 2023 23:45:52 +0000
Subject: [Users] Adding extra object file at linking stage
Message-ID: <DM6PR03MB3819F334124466CED4785221B40BA@DM6PR03MB3819.namprd03.prod.outlook.com>

Hi,

I have an extra object file ?.o? that was compiled externally, but needs to be linked in during the linking process.

Is there a standard way to include this within the build process?

Thank you,
Elias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230802/c079524b/attachment-0001.htm>

From rob6013 at gmail.com  Sat Aug  5 06:49:13 2023
From: rob6013 at gmail.com (Rob)
Date: Sat, 5 Aug 2023 12:49:13 +0100
Subject: [Users] Problem installing Einstein Toolkit on Mac
Message-ID: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>

Hi,

I hope someone can help.
I have tried to install the Einstein Toolkit but am getting an error in the
final part of the process on the Jupyter CactusTutorial:

%%bash
./simfactory/bin/sim build -j2 --thornlist thornlists/einsteintoolkit.th

Output is:

Using configuration: sim
Reconfiguring sim
Writing configuration to: /Users/rob/Cactus/configs/sim/OptionList
Cactus - version: 4.14.0
Reconfiguring sim.
Using configuration options from configure line
  Setting fds to '3,5 -j --'
End of options from configure line
Adding configuration options from '/Users/rob/Cactus/configs/sim/OptionList'...
  Setting VERSION to '2018-12-13'
  Setting CPP to 'cpp'
  Setting CC to 'gcc'
  Setting CXX to 'g++'
  Setting FPP to 'cpp'
  Setting F90 to 'gfortran'
  Setting CPPFLAGS to ''
  Setting FPPFLAGS to '-traditional'
  Setting CFLAGS to '-g -std=gnu99'
  Setting CXXFLAGS to '-g -std=gnu++11'
  Setting F90FLAGS to '-g -fcray-pointer -ffixed-line-length-none'
  Setting LDFLAGS to '-rdynamic'
  Setting DEBUG to 'no'
  Setting CPP_DEBUG_FLAGS to ''
  Setting C_DEBUG_FLAGS to ''
  Setting CXX_DEBUG_FLAGS to ''
  Setting OPTIMISE to 'yes'
  Setting CPP_OPTIMISE_FLAGS to ''
  Setting C_OPTIMISE_FLAGS to '-O2'
  Setting CXX_OPTIMISE_FLAGS to '-O2'
  Setting F90_OPTIMISE_FLAGS to '-O2'
  Setting PROFILE to 'no'
  Setting CPP_PROFILE_FLAGS to ''
  Setting C_PROFILE_FLAGS to '-pg'
  Setting CXX_PROFILE_FLAGS to '-pg'
  Setting F90_PROFILE_FLAGS to '-pg'
  Setting WARN to 'yes'
  Setting CPP_WARN_FLAGS to '-Wall'
  Setting C_WARN_FLAGS to '-Wall'
  Setting CXX_WARN_FLAGS to '-Wall'
  Setting F90_WARN_FLAGS to '-Wall'
  Setting OPENMP to 'yes'
  Setting CPP_OPENMP_FLAGS to '-fopenmp'
  Setting FPP_OPENMP_FLAGS to '-D_OPENMP'
  Setting C_OPENMP_FLAGS to '-fopenmp'
  Setting CXX_OPENMP_FLAGS to '-fopenmp'
  Setting F90_OPENMP_FLAGS to '-fopenmp'
  Setting VECTORISE to 'yes'
  Setting VECTORISE_ALIGNED_ARRAYS to 'no'
  Setting VECTORISE_INLINE to 'yes'
  Setting PTHREADS_DIR to 'NO_BUILD'
End of options from '/Users/rob/Cactus/configs/sim/OptionList'.
creating cache ./config.cache
checking host system type... x86_64-apple-darwin22.5.0
checking for mawk... no
checking for gawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets ${MAKE}... yes
checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) works... yes
checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) is a
cross-compiler... no
checking whether we are using GNU C... yes
checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) works... yes
checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) is a
cross-compiler... no
checking whether we are using GNU C++... yes
checking for nvcc... no
checking for ranlib... ranlib
checking for perl... perl
checking for gtar... no
checking for tar... tar
checking for xargs... xargs
checking for gzip... gzip
checking for gpatch... no
checking for patch... patch
checking for git... git
checking for ar... ar
checking for mkdir... mkdir
checking for bash... /bin/sh
Unknown MacOS X f90 compiler.
Please add appropriate information to
/Users/rob/Cactus/lib/make/known-architectures/darwin
and send the updated file to CactusMaint
We will try anyway ...
checking whether the Fortran compiler (gfortran  ) works... no
configure: error: installation or configuration problem: Fortran
compiler cannot create executables (see
configs/sim/config-data/config.log for details).

Error reconfiguring sim-config
make: *** [sim-config] Error 2


As you can see I get an error where it does not like the fortran compiler.

I have tried all of the precursor stuff a number of times using both
MacPorts and Homebrew and this does not seem to give any errors when
installing.


I seem to be going round in circles to resolve this, if you are able
to help that would be really appreciated.


Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230805/f819e376/attachment-0001.htm>

From rhaas at illinois.edu  Mon Aug  7 08:28:26 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Mon, 7 Aug 2023 08:28:26 -0500
Subject: [Users] Adding extra object file at linking stage
In-Reply-To: <DM6PR03MB3819F334124466CED4785221B40BA@DM6PR03MB3819.namprd03.prod.outlook.com>
References: <DM6PR03MB3819F334124466CED4785221B40BA@DM6PR03MB3819.namprd03.prod.outlook.com>
Message-ID: <20230807082826.53d85b71@ekohaes8>

Hello Elias,

This was held for moderation. For now I clicked on accept. Maybe
you need to register / update your email to emost at caltech.edu ?

To link in a single object file may be tricky. It may indeed be easier
to turn it into a lib file:

ar cru libfoo.a bar.o
ranlib libfoo.a

the add

LIBS=foo
LIBDIRS=DIR-WITH-FOO

and to you option list.

Directly adding object files is something that Erik just created a pull
request for (for different reasons, though who knows it was AMReX
related):

https://bitbucket.org/cactuscode/cactus/pull-requests/155

If you are in a thorn then there may be a way to fudge things as well,
though when quickly trying this I have not been successful.

Yours,
Roland

> Hi,
> 
> I have an extra object file ?.o? that was compiled externally, but needs to be linked in during the linking process.
> 
> Is there a standard way to include this within the build process?
> 
> Thank you,
> Elias
> 


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230807/4aa5fcd5/attachment.sig>

From rhaas at illinois.edu  Mon Aug  7 08:50:02 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Mon, 7 Aug 2023 08:50:02 -0500
Subject: [Users] Problem installing Einstein Toolkit on Mac
In-Reply-To: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
References: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
Message-ID: <20230807085002.190a9306@ekohaes8>

Hello Bob,

You said you followed the jupyter tutorial and the log output would
seem to indicate that you did, but just to be sure: did you make sure
to install the prerequisites listed on

https://github.com/einsteintoolkit/jupyter-et/blob/master/tutorial-server/notebooks/CactusTutorial.ipynb

ie for a Mac:

--8<--
On Mac, please first

    Install Xcode from the Apple App Store. In addition agree to Xcode license and install the Xcode Command Line Tools in Terminal

sudo xcodebuild -license
sudo xcode-select --install

    when using MacPorts
        install MacPorts for your version of the Mac operating system, if you have not already installed it (https://www.macports.org/install.php).
        Next, please install the following packages, using the commands:

sudo port -N install pkgconfig gcc12 openmpi-gcc12 fftw-3 gsl zlib openssl subversion ld64 hdf5 +fortran +gfortran
sudo port select mpi openmpi-gcc12-fortran
sudo port select gcc mp-gcc12

    when using HomeBrew
        install HomeBrew for your version of the Mac operating system, if you have not already installed it (https://brew.sh/).
        make sure to add /opt/homebrew/bin to your PATH as instructed by the installation script. You may need to restart your terminal.
        Next, please install the following packages, using the commands:

brew install fftw gcc gsl hdf5 hwloc jpeg open-mpi openssl zlib pkg-config bash subversion
brew link --force jpeg openssl zlib

--8<--

Also could you attach the config.log file that the error message refers
to, please? This is file configs/sim/config-data/config.log ? It will
contain the exact error message that the attempt to use gfortran as the
Fortran compiler produced.


Yours,
Roland

> Hi,
> 
> I hope someone can help.
> I have tried to install the Einstein Toolkit but am getting an error in the
> final part of the process on the Jupyter CactusTutorial:
> 
> %%bash
> ./simfactory/bin/sim build -j2 --thornlist thornlists/einsteintoolkit.th
> 
> Output is:
> 
> Using configuration: sim
> Reconfiguring sim
> Writing configuration to: /Users/rob/Cactus/configs/sim/OptionList
> Cactus - version: 4.14.0
> Reconfiguring sim.
> Using configuration options from configure line
>   Setting fds to '3,5 -j --'
> End of options from configure line
> Adding configuration options from '/Users/rob/Cactus/configs/sim/OptionList'...
>   Setting VERSION to '2018-12-13'
>   Setting CPP to 'cpp'
>   Setting CC to 'gcc'
>   Setting CXX to 'g++'
>   Setting FPP to 'cpp'
>   Setting F90 to 'gfortran'
>   Setting CPPFLAGS to ''
>   Setting FPPFLAGS to '-traditional'
>   Setting CFLAGS to '-g -std=gnu99'
>   Setting CXXFLAGS to '-g -std=gnu++11'
>   Setting F90FLAGS to '-g -fcray-pointer -ffixed-line-length-none'
>   Setting LDFLAGS to '-rdynamic'
>   Setting DEBUG to 'no'
>   Setting CPP_DEBUG_FLAGS to ''
>   Setting C_DEBUG_FLAGS to ''
>   Setting CXX_DEBUG_FLAGS to ''
>   Setting OPTIMISE to 'yes'
>   Setting CPP_OPTIMISE_FLAGS to ''
>   Setting C_OPTIMISE_FLAGS to '-O2'
>   Setting CXX_OPTIMISE_FLAGS to '-O2'
>   Setting F90_OPTIMISE_FLAGS to '-O2'
>   Setting PROFILE to 'no'
>   Setting CPP_PROFILE_FLAGS to ''
>   Setting C_PROFILE_FLAGS to '-pg'
>   Setting CXX_PROFILE_FLAGS to '-pg'
>   Setting F90_PROFILE_FLAGS to '-pg'
>   Setting WARN to 'yes'
>   Setting CPP_WARN_FLAGS to '-Wall'
>   Setting C_WARN_FLAGS to '-Wall'
>   Setting CXX_WARN_FLAGS to '-Wall'
>   Setting F90_WARN_FLAGS to '-Wall'
>   Setting OPENMP to 'yes'
>   Setting CPP_OPENMP_FLAGS to '-fopenmp'
>   Setting FPP_OPENMP_FLAGS to '-D_OPENMP'
>   Setting C_OPENMP_FLAGS to '-fopenmp'
>   Setting CXX_OPENMP_FLAGS to '-fopenmp'
>   Setting F90_OPENMP_FLAGS to '-fopenmp'
>   Setting VECTORISE to 'yes'
>   Setting VECTORISE_ALIGNED_ARRAYS to 'no'
>   Setting VECTORISE_INLINE to 'yes'
>   Setting PTHREADS_DIR to 'NO_BUILD'
> End of options from '/Users/rob/Cactus/configs/sim/OptionList'.
> creating cache ./config.cache
> checking host system type... x86_64-apple-darwin22.5.0
> checking for mawk... no
> checking for gawk... no
> checking for nawk... no
> checking for awk... awk
> checking whether make sets ${MAKE}... yes
> checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) works... yes
> checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) is a
> cross-compiler... no
> checking whether we are using GNU C... yes
> checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) works... yes
> checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) is a
> cross-compiler... no
> checking whether we are using GNU C++... yes
> checking for nvcc... no
> checking for ranlib... ranlib
> checking for perl... perl
> checking for gtar... no
> checking for tar... tar
> checking for xargs... xargs
> checking for gzip... gzip
> checking for gpatch... no
> checking for patch... patch
> checking for git... git
> checking for ar... ar
> checking for mkdir... mkdir
> checking for bash... /bin/sh
> Unknown MacOS X f90 compiler.
> Please add appropriate information to
> /Users/rob/Cactus/lib/make/known-architectures/darwin
> and send the updated file to CactusMaint
> We will try anyway ...
> checking whether the Fortran compiler (gfortran  ) works... no
> configure: error: installation or configuration problem: Fortran
> compiler cannot create executables (see
> configs/sim/config-data/config.log for details).
> 
> Error reconfiguring sim-config
> make: *** [sim-config] Error 2
> 
> 
> As you can see I get an error where it does not like the fortran compiler.
> 
> I have tried all of the precursor stuff a number of times using both
> MacPorts and Homebrew and this does not seem to give any errors when
> installing.
> 
> 
> I seem to be going round in circles to resolve this, if you are able
> to help that would be really appreciated.
> 
> 
> Rob


Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230807/7ef7b3ae/attachment.sig>

From rob6013 at gmail.com  Mon Aug  7 09:57:42 2023
From: rob6013 at gmail.com (Rob)
Date: Mon, 7 Aug 2023 15:57:42 +0100
Subject: [Users] Problem installing Einstein Toolkit on Mac
In-Reply-To: <20230807085002.190a9306@ekohaes8>
References: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
 <20230807085002.190a9306@ekohaes8>
Message-ID: <CAETPfP0pM==xDWXRPFKx_kR2TJcbCtQ7PmQk62c-spkBOsWzAQ@mail.gmail.com>

Thank you Roland for your reply,
I was just watching your Youtube video "Introduction to the Einstein
Toolkit", so seeing your name pop up was a bit of a surprise!!

I believe that I had previously done everything required and followed all
of the install instructions properly.  But I have now gone through them all
again and still have the same problem.
I have attached the config file you mentioned but it doesn't seem to give
any more details that are spooled in the Jupyter notebook when I run it.

Regards

Rob



On Mon, 7 Aug 2023 at 14:50, Roland Haas <rhaas at illinois.edu> wrote:

> Hello Bob,
>
> You said you followed the jupyter tutorial and the log output would
> seem to indicate that you did, but just to be sure: did you make sure
> to install the prerequisites listed on
>
>
> https://github.com/einsteintoolkit/jupyter-et/blob/master/tutorial-server/notebooks/CactusTutorial.ipynb
>
> ie for a Mac:
>
> --8<--
> On Mac, please first
>
>     Install Xcode from the Apple App Store. In addition agree to Xcode
> license and install the Xcode Command Line Tools in Terminal
>
> sudo xcodebuild -license
> sudo xcode-select --install
>
>     when using MacPorts
>         install MacPorts for your version of the Mac operating system, if
> you have not already installed it (https://www.macports.org/install.php).
>         Next, please install the following packages, using the commands:
>
> sudo port -N install pkgconfig gcc12 openmpi-gcc12 fftw-3 gsl zlib openssl
> subversion ld64 hdf5 +fortran +gfortran
> sudo port select mpi openmpi-gcc12-fortran
> sudo port select gcc mp-gcc12
>
>     when using HomeBrew
>         install HomeBrew for your version of the Mac operating system, if
> you have not already installed it (https://brew.sh/).
>         make sure to add /opt/homebrew/bin to your PATH as instructed by
> the installation script. You may need to restart your terminal.
>         Next, please install the following packages, using the commands:
>
> brew install fftw gcc gsl hdf5 hwloc jpeg open-mpi openssl zlib pkg-config
> bash subversion
> brew link --force jpeg openssl zlib
>
> --8<--
>
> Also could you attach the config.log file that the error message refers
> to, please? This is file configs/sim/config-data/config.log ? It will
> contain the exact error message that the attempt to use gfortran as the
> Fortran compiler produced.
>
>
> Yours,
> Roland
>
> > Hi,
> >
> > I hope someone can help.
> > I have tried to install the Einstein Toolkit but am getting an error in
> the
> > final part of the process on the Jupyter CactusTutorial:
> >
> > %%bash
> > ./simfactory/bin/sim build -j2 --thornlist thornlists/einsteintoolkit.th
> >
> > Output is:
> >
> > Using configuration: sim
> > Reconfiguring sim
> > Writing configuration to: /Users/rob/Cactus/configs/sim/OptionList
> > Cactus - version: 4.14.0
> > Reconfiguring sim.
> > Using configuration options from configure line
> >   Setting fds to '3,5 -j --'
> > End of options from configure line
> > Adding configuration options from
> '/Users/rob/Cactus/configs/sim/OptionList'...
> >   Setting VERSION to '2018-12-13'
> >   Setting CPP to 'cpp'
> >   Setting CC to 'gcc'
> >   Setting CXX to 'g++'
> >   Setting FPP to 'cpp'
> >   Setting F90 to 'gfortran'
> >   Setting CPPFLAGS to ''
> >   Setting FPPFLAGS to '-traditional'
> >   Setting CFLAGS to '-g -std=gnu99'
> >   Setting CXXFLAGS to '-g -std=gnu++11'
> >   Setting F90FLAGS to '-g -fcray-pointer -ffixed-line-length-none'
> >   Setting LDFLAGS to '-rdynamic'
> >   Setting DEBUG to 'no'
> >   Setting CPP_DEBUG_FLAGS to ''
> >   Setting C_DEBUG_FLAGS to ''
> >   Setting CXX_DEBUG_FLAGS to ''
> >   Setting OPTIMISE to 'yes'
> >   Setting CPP_OPTIMISE_FLAGS to ''
> >   Setting C_OPTIMISE_FLAGS to '-O2'
> >   Setting CXX_OPTIMISE_FLAGS to '-O2'
> >   Setting F90_OPTIMISE_FLAGS to '-O2'
> >   Setting PROFILE to 'no'
> >   Setting CPP_PROFILE_FLAGS to ''
> >   Setting C_PROFILE_FLAGS to '-pg'
> >   Setting CXX_PROFILE_FLAGS to '-pg'
> >   Setting F90_PROFILE_FLAGS to '-pg'
> >   Setting WARN to 'yes'
> >   Setting CPP_WARN_FLAGS to '-Wall'
> >   Setting C_WARN_FLAGS to '-Wall'
> >   Setting CXX_WARN_FLAGS to '-Wall'
> >   Setting F90_WARN_FLAGS to '-Wall'
> >   Setting OPENMP to 'yes'
> >   Setting CPP_OPENMP_FLAGS to '-fopenmp'
> >   Setting FPP_OPENMP_FLAGS to '-D_OPENMP'
> >   Setting C_OPENMP_FLAGS to '-fopenmp'
> >   Setting CXX_OPENMP_FLAGS to '-fopenmp'
> >   Setting F90_OPENMP_FLAGS to '-fopenmp'
> >   Setting VECTORISE to 'yes'
> >   Setting VECTORISE_ALIGNED_ARRAYS to 'no'
> >   Setting VECTORISE_INLINE to 'yes'
> >   Setting PTHREADS_DIR to 'NO_BUILD'
> > End of options from '/Users/rob/Cactus/configs/sim/OptionList'.
> > creating cache ./config.cache
> > checking host system type... x86_64-apple-darwin22.5.0
> > checking for mawk... no
> > checking for gawk... no
> > checking for nawk... no
> > checking for awk... awk
> > checking whether make sets ${MAKE}... yes
> > checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) works...
> yes
> > checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) is a
> > cross-compiler... no
> > checking whether we are using GNU C... yes
> > checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic)
> works... yes
> > checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) is a
> > cross-compiler... no
> > checking whether we are using GNU C++... yes
> > checking for nvcc... no
> > checking for ranlib... ranlib
> > checking for perl... perl
> > checking for gtar... no
> > checking for tar... tar
> > checking for xargs... xargs
> > checking for gzip... gzip
> > checking for gpatch... no
> > checking for patch... patch
> > checking for git... git
> > checking for ar... ar
> > checking for mkdir... mkdir
> > checking for bash... /bin/sh
> > Unknown MacOS X f90 compiler.
> > Please add appropriate information to
> > /Users/rob/Cactus/lib/make/known-architectures/darwin
> > and send the updated file to CactusMaint
> > We will try anyway ...
> > checking whether the Fortran compiler (gfortran  ) works... no
> > configure: error: installation or configuration problem: Fortran
> > compiler cannot create executables (see
> > configs/sim/config-data/config.log for details).
> >
> > Error reconfiguring sim-config
> > make: *** [sim-config] Error 2
> >
> >
> > As you can see I get an error where it does not like the fortran
> compiler.
> >
> > I have tried all of the precursor stuff a number of times using both
> > MacPorts and Homebrew and this does not seem to give any errors when
> > installing.
> >
> >
> > I seem to be going round in circles to resolve this, if you are able
> > to help that would be really appreciated.
> >
> >
> > Rob
>
>
> Yours,
> Roland
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230807/3e47241e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config.log
Type: application/octet-stream
Size: 2278 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230807/3e47241e/attachment-0001.obj>

From rhaas at illinois.edu  Mon Aug  7 10:27:07 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Mon, 7 Aug 2023 10:27:07 -0500
Subject: [Users] Problem installing Einstein Toolkit on Mac
In-Reply-To: <CAETPfP0pM==xDWXRPFKx_kR2TJcbCtQ7PmQk62c-spkBOsWzAQ@mail.gmail.com>
References: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
 <20230807085002.190a9306@ekohaes8>
 <CAETPfP0pM==xDWXRPFKx_kR2TJcbCtQ7PmQk62c-spkBOsWzAQ@mail.gmail.com>
Message-ID: <20230807102707.361c265c@ekohaes8>

Hello Bob,

> I believe that I had previously done everything required and followed all
> of the install instructions properly.  But I have now gone through them all
> again and still have the same problem.
> I have attached the config file you mentioned but it doesn't seem to give
> any more details that are spooled in the Jupyter notebook when I run it.

Let me see. In you case it looks for just gfortran. This is admittedly
a bit odd. On MacOS using homebrew or macports it should look for
gfortran-13 or gfortran-mp-13 respectively.

I have seen issues with Macs that change their machine name since they
get derive it from their WiFi IP address. So here's my suggestion:

* run hostname -f to find out your laptops (current) machine name

* compare to the output of `simfactory/bin/sim whoami` which should be
  similar

* look for the newest file simfactory/mdb/machines/*.ini . It should
  have quite a long name that looks similar to the output of hostname
  above

* if the file name differs (other than in having the .ini extension)
  then you laptop got a new name since you set up Cactus and you will
  need to re-run `simfactory/bin/sim setup-silent` once more. It should
  (just above the SUMMARY output) say something along the lines of:

Creating machine gdd6l52n-ofc.ncsa.illinois.edu from generic: machine gdd6l52n-ofc.ncsa.illinois.edu [/Users/rhaas/ET_Next/repos/simfactory2/mdb/machines/gdd6l52n-ofc.ncsa.illinois.edu.ini] created successfully

* take a look at the ini file mentioned, and look for the "optionlist"
  entry. There should be a file of that name in
  simfactory/mdb/optionlists and that should contain the correct
  gfortran in its F90 entry

If you laptop does indeed change names, you can force simfactory to use
a specific name by creating a file $HOME/.hostname and writing the
desired machine name to it (eg gdd6l52n-ofc.ncsa.illinois.edu in my
example above). You can also try and pass `--machine
gdd6l52n-ofc.ncsa.illinois.edu` (for example) to each simfactory
command which has the same effect (or at least should).

If any of these things turn out to be not true, could you include the
full output from `setup-silent` (text, not a screenshot please) as well
as the ini file and cfg files (machine definition and option list),
please? 

Also which package manager you are using (homebrew or macports) as well
as the output from `which gfortran-13` or `which gfortran-mp-13` as
appropriate both from within a Terminal and from withint the jupyter
notebook (in a %%bash cell or using !).

Finally of course, sometimes the easiest way to clear out mysterious
errors is to wipe Cactus (`rm -rf ~/Cactus`) and start from scratch (so
that no lingering options are left behind).

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230807/dc712168/attachment.sig>

From rhaas at illinois.edu  Mon Aug  7 15:18:02 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Mon, 07 Aug 2023 15:18:02 -0500
Subject: [Users] Agenda for Thursday's Meeting
Message-ID: <E1qT6fy-003oBU-00@ekohaes8>

Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers

From thejasanair275 at gmail.com  Tue Aug  8 04:10:44 2023
From: thejasanair275 at gmail.com (Thejas A Nair)
Date: Tue, 8 Aug 2023 14:40:44 +0530
Subject: [Users] Error while running toolkit
In-Reply-To: <20230801091258.7d62d661@ekohaes8>
References: <CA+a7ynMbnRpxdXrwxDPRhH3Xb+fJNUaQ1G_cEfBtMu9m4zMSSg@mail.gmail.com>
 <20230722211550.6beeef0d@fdea4908.spdns.org>
 <CA+a7ynOq3Tq34guWbJVRqjP1VVDm_9WxBdB69Pn88JZiHdCTbg@mail.gmail.com>
 <20230731161527.598f2073@ekohaes8>
 <CA+a7ynO9vxDReP-Wwbp0zMsE_uP9Eh3h_cbeSjT=cX0_4youPg@mail.gmail.com>
 <20230801091258.7d62d661@ekohaes8>
Message-ID: <CA+a7ynPyDGAu9CAHW-4JDoCnpd0M1+AVvuQFYA5_fzcUVg9QNQ@mail.gmail.com>

>
> >>The gallery example gives a thornlist (ET Schwarzschild release
> (ET_2023_05)) that is known to work with the parfile. Have you tried
> that one? I am asking because using it with GetComponents and
> simfactory should really give you a working ADMBase.
>
 Yes I have, but I am not able to run the poisson equation.

>
> If all else fails I would suggest starting from scratch, that is remove
> your whole Cactus directory, run GetComponents and simfactory again as
> described on
> >> I did that, currently the problem I am facing a problem, I have
> executed the static tov par file it was compiled but no output. In the
> older version the output would be in simulation, but not in the 2023
> version. This is the output
>


Iteration      Time |              ADMBASE::alp |            HYDROBASE::rho
                    |      minimum      maximum |      minimum      maximum
---------------------------------------------------------------------------
   256000  1000.000 |    0.6686480    0.9966259 | 1.000000e-10    0.0012830
INFO (Carpet): Terminating due to cctk_final_time at t = 1000.000000
--------------------------------------------------------------------------------
Done.
+ echo Stopping:
+ date
Stopping:
Tuesday 08 August 2023 01:29:38 PM IST
+ echo Done.
Done.
Simfactory Done at date: Tuesday 08 August 2023 01:29:38 PM

>> one more question is there any difference between tov_et and static_tov par file, apart from grid size.


>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230808/056f64e5/attachment.htm>

From rob6013 at gmail.com  Tue Aug  8 10:45:45 2023
From: rob6013 at gmail.com (Rob)
Date: Tue, 8 Aug 2023 16:45:45 +0100
Subject: [Users] Problem installing Einstein Toolkit on Mac
In-Reply-To: <20230807102707.361c265c@ekohaes8>
References: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
 <20230807085002.190a9306@ekohaes8>
 <CAETPfP0pM==xDWXRPFKx_kR2TJcbCtQ7PmQk62c-spkBOsWzAQ@mail.gmail.com>
 <20230807102707.361c265c@ekohaes8>
Message-ID: <CAETPfP2ksi3TB2zjNAzKCH0F3fiK35LGZMuveE9uxrGR+iYWjg@mail.gmail.com>

Hi Roland,

Thanks for all of your help on this.  I'm still failing dismally.  I have
made some notes against yours below and attached relevant files.

On Mon, 7 Aug 2023 at 16:27, Roland Haas <rhaas at illinois.edu> wrote:

> Hello Bob,
>
> > I believe that I had previously done everything required and followed all
> > of the install instructions properly.  But I have now gone through them
> all
> > again and still have the same problem.
> > I have attached the config file you mentioned but it doesn't seem to give
> > any more details that are spooled in the Jupyter notebook when I run it.
>
> Let me see. In you case it looks for just gfortran. This is admittedly
> a bit odd. On MacOS using homebrew or macports it should look for
> gfortran-13 or gfortran-mp-13 respectively.
>


> I have seen issues with Macs that change their machine name since they
> get derive it from their WiFi IP address. So here's my suggestion:
>
> * run hostname -f to find out your laptops (current) machine name
>
Robs-Mac-mini.local

>
> * compare to the output of `simfactory/bin/sim whoami` which should be
>   similar
>
Current machine: Robs-Mac-mini.local

>
> * look for the newest file simfactory/mdb/machines/*.ini . It should
>   have quite a long name that looks similar to the output of hostname
>   above
> INI file attached below:::::  The name is exactly the same



>
> * if the file name differs (other than in having the .ini extension)
>   then you laptop got a new name since you set up Cactus and you will
>   need to re-run `simfactory/bin/sim setup-silent` once more. It should
>   (just above the SUMMARY output) say something along the lines of:
>
> Creating machine gdd6l52n-ofc.ncsa.illinois.edu from generic: machine
> gdd6l52n-ofc.ncsa.illinois.edu
> [/Users/rhaas/ET_Next/repos/simfactory2/mdb/machines/gdd6l52n-ofc.ncsa.illinois.edu.ini]
> created successfully
>
> * take a look at the ini file mentioned, and look for the "optionlist"
>   entry. There should be a file of that name in
>   simfactory/mdb/optionlists and that should contain the correct
>   gfortran in its F90 entry
>
The optionlist mentioned is : optionlist      = generic.cfg
I have also attached this optionlist.  It refers to F90 = gfortran .


>
> If you laptop does indeed change names, you can force simfactory to use
> a specific name by creating a file $HOME/.hostname and writing the
> desired machine name to it (eg gdd6l52n-ofc.ncsa.illinois.edu in my
> example above). You can also try and pass `--machine
> gdd6l52n-ofc.ncsa.illinois.edu` (for example) to each simfactory
> command which has the same effect (or at least should).
>
> If any of these things turn out to be not true, could you include the
> full output from `setup-silent` (text, not a screenshot please) as well
> as the ini file and cfg files (machine definition and option list),
> please?
>
> Also which package manager you are using (homebrew or macports) as well
> as the output from `which gfortran-13` /opt/homebrew/bin/gfortran-13or
> `which gfortran-mp-13` gfortran-mp-13 not found as

appropriate both from within a Terminal and from withint the jupyter
> notebook (in a %%bash cell or using !). (*/opt/homebrew/bin/gfortran-13*
> and *gfortran-mp-13 not found* respectively)
>
II originally used MacPorts but when this failed I did the Homebrew stuff
as well.  Is this okay or will this now have led to some conflicts?

>
> Finally of course, sometimes the easiest way to clear out mysterious
> errors is to wipe Cactus (`rm -rf ~/Cactus`) and start from scratch (so
> that no lingering options are left behind).
>
I tried this and had the same problem.

I have also now tried installing VMWare and creating an Ubuntu virtual
machine but I could not get Jupyter Notebook to install on my 65bitArm
version and my shell experience is really patchy.  I feel like I've just
turned up at medical school and have been given a heart transplant to
perform!!!

I think my biggest problem here is my understanding of the underling
stuff.  I've programmed for years but in C#, Java, Python etc so rarely get
my hands dirty in the back-end.  I suspect I need to remove/uninstall some
more stuff and start again but wouldn't know where to start with this.
I've only been running this new Mac-mini for a few months so if I felt I
had a good chance of success I would be tempted to do a full clean
re-install of Ventura since all of my data is cloud backed.

>
> Yours,
> Roland
>
> --
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://pgp.mit.edu .
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230808/6495102a/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Robs-Mac-mini.local.ini
Type: application/octet-stream
Size: 1952 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230808/6495102a/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: generic.cfg
Type: application/octet-stream
Size: 1205 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230808/6495102a/attachment-0003.obj>

From rhaas at illinois.edu  Wed Aug  9 17:15:01 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Wed, 09 Aug 2023 17:15:01 -0500
Subject: [Users] Einstein Toolkit Meeting Reminder
Message-ID: <E1qTrSH-006ZFK-0G@ekohaes8>

Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers

From thejasanair275 at gmail.com  Thu Aug 10 06:28:00 2023
From: thejasanair275 at gmail.com (Thejas A Nair)
Date: Thu, 10 Aug 2023 16:58:00 +0530
Subject: [Users] (no subject)
Message-ID: <CA+a7ynO5zJSiatA3BcFvM8V3xT9cTUYnfeD7KDgCia38XH08rw@mail.gmail.com>

Hi,
I had a doubt about this graph, Why do we have a 'Mass of the sun'
parameter in the x-axis, What is the logic behind it?
Warm Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230810/85707e8a/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: first.png
Type: image/png
Size: 34301 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230810/85707e8a/attachment-0001.png>

From rhaas at illinois.edu  Thu Aug 10 09:46:42 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Thu, 10 Aug 2023 09:46:42 -0500
Subject: [Users] meeting minutes for 2023-08-10
Message-ID: <20230810094642.5787a685@fdea4908.spdns.org>

Present: Roland, Leo, Zach, Peter, Sam, Yosef

ET release
==========

* reviewers needed for AsterX. Roland to poke Bruno and EU people.
* Yosef is reviewer for SpaceTimeX, still not specific list of thorns
  to include. Sam will ask for final list on mailing list. Thorns not
  volunteered and champions stepping forward will be dropped from the
  release.
* Reviewer found for SGRID
* need testsuite runners and gallery runners
** Steve would do BBH gallery example again
** please suggest possible helpers
* Sam created / reactivated tickets for release

ET summer school
================

* recordings are still being processed (Yosef is the sole editor right
  now)

Unanswered emails
=================

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-August/009021.html
  Roland will try and follow up, but is not sure what is going on.
  Might be simfactory failure or might be in simfactory or some
  homebrew / machports / user editing issue

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-August/009020.html
  Roland had been in contact. Not sure what is going on, might be an
  issue with simfactory setting up new segments instead of creating a
  new simulation.


Tickets
=======

*
  https://bitbucket.org/einsteintoolkit/tickets/issues/2706/update-default-twopunctures-parameters-or
  Zach will coordinate with Steve about TP paramters

* pull request
  https://bitbucket.org/cactuscode/cactusnumerical/pull-requests/13
  Peter will take a look to see what / if needs to be done

Tickets for review
==================

* no updates for most of them
*
  https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy
  Peter will look into the after Zach's paper with improved methods is
  out
*
  https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-triggering-syncs-for-all
  issue is triggered if there are no b/c to apply, then PreSync will
  try and SYNC + bc every time. There probably is some better fix for
  this. Some fine tuning of warnings / errors may be required.

chair next week: Peter
minutes taker next week: Sam 

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230810/dd7d36e3/attachment.sig>

From diener at cct.lsu.edu  Thu Aug 10 12:46:15 2023
From: diener at cct.lsu.edu (Peter Diener)
Date: Thu, 10 Aug 2023 12:46:15 -0500 (CDT)
Subject: [Users] (no subject)
In-Reply-To: <CA+a7ynO5zJSiatA3BcFvM8V3xT9cTUYnfeD7KDgCia38XH08rw@mail.gmail.com>
References: <CA+a7ynO5zJSiatA3BcFvM8V3xT9cTUYnfeD7KDgCia38XH08rw@mail.gmail.com>
Message-ID: <fbe1d3eb-37d5-8596-5e8d-b84c223dccdb@cct.lsu.edu>

Hi Thejas,

In the Einstein Toolkit, we use, as is common in numerical relativity, 
units where the speed of light, c, and the gravitational constant, G, are 
chosen to be c=G=1. In addition, we measure mass in units of the solar 
mass, M_sun. 1 solar mass can be converted into a length by multiplying by 
G and dividing by c^2. This is then the unit of length. We can convert 
that into a time by dividing by c. So the unit of length and time in SI 
units are then

[L] = 1 M_sun*G/c^2 = 1477 m.
[T] = 1 M_sun*G/c^3 = 4.93 10^-6 s.

As can be seen, the unit of length is 1 half the Scharzschild radius of a 
black hole with a mass of 1 solar mass. The unit if time is then the time 
it takes light to travel that distance.

Cheers,

   Peter

On Thu, 10 Aug 2023, Thejas A Nair wrote:

> Hi,
> I had a doubt about this graph, Why do we have a 'Mass of the sun' parameter
> in the x-axis, What is the logic behind it?
> Warm Regards
> 
>

From thejasanair275 at gmail.com  Thu Aug 10 12:54:54 2023
From: thejasanair275 at gmail.com (Thejas A Nair)
Date: Thu, 10 Aug 2023 23:24:54 +0530
Subject: [Users] (no subject)
In-Reply-To: <fbe1d3eb-37d5-8596-5e8d-b84c223dccdb@cct.lsu.edu>
References: <CA+a7ynO5zJSiatA3BcFvM8V3xT9cTUYnfeD7KDgCia38XH08rw@mail.gmail.com>
 <fbe1d3eb-37d5-8596-5e8d-b84c223dccdb@cct.lsu.edu>
Message-ID: <CA+a7ynO1tM111vEf=gF6ceM=Zc7L3DogCTGr0ef2gF=M==bY5Q@mail.gmail.com>

Thank you so much for the info!

On Thu, 10 Aug 2023, 11:16 pm Peter Diener, <diener at cct.lsu.edu> wrote:

> Hi Thejas,
>
> In the Einstein Toolkit, we use, as is common in numerical relativity,
> units where the speed of light, c, and the gravitational constant, G, are
> chosen to be c=G=1. In addition, we measure mass in units of the solar
> mass, M_sun. 1 solar mass can be converted into a length by multiplying by
> G and dividing by c^2. This is then the unit of length. We can convert
> that into a time by dividing by c. So the unit of length and time in SI
> units are then
>
> [L] = 1 M_sun*G/c^2 = 1477 m.
> [T] = 1 M_sun*G/c^3 = 4.93 10^-6 s.
>
> As can be seen, the unit of length is 1 half the Scharzschild radius of a
> black hole with a mass of 1 solar mass. The unit if time is then the time
> it takes light to travel that distance.
>
> Cheers,
>
>    Peter
>
> On Thu, 10 Aug 2023, Thejas A Nair wrote:
>
> > Hi,
> > I had a doubt about this graph, Why do we have a 'Mass of the sun'
> parameter
> > in the x-axis, What is the logic behind it?
> > Warm Regards
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230810/37b75743/attachment.htm>

From scupp1 at my.apsu.edu  Fri Aug 11 12:43:01 2023
From: scupp1 at my.apsu.edu (Cupp, Samuel D.)
Date: Fri, 11 Aug 2023 17:43:01 +0000
Subject: [Users] Code champions for SpacetimeX thorns in ET_2023_11 release
Message-ID: <DM6PR01MB4011326680ABD0DED86BFFB3D910A@DM6PR01MB4011.prod.exchangelabs.com>

Hello all,
   Along with the proposed addition of the new CarpetX driver is a collection of other thorns reproducing features available in Carpet. Some of these are housed in the SpacetimeX arrangement, which is proposed for inclusion alongside CarpetX. However, I am not aware of who is 'championing' the individual thorns. Normally, this is one of the authors of the thorn who responds to any questions from reviewers, makes requested changes/additions, etc. Currently, I do not have anyone who is responsible for these thorns, at least in an official capacity. Can someone who contributed to writing these thorns please contact me in regard to who is serving as the 'code champion'? If no one is willing to take responsibility for them, they will not be included in the release.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230811/1fe156af/attachment-0001.htm>

From thejasanair275 at gmail.com  Sat Aug 12 02:25:14 2023
From: thejasanair275 at gmail.com (Thejas A Nair)
Date: Sat, 12 Aug 2023 12:55:14 +0530
Subject: [Users] (no subject)
Message-ID: <CA+a7ynMjojDp84NPsqAENVyaCXiT0rzmjRKxpbZMdqjseXGXAA@mail.gmail.com>

I installed the toolkit 2023 version, and it showed an error:

Parameter file:
/home/thejas/Cactus/arrangements/CactusExamples/HelloWorld/par/HelloWorld.par
Error: Executable /home/thejas/Cactus/exe/cactus_sim for configuration
sim does not exist or is not readable
Aborting Simfactory.


previously I used the 2022 version and did the tov code, and got the
output. I am now trying to do the multipatch wave equation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230812/2876d293/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CactusTutorialnew.ipynb
Type: application/x-ipynb+json
Size: 41521 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230812/2876d293/attachment-0001.bin>

From rhaas at illinois.edu  Sat Aug 12 09:04:52 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Sat, 12 Aug 2023 10:04:52 -0400
Subject: [Users] (no subject)
In-Reply-To: <CA+a7ynMjojDp84NPsqAENVyaCXiT0rzmjRKxpbZMdqjseXGXAA@mail.gmail.com>
References: <CA+a7ynMjojDp84NPsqAENVyaCXiT0rzmjRKxpbZMdqjseXGXAA@mail.gmail.com>
Message-ID: <20230812100452.138114ed@fdea4908.spdns.org>

Hello Thejas,

> I installed the toolkit 2023 version, and it showed an error:
> 
> Parameter file:
> /home/thejas/Cactus/arrangements/CactusExamples/HelloWorld/par/HelloWorld.par
> Error: Executable /home/thejas/Cactus/exe/cactus_sim for configuration
> sim does not exist or is not readable
> Aborting Simfactory.
> 
> 
> previously I used the 2022 version and did the tov code, and got the
> output. I am now trying to do the multipatch wave equation.

Not finding the executable (cactus_sim) would mean that compilation
failed. 

Do you see any error message when running

simfactory/bin/sim build

? If so, please try and follow the suggestions here:

http://einsteintoolkit.org/support.html#general-guidelines-for-questions

to provide all log file output.

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230812/9825c6f7/attachment.sig>

From rhaas at illinois.edu  Sat Aug 12 09:16:33 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Sat, 12 Aug 2023 10:16:33 -0400
Subject: [Users] Error while running toolkit
In-Reply-To: <CA+a7ynPyDGAu9CAHW-4JDoCnpd0M1+AVvuQFYA5_fzcUVg9QNQ@mail.gmail.com>
References: <CA+a7ynMbnRpxdXrwxDPRhH3Xb+fJNUaQ1G_cEfBtMu9m4zMSSg@mail.gmail.com>
 <20230722211550.6beeef0d@fdea4908.spdns.org>
 <CA+a7ynOq3Tq34guWbJVRqjP1VVDm_9WxBdB69Pn88JZiHdCTbg@mail.gmail.com>
 <20230731161527.598f2073@ekohaes8>
 <CA+a7ynO9vxDReP-Wwbp0zMsE_uP9Eh3h_cbeSjT=cX0_4youPg@mail.gmail.com>
 <20230801091258.7d62d661@ekohaes8>
 <CA+a7ynPyDGAu9CAHW-4JDoCnpd0M1+AVvuQFYA5_fzcUVg9QNQ@mail.gmail.com>
Message-ID: <20230812101633.5b37b56c@fdea4908.spdns.org>

Hello Thejas,

> > >>The gallery example gives a thornlist (ET Schwarzschild release  
> > (ET_2023_05)) that is known to work with the parfile. Have you tried
> > that one? I am asking because using it with GetComponents and
> > simfactory should really give you a working ADMBase.
> >  
>  Yes I have, but I am not able to run the poisson equation.

Hmm, really, when using the release thornlist linked on the gallery
page:

http://einsteintoolkit.org/gallery/poisson/index.html

it should have ADMBase present and you should not see the error you were
apparently seeing. I am not quite sure what may be going on.

> > >> I did that, currently the problem I am facing a problem, I have  
> > executed the static tov par file it was compiled but no output. In the
> > older version the output would be in simulation, but not in the 2023
> > version. This is the output
> >  
> 
> 
> Iteration      Time |              ADMBASE::alp |            HYDROBASE::rho
>                     |      minimum      maximum |      minimum      maximum
> ---------------------------------------------------------------------------
>    256000  1000.000 |    0.6686480    0.9966259 | 1.000000e-10    0.0012830
> INFO (Carpet): Terminating due to cctk_final_time at t = 1000.000000
> --------------------------------------------------------------------------------
> Done.
> + echo Stopping:
> + date
> Stopping:
> Tuesday 08 August 2023 01:29:38 PM IST
> + echo Done.
> Done.
> Simfactory Done at date: Tuesday 08 August 2023 01:29:38 PM

This does indicate that it ran to completion though (ran until
t=1000.0), so there is at least output to screen.

If this is from a 2023 checkout (though for 2023 the gallery example is
no longer static_tov see below), then the output should be in 

$HOME/simulations/static_tov/output-0000

If you have a directory $HOME/simulations/static_tov/output-0001 or
other higher numbers present then what happened is that the old
simulation was not removed (using simfactory/bin/sim purge static_tov)
before "simfactory/bin/sim submit static_tov --parfile ..." was
executed. In that case simfactory detects the existing simulation and
will instead add a new segment (output directory) to it. So if you see
any "ouput-0001" or higher directories, then you should either remove
them before you run "sim submit" or instead pass a new simulation name
eg static_tov_01 to "sim submit". Using "sim create" and "sim submit"
separately can help catch these issues since "sim create" will abort
and show an error message if the simulation already exists.

The new release updated that parameter files and

http://einsteintoolkit.org/gallery/ns/index.html

now lists three choices for parameter files

tov_gallery_polytrop.par tov_gallery_polytrop_med.par tov_gallery_polytrop_hi.par tov_gallery_polytrop_hi2.par

These also fail?

> >> one more question is there any difference between tov_et and static_tov par file, apart from grid size.  

tov_et is very much cut down to run on the tutorial server in 10
minutes or so. Almost everything, including physical correctness, if
given up to achieve this. So really, tov_et should only ever be used in
the 10minutes run on the tutorial server. The gallery example however
is (should be) a production setup, except for the potentially lower
resolution So I would expect quite a number of differences between the
two.

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230812/fdaf80d6/attachment.sig>

From rhaas at illinois.edu  Sat Aug 12 09:41:08 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Sat, 12 Aug 2023 10:41:08 -0400
Subject: [Users] Problem installing Einstein Toolkit on Mac
In-Reply-To: <CAETPfP2ksi3TB2zjNAzKCH0F3fiK35LGZMuveE9uxrGR+iYWjg@mail.gmail.com>
References: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
 <20230807085002.190a9306@ekohaes8>
 <CAETPfP0pM==xDWXRPFKx_kR2TJcbCtQ7PmQk62c-spkBOsWzAQ@mail.gmail.com>
 <20230807102707.361c265c@ekohaes8>
 <CAETPfP2ksi3TB2zjNAzKCH0F3fiK35LGZMuveE9uxrGR+iYWjg@mail.gmail.com>
Message-ID: <20230812104108.6162ce17@fdea4908.spdns.org>

Hello Rob,

> Thanks for all of your help on this.  I'm still failing dismally.  I have
> made some notes against yours below and attached relevant files.

Thank you.


> > I have seen issues with Macs that change their machine name since they
> > get derive it from their WiFi IP address. So here's my suggestion:
> >
> > * run hostname -f to find out your laptops (current) machine name
> >  
> Robs-Mac-mini.local
> 
> >
> > * compare to the output of `simfactory/bin/sim whoami` which should be
> >   similar
> >  
> Current machine: Robs-Mac-mini.local

Ok, that all looks ok to me.

> > INI file attached below:::::  The name is exactly the same  

That would also be ok.

> The optionlist mentioned is : optionlist      = generic.cfg
> I have also attached this optionlist.  It refers to F90 = gfortran .

Very strange. The only way that, on a mac, I could think of ending up
with a setup like this would be if one had run

simfactory/bin/sim setup-silent

before having installed gfortran and gcc from homebrew or machports. Or
really having run the command when say gcc-13 was not in $PATH.

Note that running 

simfactory/bin/sim setup-silent

*then* installing macport and running 

simfactory/bin/sim setup-silent

will do you no good. You have to first remove the ini file
robs-mac-mini.local.ini (or the whole Cactus tree of course), otherwise
simfactory treats your mac mini as a "known machine" and does not
generate a ini file for it (and instead uses the existing one).

Though, with Homebrew, there actually is a gfortran file installed by
Homebrew that is (at least for me) in $PATH and should be usable.


> appropriate both from within a Terminal and from withint the jupyter
> > notebook (in a %%bash cell or using !). (*/opt/homebrew/bin/gfortran-13*
> > and *gfortran-mp-13 not found* respectively)

Ok, so something is messed up with the Homebrew and MacPorts install
since they did not add their "bin" directories to $PATH. Not sure why.

You can manually edit generic.cfg and instead of just having

F90 = gfortran

add the full path, so something like

F90 = /opt/homebrew/bin/gfortran

and similar for the other compilers (so that it does *not* use Apple's
clang-in-gcc-disguise in /usr/bin).

This may help. Note that generic.cfg is part of the git checkout, so a
git pull in simfactory2 will net you conflicts.

> II originally used MacPorts but when this failed I did the Homebrew stuff
> as well.  Is this okay or will this now have led to some conflicts?

Could be ok, though mixing those two can cause trouble in case some
libraries and tools are used from one and some from the other. Then
adding eg conda to the mix would tend to make things even worse.

> I have also now tried installing VMWare and creating an Ubuntu virtual
> machine but I could not get Jupyter Notebook to install on my 65bitArm
> version and my shell experience is really patchy.  I feel like I've just
> turned up at medical school and have been given a heart transplant to
> perform!!!

Yes, that would seem too much effort. Really it should work. My guess
would be that you may have (wisely) avoided adding any of Homebrew's or
MacPort's bin directories to your $PATH and this is the source of the
problem. You could try this by removing the ini file, adding eg
Homebrew's bin to PATH in a Terminal and running simfactory/bin/sim
setup-silent and sim build in that terminal. If that works, then you
have a way to make things work and have to decide on a way to switch
between Homebrew and non-homebrew setups depending on what you would
like to do.

> I think my biggest problem here is my understanding of the underling
> stuff.  I've programmed for years but in C#, Java, Python etc so rarely get
> my hands dirty in the back-end.  I suspect I need to remove/uninstall some
> more stuff and start again but wouldn't know where to start with this.
> I've only been running this new Mac-mini for a few months so if I felt I
> had a good chance of success I would be tempted to do a full clean
> re-install of Ventura since all of my data is cloud backed.

Simfactory is a wrapper around (among other things) the Cactus build
system. With the cfg files being option list files as described in the
Cactus user guide:

http://einsteintoolkit.org/usersguide/UsersGuide.html#x1-18000B2

which are (almost) makefile fragments (though they are first parsed by
Cactus so you cannot use arbitrary make constructs in them, eg
variable expansion using $(FOO) does not work).

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230812/658250d4/attachment-0001.sig>

From thejasanair275 at gmail.com  Sun Aug 13 04:14:00 2023
From: thejasanair275 at gmail.com (Thejas A Nair)
Date: Sun, 13 Aug 2023 14:44:00 +0530
Subject: [Users] (no subject)
Message-ID: <CA+a7ynMt-8=moWF4W06hao=JftEh5kba43JDLNtL=6wBiAfAvA@mail.gmail.com>

Hi,
I would like to know more physics about the single stable neutron star
solved using the toolkit as part of my master's project. Would you be able
to throw some light on this?
And I have a question about the time scales used in this particular
problem, they are of order 10^-6, usually, neutron stars are highly compact
objects,  In this problem, the density was of the order 10^-3 why are these
scales chosen, and this paper said (https://arxiv.org/abs/1111.3344.) the
spikes in the graphs are due to the interaction with the artificial
atmosphere I would like to know more about it.
I am attaching my results.
Warm regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230813/78b8e06d/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rho_maximum.png
Type: image/png
Size: 31775 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230813/78b8e06d/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rho_maximum2.png
Type: image/png
Size: 35798 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230813/78b8e06d/attachment-0003.png>

From zachetie at gmail.com  Sun Aug 13 11:04:56 2023
From: zachetie at gmail.com (Zach Etienne)
Date: Sun, 13 Aug 2023 09:04:56 -0700
Subject: [Users] (no subject)
In-Reply-To: <CA+a7ynMt-8=moWF4W06hao=JftEh5kba43JDLNtL=6wBiAfAvA@mail.gmail.com>
References: <CA+a7ynMt-8=moWF4W06hao=JftEh5kba43JDLNtL=6wBiAfAvA@mail.gmail.com>
Message-ID: <CAP6hNvzFyki5fmCh9sHXiHdttW4ronJsZDTXjMw1Cs5KYbedmg@mail.gmail.com>

Hi Thejas,

The oscillations in the maximum density inside the neutron star are due to
numerical errors. Try increasing the numerical resolution, and you'll find
that they converge away. Numerical errors stem from many causes: the sharp
dropoff of density where the neutron star touches the artificial
atmosphere, uneven resolution (the spherical star is being modeled inside a
cubical grid), reconstruction of hydrodynamic variables, derivatives of the
metric source terms, spacetime evolution errors, time evolution errors,
etc. So while you ask about the "physics" of a single stable neutron star,
the dynamics you are seeing are the result of the approximate "numerical"
algorithms used to model it.

-Zach

*     *     *
Zachariah Etienne
Assoc. Prof. of Physics, U. of Idaho
Adjunct Assoc. Prof. of Physics & Astronomy, West Virginia U.
https://etienneresearch.com
https://blackholesathome.net


On Sun, Aug 13, 2023 at 2:14?AM Thejas A Nair <thejasanair275 at gmail.com>
wrote:

> Hi,
> I would like to know more physics about the single stable neutron star
> solved using the toolkit as part of my master's project. Would you be able
> to throw some light on this?
> And I have a question about the time scales used in this particular
> problem, they are of order 10^-6, usually, neutron stars are highly compact
> objects,  In this problem, the density was of the order 10^-3 why are these
> scales chosen, and this paper said (https://arxiv.org/abs/1111.3344.) the
> spikes in the graphs are due to the interaction with the artificial
> atmosphere I would like to know more about it.
> I am attaching my results.
> Warm regards
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230813/adb6145c/attachment.htm>

From sbrandt at cct.lsu.edu  Mon Aug 14 11:06:07 2023
From: sbrandt at cct.lsu.edu (Steven R. Brandt)
Date: Mon, 14 Aug 2023 11:06:07 -0500
Subject: [Users] Problem installing Einstein Toolkit on Mac
In-Reply-To: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
References: <CAETPfP0NUpRTL4bbj72AcB0hGJ=DexwUsFc1-nR9oiHF3qs=sw@mail.gmail.com>
Message-ID: <792c766c-b6e4-c067-4c2b-d9f2b37fce4f@cct.lsu.edu>

Are you running on the tutorial server or some other machine? It looks 
like you have a bad/missing gfortran.

--Steve

On 8/5/2023 6:49 AM, Rob wrote:
> Hi,
>
> I hope someone can help.
> I have tried to install the Einstein Toolkit but am getting an error 
> in the final part of the process on the Jupyter CactusTutorial:
>
> %%bash
> ./simfactory/bin/sim build -j2 --thornlist 
> thornlists/einsteintoolkit.th <http://einsteintoolkit.th>
>
> Output is:
>
> Using configuration: sim
> Reconfiguring sim
> Writing configuration to: /Users/rob/Cactus/configs/sim/OptionList
> Cactus - version: 4.14.0
> Reconfiguring sim.
> Using configuration options from configure line
>    Setting fds to '3,5 -j --'
> End of options from configure line
> Adding configuration options from '/Users/rob/Cactus/configs/sim/OptionList'...
>    Setting VERSION to '2018-12-13'
>    Setting CPP to 'cpp'
>    Setting CC to 'gcc'
>    Setting CXX to 'g++'
>    Setting FPP to 'cpp'
>    Setting F90 to 'gfortran'
>    Setting CPPFLAGS to ''
>    Setting FPPFLAGS to '-traditional'
>    Setting CFLAGS to '-g -std=gnu99'
>    Setting CXXFLAGS to '-g -std=gnu++11'
>    Setting F90FLAGS to '-g -fcray-pointer -ffixed-line-length-none'
>    Setting LDFLAGS to '-rdynamic'
>    Setting DEBUG to 'no'
>    Setting CPP_DEBUG_FLAGS to ''
>    Setting C_DEBUG_FLAGS to ''
>    Setting CXX_DEBUG_FLAGS to ''
>    Setting OPTIMISE to 'yes'
>    Setting CPP_OPTIMISE_FLAGS to ''
>    Setting C_OPTIMISE_FLAGS to '-O2'
>    Setting CXX_OPTIMISE_FLAGS to '-O2'
>    Setting F90_OPTIMISE_FLAGS to '-O2'
>    Setting PROFILE to 'no'
>    Setting CPP_PROFILE_FLAGS to ''
>    Setting C_PROFILE_FLAGS to '-pg'
>    Setting CXX_PROFILE_FLAGS to '-pg'
>    Setting F90_PROFILE_FLAGS to '-pg'
>    Setting WARN to 'yes'
>    Setting CPP_WARN_FLAGS to '-Wall'
>    Setting C_WARN_FLAGS to '-Wall'
>    Setting CXX_WARN_FLAGS to '-Wall'
>    Setting F90_WARN_FLAGS to '-Wall'
>    Setting OPENMP to 'yes'
>    Setting CPP_OPENMP_FLAGS to '-fopenmp'
>    Setting FPP_OPENMP_FLAGS to '-D_OPENMP'
>    Setting C_OPENMP_FLAGS to '-fopenmp'
>    Setting CXX_OPENMP_FLAGS to '-fopenmp'
>    Setting F90_OPENMP_FLAGS to '-fopenmp'
>    Setting VECTORISE to 'yes'
>    Setting VECTORISE_ALIGNED_ARRAYS to 'no'
>    Setting VECTORISE_INLINE to 'yes'
>    Setting PTHREADS_DIR to 'NO_BUILD'
> End of options from '/Users/rob/Cactus/configs/sim/OptionList'.
> creating cache ./config.cache
> checking host system type... x86_64-apple-darwin22.5.0
> checking for mawk... no
> checking for gawk... no
> checking for nawk... no
> checking for awk... awk
> checking whether make sets ${MAKE}... yes
> checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) works... yes
> checking whether the C compiler (gcc -g -std=gnu99 -rdynamic) is a cross-compiler... no
> checking whether we are using GNU C... yes
> checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) works... yes
> checking whether the C++ compiler (g++ -g -std=gnu++11 -rdynamic) is a cross-compiler... no
> checking whether we are using GNU C++... yes
> checking for nvcc... no
> checking for ranlib... ranlib
> checking for perl... perl
> checking for gtar... no
> checking for tar... tar
> checking for xargs... xargs
> checking for gzip... gzip
> checking for gpatch... no
> checking for patch... patch
> checking for git... git
> checking for ar... ar
> checking for mkdir... mkdir
> checking for bash... /bin/sh
> Unknown MacOS X f90 compiler.
> Please add appropriate information to
> /Users/rob/Cactus/lib/make/known-architectures/darwin
> and send the updated file to CactusMaint
> We will try anyway ...
> checking whether the Fortran compiler (gfortran  ) works... no
> configure: error: installation or configuration problem: Fortran compiler cannot create executables (see configs/sim/config-data/config.log for details).
>
> Error reconfiguring sim-config
> make: *** [sim-config] Error 2
> As you can see I get an error where it does not like the fortran 
> compiler.
> I have tried all of the precursor stuff a number of times using both 
> MacPorts and Homebrew and this does not seem to give any errors when 
> installing.
> I seem to be going round in circles to resolve this, if you are able 
> to help that would be really appreciated.
> Rob
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230814/b3976edd/attachment.htm>

From rhaas at illinois.edu  Mon Aug 14 15:18:02 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Mon, 14 Aug 2023 15:18:02 -0500
Subject: [Users] Agenda for Thursday's Meeting
Message-ID: <E1qVe0o-00ACTI-0v@ekohaes8>

Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers

From max.praveer at gmail.com  Wed Aug 16 13:48:01 2023
From: max.praveer at gmail.com (Praveer Krishna)
Date: Wed, 16 Aug 2023 20:48:01 +0200
Subject: [Users] Problems running BBH example
Message-ID: <CAMSC+KKzj1Q3GMw=UzuhUsTeEJcabMU2a-Ta9OQNu2HGFtkMrA@mail.gmail.com>

Hello,

I've downloaded and configured the Einstein Toolkit on my desktop, but I'm
having an issue when I try to run the BBH example provided here
<https://einsteintoolkit.org/gallery/bbh/index.html> and the BNS simulation
from here <https://einsteintoolkit.org/gallery/bns/index.html>. The job
finishes in an instant, producing the attached error file. (I'm using the
command "simfactory/bin/sim create-submit GW150914_28 --define N 28 --parfile
par/GW150914/GW150914.rpar --procs 24 --walltime 24:00:00")

There seems to be an issue with the 'num_pus' to 'num_cores' ratio in the
"system_topology.cc" file, but the error oddly persists even when I
manually equate them, or cut out line 471 altogether. This error does not
appear when I run the tov_ET example or the HelloWorld one. I'm also able
to run this mini BNS tutorial <https://www.youtube.com/watch?v=3euS5yMBFmI>
without any issues, so I'm a bit baffled. I've attached my .ini file and
the .cc file along with the error file in case it helps.
Please take a look and let me know if I'm missing something obvious.

PS: My PC has 128GB of memory, a 13900K(24C/32T) processor and 1TB of
storage.

Regards,
Praveer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230816/e148270d/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: GW150914_28.err
Type: application/octet-stream
Size: 1655 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230816/e148270d/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Astra.U.ini
Type: application/octet-stream
Size: 1916 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230816/e148270d/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: system_topology.cc
Type: text/x-c++src
Size: 29689 bytes
Desc: not available
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230816/e148270d/attachment-0001.bin>

From rhaas at illinois.edu  Wed Aug 16 17:15:01 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Wed, 16 Aug 2023 17:15:01 -0500
Subject: [Users] Einstein Toolkit Meeting Reminder
Message-ID: <E1qWOn7-00BZRW-2b@ekohaes8>

Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers

From scupp1 at my.apsu.edu  Thu Aug 17 09:31:41 2023
From: scupp1 at my.apsu.edu (Cupp, Samuel D.)
Date: Thu, 17 Aug 2023 14:31:41 +0000
Subject: [Users] Meeting Minutes, August 17, 2023
Message-ID: <DM6PR01MB401158321A8904853F214231D91AA@DM6PR01MB4011.prod.exchangelabs.com>

Chair - Peter
Minute taker - Sam
Attendees - Leo, Steve

ET_2023_11 release
AsterX still needs reviewer
SpacetimeX might need a second reviewer

As a reminder, reviews should be completed by 9/20/23. Please add any questions/comments to the respective tickets on the ticket tracker<https://bitbucket.org/einsteintoolkit/tickets/issues?status=new&status=open>.

Unanswered emails:
"Problems running BBH example" - Steve replied and it was resolved; may not have been reply all

"Issues at installation" - Sam, Leo, and/or Zach will check if anyone replied without a reply all; otherwise, they will reply to this

"Error Estimation for RK45CK" - Peter will work on replying to this

Next week (8/24):
Chair - Leo
Minute Taker - Steve

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230817/2e76c748/attachment.htm>

From cuosakwe at ucalgary.ca  Thu Aug 17 13:57:56 2023
From: cuosakwe at ucalgary.ca (CJ Osakwe)
Date: Thu, 17 Aug 2023 18:57:56 +0000
Subject: [Users] Installing kuibit in Ubuntu 23.04
Message-ID: <YT2PR01MB5228EDBF8183AB43BC0E86E4D21AA@YT2PR01MB5228.CANPRD01.PROD.OUTLOOK.COM>

Hello,

I'm not sure who to talk to about this, but I can't get the command to install Kuibit to work.

I've been using the Einstein Toolkit for a few months now, and I wanted to update everything in light of the recent 'Schwarzschild' release. The toolkit itself was fine, but when I input the shell command to install Kuibit:

pip install --user -U kuibit==1.4.0

I receive the following error message:

error: externally-managed-environment

This environment is externally managed
??> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I should mention that my version of Python no longer recognizes that kuibit is installed, i.e., when I run Python I get the error: ModuleNotFoundError: No module named 'kuibit'. This is what motivated me to update everything in the first place.

Thanks,
CJ Osakwe
Ph.D. candidate, University of Calgary



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230817/da09dbf9/attachment-0001.htm>

From bozzola.gabriele at gmail.com  Thu Aug 17 14:24:41 2023
From: bozzola.gabriele at gmail.com (Gabriele Bozzola)
Date: Thu, 17 Aug 2023 12:24:41 -0700
Subject: [Users] Installing kuibit in Ubuntu 23.04
In-Reply-To: <YT2PR01MB5228EDBF8183AB43BC0E86E4D21AA@YT2PR01MB5228.CANPRD01.PROD.OUTLOOK.COM>
References: <YT2PR01MB5228EDBF8183AB43BC0E86E4D21AA@YT2PR01MB5228.CANPRD01.PROD.OUTLOOK.COM>
Message-ID: <CAMLD_5FH8PoR3+K6R8qjBgxS1FKKLQgVUUjHfSBCinoS_=68BA@mail.gmail.com>

Hi CJ Osakwe,

Ubuntu does not allow you to use pip to install packages outside of virtual
environments. This is a choice that the Ubuntu developers took to reduce
the chances that new packages break the existing ones.

This is not an issue that is specific to kuibit, but it applies to any
python package. There are multiple possible ways around this. I personally
use Poetry (https://python-poetry.org/) because it helps me with managing
the development of the package itself. I am not familiar with the other
solutions to the problem, so I cannot tell you which one is the best. I'd
recommend using your preferred search engine and following the some
instruction you understand. Let us know if you face any issues in doing so.

Best,
Gabriele

On Thu, Aug 17, 2023 at 11:58?AM CJ Osakwe <cuosakwe at ucalgary.ca> wrote:

> Hello,
>
> I'm not sure who to talk to about this, but I can't get the command to
> install Kuibit to work.
>
> I've been using the Einstein Toolkit for a few months now, and I wanted to
> update everything in light of the recent 'Schwarzschild' release. The
> toolkit itself was fine, but when I input the shell command to install
> Kuibit:
>
> *pip install --user -U kuibit==1.4.0*
>
> I receive the following error message:
>
>
> *error: externally-managed-environment *
>
>
> *This environment is externally managed *
>
> *??> To install Python packages system-wide, try apt install *
>
> *    python3-xyz, where xyz is the package you are trying to *
>
> *    install. *
>
>
>
> *    If you wish to install a non-Debian-packaged Python package, *
>
> *    create a virtual environment using python3 -m venv path/to/venv. *
>
> *    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make *
>
> *    sure you have python3-full installed. *
>
>
>
> *    If you wish to install a non-Debian packaged Python application, *
>
> *    it may be easiest to use pipx install xyz, which will manage a *
>
> *    virtual environment for you. Make sure you have pipx installed. *
>
>
>
> *    See /usr/share/doc/python3.11/README.venv for more information. *
>
>
> *note: If you believe this is a mistake, please contact your Python
> installation or OS distribution provider. You can override this, at the
> risk of breaking your Python installation or OS, by passing
> --break-system-packages. *
> *hint: See PEP 668 for the detailed specification.*
>
> I should mention that my version of Python no longer recognizes that
> kuibit is installed, i.e., when I run Python I get the error: *ModuleNotFoundError:
> No module named 'kuibit'*. This is what motivated me to update everything
> in the first place.
>
> Thanks,
> CJ Osakwe
> Ph.D. candidate, University of Calgary
>
>
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230817/a04d5451/attachment.htm>

From nils.vu at black-holes.org  Fri Aug 18 13:21:54 2023
From: nils.vu at black-holes.org (Nils Vu)
Date: Fri, 18 Aug 2023 11:21:54 -0700
Subject: [Users] Registration open for NR community calls in the Fall
Message-ID: <4C53C1A5-D0A5-452C-811A-75964C23EF47@black-holes.org>

Dear Numerical Relativists,

A new season of NR community calls is approaching! Registration is now open to present at one of the calls starting on Sep 11, 2023. Please sign up here:

- Request a slot to present at the NR community calls: https://github.com/sxs-collaboration/nr-community-call/wiki

The NR community call is a monthly meeting to foster exchange between our numerical-relativity groups and codes. We usually have two speakers from different groups give brief (10-15min) talks, often on a technical aspect of the numerical-relativity code that they are working with, presenting what works well and what they are struggling with. The talks are followed by 15min of discussion. The speakers are preferably early-career researchers, so the calls are also a good opportunity to increase your visibility to our community.

As always:

- Feel free to circulate this information within the NR community.
- Please subscribe to our mailing list <https://groups.google.com/u/1/a/black-holes.org/g/nr-community-call/> to receive info on the community calls in the future (or send a short message to nr-community-call+subscribe at black-holes.org <mailto:nr-community-call+subscribe at black-holes.org> to subscribe) and join our Slack <https://join.slack.com/t/generalrelativity/shared_invite/zt-dl4gqb6v-aAuw~post8OOMGCS2i~VkA> for discussions (channel: #nr-community-call <https://generalrelativity.slack.com/archives/C021LLTLK37>).
- Details and schedule in the wiki: https://github.com/sxs-collaboration/nr-community-call/wiki


Best,
Nils

?
Nils Vu, Ph.D. (he/him)
Sherman Fairchild Postdoctoral Fellow in Theoretical Astrophysics
California Institute of Technology

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230818/305d1f0c/attachment.htm>

From rhaas at illinois.edu  Mon Aug 21 15:18:01 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Mon, 21 Aug 2023 15:18:01 -0500
Subject: [Users] Agenda for Thursday's Meeting
Message-ID: <E1qYBLd-00Espw-2p@ekohaes8>

Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers

From hayleyjmacpherson at gmail.com  Wed Aug 23 10:56:19 2023
From: hayleyjmacpherson at gmail.com (Hayley Macpherson)
Date: Wed, 23 Aug 2023 10:56:19 -0500
Subject: [Users] Loss of precision after postintial?
Message-ID: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>

Hi all,

I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms. 

I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up. 

I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.

I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical. 

So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps. 

The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).

Any help is much appreciated!
Best wishes,
Hayley 

----

Hayley Macpherson | NASA Einstein Fellow

Email: hjmacpherson at uchicago.edu
Pronouns: she/her/hers

Office: ERC 479
Kavli Institute for Cosmological Physics
Eckhardt Research Center
5640 South Ellis Avenue
Chicago, IL, 60637

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230823/82782874/attachment-0001.htm>

From schnetter at gmail.com  Wed Aug 23 11:31:37 2023
From: schnetter at gmail.com (Erik Schnetter)
Date: Wed, 23 Aug 2023 12:31:37 -0400
Subject: [Users] Loss of precision after postintial?
In-Reply-To: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
References: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
Message-ID: <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>

Hayley

Are you using mesh refinement in your calculations? If so, the ET
would fill the ghost and the buffer zones with interpolated data.
These are generally less accurate than properly calculated data. If
so, are you ignoring ghost and buffer zones? These should not be used
for visualization and post-processing.

Which metric variables are you accessing? I assume you are looking at
ADMBase for the metric and extrinsic curvature? These should change,
unless you look at them too early: During initialization, we set up
the ADMBase variables, then define the BSSN variables from these, and
then recalculate the ADMBase variables. This would change them
slightly. Analysis tools should only look at the ADMBase variables
after they have been reset.

-erik


On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson
<hayleyjmacpherson at gmail.com> wrote:
>
> Hi all,
>
> I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms.
>
> I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up.
>
> I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.
>
> I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical.
>
> So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps.
>
> The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).
>
> Any help is much appreciated!
> Best wishes,
> Hayley
>
> ----
>
> Hayley Macpherson | NASA Einstein Fellow
>
> Email: hjmacpherson at uchicago.edu
> Pronouns: she/her/hers
>
> Office: ERC 479
> Kavli Institute for Cosmological Physics
> Eckhardt Research Center
> 5640 South Ellis Avenue
> Chicago, IL, 60637
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users



-- 
Erik Schnetter <schnetter at gmail.com>
http://www.perimeterinstitute.ca/personal/eschnetter/

From hayleyjmacpherson at gmail.com  Wed Aug 23 11:37:44 2023
From: hayleyjmacpherson at gmail.com (Hayley Macpherson)
Date: Wed, 23 Aug 2023 11:37:44 -0500
Subject: [Users] Loss of precision after postintial?
In-Reply-To: <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>
References: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
 <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>
Message-ID: <47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com>

Hi Erik,
Thanks for the quick response!

I?m not using mesh refinement, and yes I ignore the ghost zones in my calculations. I use periodicity within the original (no ghosts) grid for derivatives. 

And yes I?m using the ADMBase metric and extrinsic curvature. I get the same result whether I do the calculation in the postpostinitial stage or using the 3D dumps in post-processing. Would there be a loss of precision in ADMBase gij and kij when translating back and forth to BSSN metric/kij, which could then be propagated to the output files? 

Best,
Hayley

> On 23 Aug 2023, at 11:31, Erik Schnetter <schnetter at gmail.com> wrote:
> 
> Hayley
> 
> Are you using mesh refinement in your calculations? If so, the ET
> would fill the ghost and the buffer zones with interpolated data.
> These are generally less accurate than properly calculated data. If
> so, are you ignoring ghost and buffer zones? These should not be used
> for visualization and post-processing.
> 
> Which metric variables are you accessing? I assume you are looking at
> ADMBase for the metric and extrinsic curvature? These should change,
> unless you look at them too early: During initialization, we set up
> the ADMBase variables, then define the BSSN variables from these, and
> then recalculate the ADMBase variables. This would change them
> slightly. Analysis tools should only look at the ADMBase variables
> after they have been reset.
> 
> -erik
> 
> 
> On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson
> <hayleyjmacpherson at gmail.com> wrote:
>> 
>> Hi all,
>> 
>> I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms.
>> 
>> I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up.
>> 
>> I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.
>> 
>> I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical.
>> 
>> So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps.
>> 
>> The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).
>> 
>> Any help is much appreciated!
>> Best wishes,
>> Hayley
>> 
>> ----
>> 
>> Hayley Macpherson | NASA Einstein Fellow
>> 
>> Email: hjmacpherson at uchicago.edu
>> Pronouns: she/her/hers
>> 
>> Office: ERC 479
>> Kavli Institute for Cosmological Physics
>> Eckhardt Research Center
>> 5640 South Ellis Avenue
>> Chicago, IL, 60637
>> 
>> _______________________________________________
>> Users mailing list
>> Users at einsteintoolkit.org
>> http://lists.einsteintoolkit.org/mailman/listinfo/users
> 
> 
> 
> -- 
> Erik Schnetter <schnetter at gmail.com>
> http://www.perimeterinstitute.ca/personal/eschnetter/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230823/2c88d085/attachment.htm>

From rhaas at illinois.edu  Wed Aug 23 17:15:01 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Wed, 23 Aug 2023 17:15:01 -0500
Subject: [Users] Einstein Toolkit Meeting Reminder
Message-ID: <E1qYw7x-00GFRc-2g@ekohaes8>

Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers

From scupp1 at my.apsu.edu  Thu Aug 24 09:43:44 2023
From: scupp1 at my.apsu.edu (Cupp, Samuel D.)
Date: Thu, 24 Aug 2023 14:43:44 +0000
Subject: [Users] Meeting Minutes, August 24, 2023
Message-ID: <DM6PR01MB40110A57E221828A7CA84655D91DA@DM6PR01MB4011.prod.exchangelabs.com>

Chair - Leo
Minutes - Sam
Attendees - Roland Peter Yosef

ET_2023_11 release
SpaceTimeX likely will not be included
Yosef - this means there's no spacetime evolution thorn, greatly reducing what can be done with CarpetX

While true, these thorns are also on Erik's plate, for the most part, and he is busy with CarpetX release. They also haven't been used for a publication or an equivalent quality control. Even if CarpetX is included without a spacetime evolution thorn, having CarpetX in the toolkit will simplify future releases with CarpetX-based codes. Also, if people want to use the Z4c thorn in SpacetimeX, they can still check it out at their own peril.


AsterX also doesn't have a publication, so the reviewer will need to check whether AsterX is actually ready for inclusion. Is the paper just limited by writing, or is something still not working correctly?

Roland - I think there was an issue with TOV star evolution, but I don't know if that was resolved. Bruno hasn't responded in regard to being a reviewer for the AsterX arrangement.

Sam - Samuel Tootle volunteered to review it, though I don't know if one person is enough for the whole arrangement. I'll make sure the reviewer(s) clarify whether AsterX is completely working and the paper is just a matter of writing, or if there's still something being debugged.


Unanswered emails:
"Loss of precision after postintial?"
Con2Prim runs after initial data is set, even with checkpoint/restart. This means that the data can potentially change depending on which evolution thorn is being used. Since finite differences were mentioned, one could also try increasing resolution and see how this affects the precision.


Tickets:
Ticket 2749<https://bitbucket.org/einsteintoolkit/tickets/issues/2749/wvuthorns-sprintf-use-violates-standard> - Leo will address this issue.


Next week's meeting:
Chair - Peter
Minute taker - Leo

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230824/ff962c89/attachment-0001.htm>

From schnetter at gmail.com  Thu Aug 24 12:55:05 2023
From: schnetter at gmail.com (Erik Schnetter)
Date: Thu, 24 Aug 2023 13:55:05 -0400
Subject: [Users] Loss of precision after postintial?
In-Reply-To: <47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com>
References: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
 <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>
 <47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com>
Message-ID: <CADKQjjfq0azMPSu8dfcY=sWeoX+5EodYqhkn+=4T3fm=64RNyQ@mail.gmail.com>

Hayley

No, there should not be any loss of precision. The difference in the
ADMBase variables should be at round-off level.

-erik

On Wed, Aug 23, 2023 at 12:37?PM Hayley Macpherson
<hayleyjmacpherson at gmail.com> wrote:
>
> Hi Erik,
> Thanks for the quick response!
>
> I?m not using mesh refinement, and yes I ignore the ghost zones in my calculations. I use periodicity within the original (no ghosts) grid for derivatives.
>
> And yes I?m using the ADMBase metric and extrinsic curvature. I get the same result whether I do the calculation in the postpostinitial stage or using the 3D dumps in post-processing. Would there be a loss of precision in ADMBase gij and kij when translating back and forth to BSSN metric/kij, which could then be propagated to the output files?
>
> Best,
> Hayley
>
> On 23 Aug 2023, at 11:31, Erik Schnetter <schnetter at gmail.com> wrote:
>
> Hayley
>
> Are you using mesh refinement in your calculations? If so, the ET
> would fill the ghost and the buffer zones with interpolated data.
> These are generally less accurate than properly calculated data. If
> so, are you ignoring ghost and buffer zones? These should not be used
> for visualization and post-processing.
>
> Which metric variables are you accessing? I assume you are looking at
> ADMBase for the metric and extrinsic curvature? These should change,
> unless you look at them too early: During initialization, we set up
> the ADMBase variables, then define the BSSN variables from these, and
> then recalculate the ADMBase variables. This would change them
> slightly. Analysis tools should only look at the ADMBase variables
> after they have been reset.
>
> -erik
>
>
> On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson
> <hayleyjmacpherson at gmail.com> wrote:
>
>
> Hi all,
>
> I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms.
>
> I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up.
>
> I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.
>
> I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical.
>
> So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps.
>
> The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).
>
> Any help is much appreciated!
> Best wishes,
> Hayley
>
> ----
>
> Hayley Macpherson | NASA Einstein Fellow
>
> Email: hjmacpherson at uchicago.edu
> Pronouns: she/her/hers
>
> Office: ERC 479
> Kavli Institute for Cosmological Physics
> Eckhardt Research Center
> 5640 South Ellis Avenue
> Chicago, IL, 60637
>
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>
>
>
> --
> Erik Schnetter <schnetter at gmail.com>
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
>


-- 
Erik Schnetter <schnetter at gmail.com>
http://www.perimeterinstitute.ca/personal/eschnetter/

From rhaas at illinois.edu  Thu Aug 24 15:39:26 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Thu, 24 Aug 2023 16:39:26 -0400
Subject: [Users] Loss of precision after postintial?
In-Reply-To: <CADKQjjfq0azMPSu8dfcY=sWeoX+5EodYqhkn+=4T3fm=64RNyQ@mail.gmail.com>
References: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
 <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>
 <47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com>
 <CADKQjjfq0azMPSu8dfcY=sWeoX+5EodYqhkn+=4T3fm=64RNyQ@mail.gmail.com>
Message-ID: <20230824163926.7ff601f9@fdea4908.spdns.org>

Hello all,

just to be sure: is there a fluid present (and GRHydro)? Then there is
also the issue of con2prim to consider (and Tmunu).

Yours,
Roland

> Hayley
> 
> No, there should not be any loss of precision. The difference in the
> ADMBase variables should be at round-off level.
> 
> -erik
> 
> On Wed, Aug 23, 2023 at 12:37?PM Hayley Macpherson
> <hayleyjmacpherson at gmail.com> wrote:
> >
> > Hi Erik,
> > Thanks for the quick response!
> >
> > I?m not using mesh refinement, and yes I ignore the ghost zones in my calculations. I use periodicity within the original (no ghosts) grid for derivatives.
> >
> > And yes I?m using the ADMBase metric and extrinsic curvature. I get the same result whether I do the calculation in the postpostinitial stage or using the 3D dumps in post-processing. Would there be a loss of precision in ADMBase gij and kij when translating back and forth to BSSN metric/kij, which could then be propagated to the output files?
> >
> > Best,
> > Hayley
> >
> > On 23 Aug 2023, at 11:31, Erik Schnetter <schnetter at gmail.com> wrote:
> >
> > Hayley
> >
> > Are you using mesh refinement in your calculations? If so, the ET
> > would fill the ghost and the buffer zones with interpolated data.
> > These are generally less accurate than properly calculated data. If
> > so, are you ignoring ghost and buffer zones? These should not be used
> > for visualization and post-processing.
> >
> > Which metric variables are you accessing? I assume you are looking at
> > ADMBase for the metric and extrinsic curvature? These should change,
> > unless you look at them too early: During initialization, we set up
> > the ADMBase variables, then define the BSSN variables from these, and
> > then recalculate the ADMBase variables. This would change them
> > slightly. Analysis tools should only look at the ADMBase variables
> > after they have been reset.
> >
> > -erik
> >
> >
> > On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson
> > <hayleyjmacpherson at gmail.com> wrote:
> >
> >
> > Hi all,
> >
> > I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms.
> >
> > I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up.
> >
> > I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.
> >
> > I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical.
> >
> > So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps.
> >
> > The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).
> >
> > Any help is much appreciated!
> > Best wishes,
> > Hayley
> >
> > ----
> >
> > Hayley Macpherson | NASA Einstein Fellow
> >
> > Email: hjmacpherson at uchicago.edu
> > Pronouns: she/her/hers
> >
> > Office: ERC 479
> > Kavli Institute for Cosmological Physics
> > Eckhardt Research Center
> > 5640 South Ellis Avenue
> > Chicago, IL, 60637
> >
> > _______________________________________________
> > Users mailing list
> > Users at einsteintoolkit.org
> > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBagF6wcfg$ 
> >
> >
> >
> >
> > --
> > Erik Schnetter <schnetter at gmail.com>
> > https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBbMw-NSHw$ 
> >
> >  
> 
> 



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230824/8b871fce/attachment.sig>

From rhaas at illinois.edu  Thu Aug 24 15:58:22 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Thu, 24 Aug 2023 16:58:22 -0400
Subject: [Users] Error Estimation for RK45CK
In-Reply-To: <7178F738-7DD0-4E42-924E-BD9FAB1FCC90@uhcl.edu>
References: <191B5CE9-20CA-441B-B876-F3058FB02C48@uhcl.edu>
 <CADKQjjemaUXi2wS7ZqAaADbtpC7QJdw4JancBGV4u2e=nQhkyQ@mail.gmail.com>
 <7178F738-7DD0-4E42-924E-BD9FAB1FCC90@uhcl.edu>
Message-ID: <20230824165822.4f5d1154@fdea4908.spdns.org>

Hello David,

MoL dynamically allocates and deallocates (since its use of
timelevels conflicts with Carpet's requirements for timelevels) storage
for ErrorEstimate in the the MoL_AllocateScratchSpace and
MoL_FreeScratchSpace schedule functions, which are both in
MoL_Evolution.

So the grid variable is not accessible outside of MoL_Evolution.

For debugging and if you know that there is e.g. only a few variable
with an error estimate you can comment out the MoL_FreeScratchSpace
block. A few here means "fewer than there are timelevels declared to
Carpet" since each variable is allocated a timelevel of ErrorEstimate
(and Carpet will abort the run if there are more timelevels in any grid
variable than it can handle).

Yours,
Roland

> Hello,
> 
> I am trying to read the MoL:ErrorEstimate outside of the StepSize.c
> routines.  For some reason it doesn?t work.  Is there something
> special that I need to do to get access to this GridFunction?
> 
> -DG
> 
> --
> David Garrison, Ph.D.
> Associate Dean for the College of Science and Engineering,
> Professor and Founding Chair of Physics
> University of Houston-Clear Lake
> Bayou 3611
> Houston, TX 77058
> 
> Phone: 281-283-3796
> https://urldefense.com/v3/__https://www.uhcl.edu/science-engineering/faculty/garrison-david__;!!DZ3fjg!55rKO2zffKCTddaTep4WnMwSLdzp1OiCJEfvBP1s-J2c79MnJwDQ_J87EbVvAT9whvuDYKysdl69ncMp6jU$ 
> https://urldefense.com/v3/__http://www.uhcl.edu/physics__;!!DZ3fjg!55rKO2zffKCTddaTep4WnMwSLdzp1OiCJEfvBP1s-J2c79MnJwDQ_J87EbVvAT9whvuDYKysdl69e49iuXY$ 
> 
> "If we knew what it was we were doing, it would not be called
> research, would it?" ? Albert Einstein.
> 
> 
> 
> 
> On Jul 26, 2023, at 3:10 PM, Erik Schnetter <schnetter at gmail.com>
> wrote:
> 
> David
> 
> This happens in the file "RK45.c" in line 252. The loop there accesses
> the "gamma_array" with the error estimation coefficients and then
> calls "MoL_LinearCombination" to estimate the error.
> 
> -erik
> 
> On Wed, Jul 26, 2023 at 3:02?PM Garrison, David
> <Garrison at uhcl.edu<mailto:Garrison at uhcl.edu>> wrote:
> 
> Hello,
> 
> Where in the MoL is the error estimates for RK45 calculated?  I can?t
> seem to find that in the source code.  Somehow they are written to
> the MOL::ERRORESTIMATE grid function but I can?t see where that is
> done.
> 
> -DG
> 
> --
> David Garrison, Ph.D.
> Associate Dean for the College of Science and Engineering,
> Professor and Founding Chair of Physics
> University of Houston-Clear Lake
> Bayou 3611
> Houston, TX 77058
> 
> Phone: 281-283-3796
> https://urldefense.com/v3/__https://www.uhcl.edu/science-engineering/faculty/garrison-david__;!!DZ3fjg!55rKO2zffKCTddaTep4WnMwSLdzp1OiCJEfvBP1s-J2c79MnJwDQ_J87EbVvAT9whvuDYKysdl69ncMp6jU$ 
> https://urldefense.com/v3/__http://www.uhcl.edu/physics__;!!DZ3fjg!55rKO2zffKCTddaTep4WnMwSLdzp1OiCJEfvBP1s-J2c79MnJwDQ_J87EbVvAT9whvuDYKysdl69e49iuXY$ 
> 
> "If we knew what it was we were doing, it would not be called
> research, would it?" ? Albert Einstein.
> 
> 
> 
> 
> _______________________________________________
> Users mailing list
> Users at einsteintoolkit.org
> https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!BCR0FSePrR4x!Em13k9KPwXveeqJOOqlFutrEYS-ZUde1eKk2-LVx-O4_SOfK0tMIjXjtuQk_d5z7g1NIZy5WdrSXBljmrA$
> 
> 
> 
> --
> Erik Schnetter <schnetter at gmail.com<mailto:schnetter at gmail.com>>
> https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!BCR0FSePrR4x!Em13k9KPwXveeqJOOqlFutrEYS-ZUde1eKk2-LVx-O4_SOfK0tMIjXjtuQk_d5z7g1NIZy5WdrRu-sGDzw$
> 



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230824/9b46c7dc/attachment-0001.sig>

From rhaas at illinois.edu  Thu Aug 24 16:40:49 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Thu, 24 Aug 2023 17:40:49 -0400
Subject: [Users] Problems running BBH example
In-Reply-To: <CAMSC+KKzj1Q3GMw=UzuhUsTeEJcabMU2a-Ta9OQNu2HGFtkMrA@mail.gmail.com>
References: <CAMSC+KKzj1Q3GMw=UzuhUsTeEJcabMU2a-Ta9OQNu2HGFtkMrA@mail.gmail.com>
Message-ID: <20230824174049.64b6b36d@fdea4908.spdns.org>

Hello Krishna,

The abort is triggered by the SystemTopology / HWLOC thorns having
encountered a processor / core / PU combination that they did not
expect. Is there anything in the stdout file?

I am not quite familiar enough with hwloc in particular to really make
a good suggestion.

You can certainly try and remove 

SystemTopology

and 

hwloc 

from your parameter file's ActiveThorn line(s). This will avoid the
issue but may, depending on your cluster (mis-)configuration, yield
non-ideal speeds (basically you must ensure manually that there is at
most one thread per CPU core). Carpet has an option

Carpet::set_cpu_affinity = "yes"

that probably gives you almost all you would get from SystemTopology so
you can try that one instead.

Can you provide details on the cluster / CPU of the cluster?

Yours,
Roland

> Hello,
> 
> I've downloaded and configured the Einstein Toolkit on my desktop,
> but I'm having an issue when I try to run the BBH example provided
> here
> <https://urldefense.com/v3/__https://einsteintoolkit.org/gallery/bbh/index.html__;!!DZ3fjg!-XFQ4fosXbEkKSRrQrEdwsGzgJCJQRkES951alRCNP9y0etbO9hlBmHws6LOl8wf57_50ErWvtxsYc05tAcu$
> > and the BNS simulation from here
> > <https://urldefense.com/v3/__https://einsteintoolkit.org/gallery/bns/index.html__;!!DZ3fjg!-XFQ4fosXbEkKSRrQrEdwsGzgJCJQRkES951alRCNP9y0etbO9hlBmHws6LOl8wf57_50ErWvtxsYWDkk_Pq$
> > >. The job finishes in an instant, producing the attached error
> > >file. (I'm using the
> command "simfactory/bin/sim create-submit GW150914_28 --define N 28
> --parfile par/GW150914/GW150914.rpar --procs 24 --walltime 24:00:00")
> 
> There seems to be an issue with the 'num_pus' to 'num_cores' ratio in
> the "system_topology.cc" file, but the error oddly persists even when
> I manually equate them, or cut out line 471 altogether. This error
> does not appear when I run the tov_ET example or the HelloWorld one.
> I'm also able to run this mini BNS tutorial
> <https://urldefense.com/v3/__https://www.youtube.com/watch?v=3euS5yMBFmI__;!!DZ3fjg!-XFQ4fosXbEkKSRrQrEdwsGzgJCJQRkES951alRCNP9y0etbO9hlBmHws6LOl8wf57_50ErWvtxsYZtrb0UG$
> > without any issues, so I'm a bit baffled. I've attached my .ini
> > file and the .cc file along with the error file in case it helps.
> Please take a look and let me know if I'm missing something obvious.
> 
> PS: My PC has 128GB of memory, a 13900K(24C/32T) processor and 1TB of
> storage.
> 
> Regards,
> Praveer



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230824/949e241f/attachment.sig>

From hayleyjmacpherson at gmail.com  Fri Aug 25 14:05:24 2023
From: hayleyjmacpherson at gmail.com (Hayley Macpherson)
Date: Fri, 25 Aug 2023 14:05:24 -0500
Subject: [Users] Loss of precision after postintial?
In-Reply-To: <20230824163926.7ff601f9@fdea4908.spdns.org>
References: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
 <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>
 <47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com>
 <CADKQjjfq0azMPSu8dfcY=sWeoX+5EodYqhkn+=4T3fm=64RNyQ@mail.gmail.com>
 <20230824163926.7ff601f9@fdea4908.spdns.org>
Message-ID: <EEB53486-3D7D-4A6D-81DB-6AF5E1A5CB9A@gmail.com>

Hi Roland,

Yes, there is a fluid and I am using GRHydro. But I have checked individual terms, and the matter terms are identical between my checks in HydroBase_Initial and my analysis of the HDF5 output. 

I have pinpointed the difference to be coming from the terms with spatial derivatives of metric quantities, e.g., D_i trK and 3-Ricci tensor. The codes I use for these two calculations use the same routines, so all I can think of now is a possible precision issue somewhere. 

I?ll keep investigating, but if you have any other ideas please let me know!

Best,
Hayley

> On 24 Aug 2023, at 15:39, Roland Haas <rhaas at illinois.edu> wrote:
> 
> Hello all,
> 
> just to be sure: is there a fluid present (and GRHydro)? Then there is
> also the issue of con2prim to consider (and Tmunu).
> 
> Yours,
> Roland
> 
>> Hayley
>> 
>> No, there should not be any loss of precision. The difference in the
>> ADMBase variables should be at round-off level.
>> 
>> -erik
>> 
>> On Wed, Aug 23, 2023 at 12:37?PM Hayley Macpherson
>> <hayleyjmacpherson at gmail.com> wrote:
>>> 
>>> Hi Erik,
>>> Thanks for the quick response!
>>> 
>>> I?m not using mesh refinement, and yes I ignore the ghost zones in my calculations. I use periodicity within the original (no ghosts) grid for derivatives.
>>> 
>>> And yes I?m using the ADMBase metric and extrinsic curvature. I get the same result whether I do the calculation in the postpostinitial stage or using the 3D dumps in post-processing. Would there be a loss of precision in ADMBase gij and kij when translating back and forth to BSSN metric/kij, which could then be propagated to the output files?
>>> 
>>> Best,
>>> Hayley
>>> 
>>> On 23 Aug 2023, at 11:31, Erik Schnetter <schnetter at gmail.com> wrote:
>>> 
>>> Hayley
>>> 
>>> Are you using mesh refinement in your calculations? If so, the ET
>>> would fill the ghost and the buffer zones with interpolated data.
>>> These are generally less accurate than properly calculated data. If
>>> so, are you ignoring ghost and buffer zones? These should not be used
>>> for visualization and post-processing.
>>> 
>>> Which metric variables are you accessing? I assume you are looking at
>>> ADMBase for the metric and extrinsic curvature? These should change,
>>> unless you look at them too early: During initialization, we set up
>>> the ADMBase variables, then define the BSSN variables from these, and
>>> then recalculate the ADMBase variables. This would change them
>>> slightly. Analysis tools should only look at the ADMBase variables
>>> after they have been reset.
>>> 
>>> -erik
>>> 
>>> 
>>> On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson
>>> <hayleyjmacpherson at gmail.com> wrote:
>>> 
>>> 
>>> Hi all,
>>> 
>>> I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms.
>>> 
>>> I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up.
>>> 
>>> I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.
>>> 
>>> I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical.
>>> 
>>> So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps.
>>> 
>>> The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).
>>> 
>>> Any help is much appreciated!
>>> Best wishes,
>>> Hayley
>>> 
>>> ----
>>> 
>>> Hayley Macpherson | NASA Einstein Fellow
>>> 
>>> Email: hjmacpherson at uchicago.edu
>>> Pronouns: she/her/hers
>>> 
>>> Office: ERC 479
>>> Kavli Institute for Cosmological Physics
>>> Eckhardt Research Center
>>> 5640 South Ellis Avenue
>>> Chicago, IL, 60637
>>> 
>>> _______________________________________________
>>> Users mailing list
>>> Users at einsteintoolkit.org
>>> https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBagF6wcfg$ <https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBagF6wcfg$> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Erik Schnetter <schnetter at gmail.com <mailto:schnetter at gmail.com>>
>>> https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBbMw-NSHw$ <https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBbMw-NSHw$> 
>>> 
>>> 
>> 
>> 
> 
> 
> 
> -- 
> My email is as private as my paper mail. I therefore support encrypting
> and signing email messages. Get my PGP key from http://keys.gnupg.net <http://keys.gnupg.net/>.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230825/66bb5169/attachment-0001.htm>

From rhaas at illinois.edu  Mon Aug 28 15:18:01 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Mon, 28 Aug 2023 15:18:01 -0500
Subject: [Users] Agenda for Thursday's Meeting
Message-ID: <E1qaigT-000ahg-0S@ekohaes8>

Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers

From aak00009 at mix.wvu.edu  Wed Aug 30 13:36:41 2023
From: aak00009 at mix.wvu.edu (Anuj Kankani)
Date: Wed, 30 Aug 2023 14:36:41 -0400
Subject: [Users] POWER code issues
Message-ID: <CAOM758MKz7YvN4-iYptx4i_5otjdfWmYb+wcbnhRsJaVJEdx1A@mail.gmail.com>

Hi,

I was recently suggested the POWER (
https://git.ncsa.illinois.edu/elihu/Gravitational_Waveform_Extractor/) code
for some GW extrapolation work I was doing by Gabriele Bozzola but I
believe I found some bugs and he suggested I send an email here.

Running the script as is on the GW150914 gallery example gives me results
significantly different from what the plots on the ET website show. I found
replacing where the code currently fits the strain amplitude and phase to a
polynomial expression with numpy polyfit seems to fix the main problem.

I also noticed in the getCutoffFrequencyFromTwoPuncturesBBH function, lines
341 - 347 have an incorrect equation for angular momentum, it looks like a
typo between position and momentum. Right now its position *  momentum -
momentum^2.

I'm not sure if this code is being actively used, but I saw there were git
branches for the latest ET release.

Thanks,
Anuj Kankani
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230830/1782bdf3/attachment.htm>

From rhaas at illinois.edu  Wed Aug 30 17:15:01 2023
From: rhaas at illinois.edu (rhaas at illinois.edu)
Date: Wed, 30 Aug 2023 17:15:01 -0500
Subject: [Users] Einstein Toolkit Meeting Reminder
Message-ID: <E1qbTSn-000vNX-38@ekohaes8>

Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers

From rhaas at illinois.edu  Thu Aug 31 09:54:08 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Thu, 31 Aug 2023 10:54:08 -0400
Subject: [Users] Loss of precision after postintial?
In-Reply-To: <EEB53486-3D7D-4A6D-81DB-6AF5E1A5CB9A@gmail.com>
References: <D8AD6CD3-8171-4A02-9620-A0D0F3E1CECE@gmail.com>
 <CADKQjjfkjjKtOOCZ5qtBH-spDGS2z6YG6AFRHh0yD1G_dsOkPA@mail.gmail.com>
 <47C899DA-E45A-486E-987B-4F8ADD3E2EE1@gmail.com>
 <CADKQjjfq0azMPSu8dfcY=sWeoX+5EodYqhkn+=4T3fm=64RNyQ@mail.gmail.com>
 <20230824163926.7ff601f9@fdea4908.spdns.org>
 <EEB53486-3D7D-4A6D-81DB-6AF5E1A5CB9A@gmail.com>
Message-ID: <20230831105408.2d3944e2@fdea4908.spdns.org>

Hello Hayley,

Sorry for the delay. I am currently out of office and slow with emails.

GRHydro was really my only guess, and I ma pretty stumped. We discussed
this in the ET call today but could not come up with a good answer.

One question that came up was if the difference you see in say 3-Ricci
is present everywhere or only at say refinement boundaries?

If you have not already done so (if you have, then sorry, I missed
them) could you possible send a (cut down, does not have to include the
analysis code or private codes) parameter file that would show the
problem? Is the issue present only in quantities that are being
re-computed (3-Ricci) by the analysis code or already in quantities
that are in the HDF5 files (gij, kij in ADMBase and gammaij, Aij and
the traces in the BSSN code)?

Yours,
Roland

> Hi Roland,
> 
> Yes, there is a fluid and I am using GRHydro. But I have checked individual terms, and the matter terms are identical between my checks in HydroBase_Initial and my analysis of the HDF5 output. 
> 
> I have pinpointed the difference to be coming from the terms with spatial derivatives of metric quantities, e.g., D_i trK and 3-Ricci tensor. The codes I use for these two calculations use the same routines, so all I can think of now is a possible precision issue somewhere. 
> 
> I?ll keep investigating, but if you have any other ideas please let me know!
> 
> Best,
> Hayley
> 
> > On 24 Aug 2023, at 15:39, Roland Haas <rhaas at illinois.edu> wrote:
> > 
> > Hello all,
> > 
> > just to be sure: is there a fluid present (and GRHydro)? Then there is
> > also the issue of con2prim to consider (and Tmunu).
> > 
> > Yours,
> > Roland
> >   
> >> Hayley
> >> 
> >> No, there should not be any loss of precision. The difference in the
> >> ADMBase variables should be at round-off level.
> >> 
> >> -erik
> >> 
> >> On Wed, Aug 23, 2023 at 12:37?PM Hayley Macpherson
> >> <hayleyjmacpherson at gmail.com> wrote:  
> >>> 
> >>> Hi Erik,
> >>> Thanks for the quick response!
> >>> 
> >>> I?m not using mesh refinement, and yes I ignore the ghost zones in my calculations. I use periodicity within the original (no ghosts) grid for derivatives.
> >>> 
> >>> And yes I?m using the ADMBase metric and extrinsic curvature. I get the same result whether I do the calculation in the postpostinitial stage or using the 3D dumps in post-processing. Would there be a loss of precision in ADMBase gij and kij when translating back and forth to BSSN metric/kij, which could then be propagated to the output files?
> >>> 
> >>> Best,
> >>> Hayley
> >>> 
> >>> On 23 Aug 2023, at 11:31, Erik Schnetter <schnetter at gmail.com> wrote:
> >>> 
> >>> Hayley
> >>> 
> >>> Are you using mesh refinement in your calculations? If so, the ET
> >>> would fill the ghost and the buffer zones with interpolated data.
> >>> These are generally less accurate than properly calculated data. If
> >>> so, are you ignoring ghost and buffer zones? These should not be used
> >>> for visualization and post-processing.
> >>> 
> >>> Which metric variables are you accessing? I assume you are looking at
> >>> ADMBase for the metric and extrinsic curvature? These should change,
> >>> unless you look at them too early: During initialization, we set up
> >>> the ADMBase variables, then define the BSSN variables from these, and
> >>> then recalculate the ADMBase variables. This would change them
> >>> slightly. Analysis tools should only look at the ADMBase variables
> >>> after they have been reset.
> >>> 
> >>> -erik
> >>> 
> >>> 
> >>> On Wed, Aug 23, 2023 at 11:56?AM Hayley Macpherson
> >>> <hayleyjmacpherson at gmail.com> wrote:
> >>> 
> >>> 
> >>> Hi all,
> >>> 
> >>> I?m the author of a cosmological initial data thorn in the ET; FLRWSolver. I?m currently working on improving the initial data to solve the constraint equations exactly (instead of previously using an approximation), for a given metric and Kij. The way this works is to calculate the metric terms on the LHS of both the constraints and solve for the relevant rest mass density and peculiar 3-velocities from the matter terms.
> >>> 
> >>> I have my own routines from a post-processing analysis code to calculate the metric terms, and I?ve incorporated these into the ET for both generating the initial data but also then double checking the constraint violation after the initial data is set up.
> >>> 
> >>> I noticed something strange: my checks immediately after FLRWSolver is called in the ET show the constraints are satisfied essentially to roundoff error (i.e. the momentum constraint violation has magnitude ~ 1.e-15), but when I take the 3D dump of the initial time slice and run this through my analysis code (which uses the same routines as I use to set up initial data), I see the momentum constraint is violated at the ~ 1e-7 level.
> >>> 
> >>> I thought this might be my post processing code, so I added a second call to check the constraints using my routines after the full initial process is finished. The first call which immediately follows my FLRWSolver routine is placed in group HydroBase_Initial, and I added another call in CCTK_POSTINITIAL which gives the same result, however, if I instead schedule this call in POSTPOSTINITIAL I see the momentum constraint violation is identical to what I see when post processing the initial dump, at ~ 1.e-7. I can see the specific terms which are causing this difference are the terms which use finite-difference derivatives (the curvature terms in the momentum constraint), while all others are identical.
> >>> 
> >>> So my question is the following: is there any way that the metric and curvature variables could suffer a loss of precision between the POSTINITIAL and POSTPOSTINITIAL phases of the run? Especially, a loss in precision which is then translated to the 3D dumps.
> >>> 
> >>> The momentum constraint violation I have is satisfactory, but I am trying to pinpoint why this jump happens to make sure it?s not a bug in my separate code somewhere (also to explain why the constraints aren?t satisfied to roundoff level when they should be, by construction of my initial data :) ).
> >>> 
> >>> Any help is much appreciated!
> >>> Best wishes,
> >>> Hayley
> >>> 
> >>> ----
> >>> 
> >>> Hayley Macpherson | NASA Einstein Fellow
> >>> 
> >>> Email: hjmacpherson at uchicago.edu
> >>> Pronouns: she/her/hers
> >>> 
> >>> Office: ERC 479
> >>> Kavli Institute for Cosmological Physics
> >>> Eckhardt Research Center
> >>> 5640 South Ellis Avenue
> >>> Chicago, IL, 60637
> >>> 
> >>> _______________________________________________
> >>> Users mailing list
> >>> Users at einsteintoolkit.org
> >>> https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBagF6wcfg$ <https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBagF6wcfg$> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> --
> >>> Erik Schnetter <schnetter at gmail.com <mailto:schnetter at gmail.com>>
> >>> https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBbMw-NSHw$ <https://urldefense.com/v3/__http://www.perimeterinstitute.ca/personal/eschnetter/__;!!DZ3fjg!6h4G9d13wz3CvnZuXtOUn3T5cMOIpEMAdnA3o2nTka8JLWDxyoWPws-NTWy-99dGdNk6LmRxzBbMw-NSHw$> 
> >>> 
> >>>   
> >> 
> >>   
> > 
> > 
> > 
> > -- 
> > My email is as private as my paper mail. I therefore support encrypting
> > and signing email messages. Get my PGP key from https://urldefense.com/v3/__http://keys.gnupg.net__;!!DZ3fjg!7tqgOCho0o2qv2M3wbGJgaL-UlExqMms_TgqHTqS06axk1z1N7PN7DZ-xRG7RjL4tcMN8BrEgpF0V6guM-_619mXG18$  <https://urldefense.com/v3/__http://keys.gnupg.net/__;!!DZ3fjg!7tqgOCho0o2qv2M3wbGJgaL-UlExqMms_TgqHTqS06axk1z1N7PN7DZ-xRG7RjL4tcMN8BrEgpF0V6guM-_6td95u-M$ >.  
> 



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230831/04675733/attachment.sig>

From rhaas at illinois.edu  Thu Aug 31 09:59:42 2023
From: rhaas at illinois.edu (Roland Haas)
Date: Thu, 31 Aug 2023 10:59:42 -0400
Subject: [Users] POWER code issues
In-Reply-To: <CAOM758MKz7YvN4-iYptx4i_5otjdfWmYb+wcbnhRsJaVJEdx1A@mail.gmail.com>
References: <CAOM758MKz7YvN4-iYptx4i_5otjdfWmYb+wcbnhRsJaVJEdx1A@mail.gmail.com>
Message-ID: <20230831105942.33864b73@fdea4908.spdns.org>

Hello Anuj,

Thank you or your email.

Those sound like bugs indeed. Could you create an ET ticket at 

https://bitbucket.org/einsteintoolkit/tickets/issues/new

and I will try and see if I can incorporate the fixes you suggested
quickly.

Can you provide your replacement fitting code that you refer to in:

> I found
> replacing where the code currently fits the strain amplitude and phase to a
> polynomial expression with numpy polyfit seems to fix the main problem.

Yours,
Roland

> Hi,
> 
> I was recently suggested the POWER (
> https://git.ncsa.illinois.edu/elihu/Gravitational_Waveform_Extractor/) code
> for some GW extrapolation work I was doing by Gabriele Bozzola but I
> believe I found some bugs and he suggested I send an email here.
> 
> Running the script as is on the GW150914 gallery example gives me results
> significantly different from what the plots on the ET website show. I found
> replacing where the code currently fits the strain amplitude and phase to a
> polynomial expression with numpy polyfit seems to fix the main problem.
> 
> I also noticed in the getCutoffFrequencyFromTwoPuncturesBBH function, lines
> 341 - 347 have an incorrect equation for angular momentum, it looks like a
> typo between position and momentum. Right now its position *  momentum -
> momentum^2.
> 
> I'm not sure if this code is being actively used, but I saw there were git
> branches for the latest ET release.
> 
> Thanks,
> Anuj Kankani



-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230831/e9bf0590/attachment-0001.sig>

From wernecklr at gmail.com  Thu Aug 31 10:00:02 2023
From: wernecklr at gmail.com (Leo Rosa Werneck)
Date: Thu, 31 Aug 2023 08:00:02 -0700
Subject: [Users] Meeting minutes for 2023-08-31
Message-ID: <52BD2F5A-9A28-43B8-AE09-6E8F5B336B8B@gmail.com>

Hi all,

Einstein Toolkit Meeting - August 31, 2023

Chair        : Peter
Minute taker : Leo
Present      : Sam Cupp, Yosef Zlochower, Peter Diener, Leo Werneck,
               Zach Etienne, Roland Haas

* Next ET Release

  - Should CarpetX be enabled or disabled by default? Sam believes it would be
    weird to include all of the new thorns that depend on CarpetX and then have
    people do extra work to have access to them. This will force everyone to
    have a compiler that supports C++17, but this is not a particularly high
    bar. Erik also supports enabling it by default.

  - Roland said some modification might be needed for simulation factory to
    account for this. There is also some consideration about the default
    compiler, which would mostly affect the "generic" configuration. The new
    configuration would need to check the system until it finds a
    C++17-compatible compiler, something that is currently not done. Further,
    some older versions of GCC require you to pass additional options to link
    against the correct, "experimental" standard library.

  - Zach shared a useful list of supported C++ features by compiler:
    https://en.cppreference.com/w/cpp/compiler_support/17

  - Roland suggested enabling CarpetX now so that problems can be fixed as they
    appear.

  - Loop control, as it exists, prevents Carpet and CarpetX from existing in the
    same executable. The provided macros are drastically different for the two
    drivers because of the way looping is done in AMReX. One option is to have
    thorns require "Loop control" OR "Loop control X" so that the header file
    can pick the appropriate macros at compile time. This is not yet
    implemented, but Roland will work on it.

  - Peter should review GRHayLHDX, but skip GRHayLMHDX.

* Unanswered Questions on the Mailing List

  - There are bugs in POWER. Roland will look into it.

* Open Tickets:

  - #2754: there are different definitions of magnetic fields in the toolkit,
           differing by a factor of sqrt(4pi). Note that the evolution of GRMHD
           quantities is still consistent, as these factors are accounted for
           when computing the magnetic pressure that is used to compute the
           energy-momentum tensor. As Zach points out, this only affects very
           specific scenarios where direct comparisons between e.g., GRHydro and
           IllinoisGRMHD are performed.

  - In the future, it would be nice do modify the Seed_Magnetic_Fields thorns
    from WVUThorns* to provide a more user friendly way of setting up the
    magnetic fields (for example, the user can specify the ratio of magnetic to
    fluid pressure).

Next week's chair        : Zach
Next week's minute taker : Roland

Cheers,
Leo


------
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leonardo at uidaho.edu <mailto:leonardo at uidaho.edu>
https://leowerneck.github.io <https://leowerneck.github.io/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einsteintoolkit.org/pipermail/users/attachments/20230831/4b4e61a0/attachment.htm>